DocumentationNeuronDB Reference
Documentation Branch: You are viewing documentation for the main branch (3.0.0-devel). Select a branch to view its documentation:
Embedding Compatibility Guide
Overview
This guide covers embedding compatibility, storage, and performance. Use it to plan your vector dimensions and storage requirements.
NeuronDB supports vector dimensions from 1 to 16,000 dimensions.
Supported Vector Dimensions
Standard Dimensions
NeuronDB supports vector dimensions from 1 to 16,000 dimensions.
Dimension Limits
- Minimum: 1 dimension
- Maximum: 16,000 dimensions
- Recommended: 128-4096 dimensions for optimal performance
- Performance impact: Higher dimensions = slower queries, more memory
Common Embedding Models
| Model | Dimensions | Use Case |
|---|---|---|
all-MiniLM-L6-v2 | 384 | Fast, general-purpose |
all-mpnet-base-v2 | 768 | Higher quality, general-purpose |
text-embedding-ada-002 | 1536 | OpenAI embeddings |
text-embedding-3-small | 1536 | OpenAI (small) |
text-embedding-3-large | 3072 | OpenAI (large) |
multilingual-e5-base | 768 | Multilingual |
paraphrase-multilingual-mpnet-base-v2 | 768 | Multilingual |
Storage Layout
Vector Type Storage
vector(n) type:
- Storage: 4 bytes per dimension (float32)
- Overhead: 8 bytes header (dimension count + padding)
- Total size:
8 + (n * 4)bytes per vector
Example Sizes
| Dimensions | Size per Vector |
|---|---|
| 128 | 520 bytes |
| 384 | 1,544 bytes |
| 768 | 3,080 bytes |
| 1536 | 6,152 bytes |
TOAST Behavior
PostgreSQL automatically uses TOAST for large values:
- Inline storage: Vectors < 2KB (512 dimensions)
- Extended storage: Vectors ≥ 2KB (512+ dimensions)
- Compression: Enabled by default for extended storage
Memory Behavior
Per-Vector Memory
Storage size:
- On-disk:
8 + (n * 4)bytes - In-memory:
8 + (n * 4)bytes (plus PostgreSQL tuple overhead)
Example for 1M vectors (768 dims):
- On-disk: ~3.08 GB
- In-memory: ~3.08 GB (plus ~10% overhead) = ~3.4 GB
Index Memory
HNSW index:
- Memory: ~3-4x vector data size
- Example: 3.4 GB vectors → ~10-14 GB index memory
IVF index:
- Memory: ~1.5-2x vector data size
- Example: 3.4 GB vectors → ~5-7 GB index memory
Limits and Performance
Hard Limits
| Limit | Value |
|---|---|
| Max dimensions | 16,000 |
| Max vectors per table | Unlimited (billions) |
| Max index size | ~2TB |
| Recommended max batch size | 10,000 |
Performance Thresholds
| Dimensions | Performance | Approx QPS |
|---|---|---|
| < 128 | Very fast | 1,200-1,500 |
| 128-512 | Fast | 800-1,000 |
| 512-1024 | Moderate | 500-700 |
| > 2048 | Slow | 300-400 |
Migration Between Embedding Models
Changing Dimensions
Scenario: Migrating from 384-dim to 768-dim embeddings.
Steps:
- Add new column with new dimension
- Generate new embeddings for existing data
- Rebuild indexes
- Update queries to use new column
- Drop old column when ready
Best Practices
- Plan dimension changes during low-traffic periods
- Test migration on a subset of data first
- Keep old embeddings until new ones are verified
- Monitor query performance after migration