AI Development
Vector Databases
Learn about vector databases — the backbone of modern AI search. Understand embeddings, similarity search, and how to choose between Pinecone, ChromaDB, Weaviate, FAISS, and Qdrant.
What are Vector Databases?
Vector databases are specialized data stores designed to efficiently store, index, and query high-dimensional vector embeddings. They are the backbone of RAG systems, semantic search, recommendation engines, and any AI application that needs to find similar items.
Traditional databases search by exact matches (SQL WHERE clauses). Vector databases search by meaning — finding the most semantically similar items to a query, even if no exact keywords match. This is what makes AI search feel "magical."
Comparison of Popular Vector DBs
| Database | Hosting | Best For | Free Tier |
|---|---|---|---|
| Pinecone | Fully managed | Production RAG | Yes (100K vectors) |
| ChromaDB | Local / embedded | Prototyping | Free (OSS) |
| Weaviate | Self-hosted / cloud | Hybrid search | Free (OSS) |
| Qdrant | Self-hosted / cloud | High performance | Free (OSS) |
| FAISS | In-memory library | Research / large scale | Free (Meta OSS) |
| pgvector | PostgreSQL extension | Existing Postgres users | Free (OSS) |
Code Example — ChromaDB
# pip install chromadb
import chromadb
client = chromadb.Client()
collection = client.create_collection("my_docs")
# Add documents (auto-embedded)
collection.add(
documents=["RAG combines retrieval with generation",
"Vector databases store embeddings",
"LLMs generate text from prompts"],
ids=["doc1", "doc2", "doc3"]
)
# Query by meaning
results = collection.query(
query_texts=["How do AI search systems work?"],
n_results=2
)
print(results["documents"])
# Returns: docs about RAG and vector DBs (semantic match!)Embedding Models
- OpenAI text-embedding-3-small: Best balance of quality and cost. 1536 dimensions.
- OpenAI text-embedding-3-large: Highest quality. 3072 dimensions.
- Cohere embed-v3: Excellent multilingual support.
- BGE / GTE (Open Source): Free, runs locally, competitive quality.
- Sentence Transformers: Hugging Face library with many free models.
