Understanding the Math Behind Modern AI

Vector embeddings are everywhere in AI now. They power RAG systems, semantic search, and recommendation engines. But what are they really? And when should you actually use them?

What Are Vector Embeddings?

A vector embedding is a list of numbers that represents meaning.

Simple example: The word “king” might be represented as:

[0.2, -0.5, 0.8, 0.1, -0.3, ...]

These numbers capture semantic properties of the word. Words with similar meanings have similar vectors.

Why numbers? Computers understand numbers. By converting text to numbers, we can:

• Compare similarity (which documents are related?)
• Find patterns (what topics appear together?)
• Feed data to machine learning models

How Embeddings Are Created

1. Training An embedding model is trained on massive amounts of text to learn relationships between words and concepts.

2. Encoding When you input text, the model converts it to a vector based on what it learned.

3. Storage Vectors are stored in a vector database for fast retrieval.

4. Comparison New queries are encoded the same way, then compared to stored vectors.

The Embedding Space

Imagine a vast space where:

• Similar concepts are close together
• Dissimilar concepts are far apart
• Relationships are preserved

Example relationships in embedding space:

king - man + woman ≈ queen
Paris - France + Germany ≈ Berlin

This is why embeddings work: they capture semantic relationships mathematically.

Why Embeddings Matter

Semantic search: Find documents by meaning, not just keywords.

Similarity matching: “Find products similar to this one” “Recommend articles based on reading history”

Clustering: Group similar items without labels.

Anomaly detection: Identify unusual patterns.

Deduplication: Find duplicate content that uses different words.

Common Embedding Models

General-purpose:

• OpenAI’s text-embedding-3-large
• Google’s text-embedding-004
• Open source: all-MiniLM-L6-v2

Domain-specific:

• Legal documents: specialized legal embeddings
• Medical: biomedical embeddings
• Code: code-specific embeddings

Multimodal:

• CLIP: images and text
• Embeddings that understand both

Building with Embeddings

Step 1: Choose an embedding model General-purpose works for most cases. Consider domain-specific if you have specialized content.

Step 2: Embed your data Convert all documents to vectors. This happens once (or when documents update).

Step 3: Store in vector database Pinecone, Weaviate, Milvus, or others. They’re optimized for similarity search.

Step 4: Embed queries When users search, convert their query to a vector using the same model.

Step 5: Find similar vectors Vector databases return the closest matches.

Step 6: Retrieve and rank Get the actual documents and rank by relevance.

Embedding Pitfalls

Wrong model for the task: Using a general embedding model for specialized legal documents may miss nuances.

Stale embeddings: If documents update but embeddings don’t, you get outdated results.

Dimension mismatch: Different embedding models produce different-sized vectors. You can’t mix them.

Over-reliance on embeddings: Embeddings capture semantic similarity, not factual correctness. Always verify results.

Poor chunking: If documents are split poorly, embeddings won’t capture full context.

Embeddings vs. Other Approaches

Embeddings for semantic similarity: “Find documents about similar topics”

Keyword search for exact matches: “Find documents containing ‘API authentication’”

Hybrid search: Combine both for best results.

Full-text search for phrase matching: “Find exact phrases”

BM25 for relevance ranking: Traditional information retrieval algorithm.

Cost Considerations

Embedding API calls: OpenAI charges per embedding. Large datasets can get expensive.

Vector database: Storage and query costs vary by provider.

Model hosting: Self-hosting open-source models avoids API costs.

Update frequency: Re-embedding all documents when they change adds cost.

When to Use Embeddings

Good use cases:

• Semantic search over documents
• Recommendation systems
• Duplicate detection
• Content clustering
• RAG systems

Consider alternatives when:

• You need exact keyword matching
• Your data is highly structured (use databases)
• Latency is critical (embeddings add compute)
• Your data is constantly changing

The Embedding Checklist

Before building with embeddings:

• Do you need semantic similarity or exact matching?
• Is the embedding model appropriate for your domain?
• Can you afford the embedding API costs?
• Do you have a vector database solution?
• How will you handle document updates?
• Have you tested retrieval quality?
• Can you explain results to users?

Embeddings in Calliope

Calliope handles embeddings for you:

Chat Studio: Automatically embeds documents for semantic search.

AI Lab: Build custom embedding pipelines with your choice of models.

Langflow: Visual embedding pipeline construction.

Vector database integration: Connect to Pinecone, Weaviate, or others.

The Reality

Embeddings are powerful but not magic. They:

• Work best for semantic similarity
• Require good data and chunking
• Need the right model for your domain
• Should be combined with other techniques

They’re a tool in your AI toolkit, not a solution for everything.

Build semantic search with Calliope →