Vector Embeddings Cheat Sheet

Updated 2026-04-04

Next Topic: Vision-Language Models (VLMs) Cheat Sheet

Vector embeddings transform complex data—text, images, audio—into numerical representations in high-dimensional space where semantic similarity becomes geometric proximity. They power modern AI applications from semantic search to retrieval-augmented generation (RAG), enabling machines to understand meaning rather than just match keywords. The key insight: distance metrics in embedding space directly measure conceptual similarity, allowing algorithms to find related items without exact string matching. Modern advances including multi-vector late interaction, instruction-tuned embeddings, and Matryoshka Representation Learning have pushed retrieval accuracy and efficiency to new heights. Understanding embedding dimensions, distance metrics, indexing strategies, and the full retrieval pipeline is essential for building efficient, production-scale AI systems.

What This Cheat Sheet Covers

This topic spans 14 focused tables and 113 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Core Embedding ConceptsTable 2: Embedding Models and ArchitecturesTable 3: Distance and Similarity MetricsTable 4: Vector Databases and StoresTable 5: Approximate Nearest Neighbor (ANN) AlgorithmsTable 6: Embedding Training and Fine-TuningTable 7: Retrieval Strategies and ApplicationsTable 8: Embedding Optimization TechniquesTable 9: Pooling StrategiesTable 10: Chunking Strategies for RAGTable 11: Sparse Retrieval MethodsTable 12: Indexing and Storage ConsiderationsTable 13: Embedding Evaluation MetricsTable 14: Advanced Embedding Techniques

Table 1: Core Embedding Concepts

Concept	Example	Description
Dense embedding	`[0.23, -0.45, 0.12, ...]` 768 dims	• Fixed-size vector where most values are non-zero • captures semantic meaning in compact, continuous space • produced by neural models like BERT or OpenAI.
Sparse embedding	`{42: 2.3, 103: 1.7, 891: 0.9}`	• Vector where most values are zero • uses explicit term-based features • interpretable dimensions often correspond to vocabulary tokens • models like SPLADE or BM25.
Embedding dimension	76815363072	• Number of elements in the vector • higher dimensions capture finer nuances but increase memory and compute cost • typical ranges: 384–1536 for text, 512–2048 for images.
Semantic similarity	`cosine_sim("king", "queen") = 0.87`	• Degree to which two items share conceptual meaning rather than surface form • measured via distance metrics • embeddings map similarity to proximity.
Vector space	$\mathbb{R}^{1536}$	• High-dimensional geometric space where each axis represents a learned feature • semantically similar items cluster together • structure emerges from training.

Table 1: Core Embedding Concepts

Concept	Example	Description
Dense embedding	`[0.23, -0.45, 0.12, ...]` 768 dims	• Fixed-size vector where most values are non-zero • captures semantic meaning in compact, continuous space • produced by neural models like BERT or OpenAI.
Sparse embedding	`{42: 2.3, 103: 1.7, 891: 0.9}`	• Vector where most values are zero • uses explicit term-based features • interpretable dimensions often correspond to vocabulary tokens • models like SPLADE or BM25.
Embedding dimension	76815363072	• Number of elements in the vector • higher dimensions capture finer nuances but increase memory and compute cost • typical ranges: 384–1536 for text, 512–2048 for images.
Semantic similarity	`cosine_sim("king", "queen") = 0.87`	• Degree to which two items share conceptual meaning rather than surface form • measured via distance metrics • embeddings map similarity to proximity.
Vector space	$\mathbb{R}^{1536}$	• High-dimensional geometric space where each axis represents a learned feature • semantically similar items cluster together • structure emerges from training.