Vector Databases Explained

A vector database is a specialized database designed to store high-dimensional numeric representations of content, called embeddings, and to quickly find the items whose embeddings are most similar to a given query, enabling search based on meaning rather than exact keyword matches.

What an Embedding Actually Is

An embedding is a list of numbers (a vector) produced by an AI model that represents the meaning of a piece of text, image or other content. Content with similar meaning ends up with embeddings that are numerically close together, even if the actual wording is completely different. This is what lets a search for "how do I cancel my subscription" match a document titled "ending your plan," despite sharing almost no keywords.

Why Regular Databases Aren't Built for This

Traditional databases are optimized for exact matches and structured queries, such as finding a row where a customer ID equals a specific value. They are not designed to efficiently answer "which of these million passages is closest in meaning to this new passage," a computation that requires comparing high-dimensional vectors, not exact values. Vector databases are purpose-built for this kind of similarity search at scale, using indexing techniques that make approximate nearest-neighbor search fast even across large collections.

How Vector Databases Fit Into RAG

• Source documents are split into passages and each passage is converted into an embedding.
• Those embeddings are stored in the vector database, often alongside metadata (source document, date, access permissions).
• When a user asks a question, it is embedded the same way, and the vector database returns the passages with the closest matching embeddings.
• Those passages are passed to a language model to generate a grounded answer, the retrieval-augmented generation pattern described in our article What Is RAG?.

Beyond RAG

Vector databases are also used for semantic product search, recommendation systems, and duplicate or similar-content detection, anywhere "find things similar in meaning" is more useful than exact keyword matching.

Frequently Asked Questions

Do I need a vector database if I only have a small number of documents?

For a very small, static set of documents, simpler approaches can sometimes suffice, but a vector database scales cleanly as content grows and is generally worth setting up correctly from the start for any production knowledge assistant.

Can vector search respect existing access permissions?

Yes; metadata filtering alongside similarity search lets a system restrict retrieved results to content a specific user is permitted to see.

How is content kept up to date in a vector database?

Through an ingestion pipeline that re-processes and re-indexes documents when they change, on a schedule appropriate to how frequently the source content is updated.