Search

Lucene Vector Quantization: What You May Not Expect

Hi there,

We are back with another useful ‘Tips & Tricks’ blog post.

This time, we are diving into vector quantization and, most importantly, some of the hidden implications you might not expect when using Lucene’s implementation.

Vector quantization is a compression technique used to represent high-dimensional vectors (like embeddings) with fewer bits, while trying to preserve as much information as possible.
Lucene introduced scalar quantization in version 9.9 and has continued to evolve its vector quantization support since then, including binary quantization and optimisations.

You might think that quantizing your vectors simply means replacing those big full-precision vectors with a smaller, more efficient quantized representation. Well… not quite! There is a little more happening behind the scenes, and we are here to uncover it and explain, in a simple way, what all of this actually means for your index and your searches.

Lucene Implementation

Enabling quantization in Lucene has some implications that may not be immediately obvious.
In fact, Lucene stores both the full-precision (raw) vectors and the quantized vectors. The quantized representation is therefore additional to the original vectors rather than simply replacing them. 

This is important when estimating disk usage because quantization introduces some additional storage overhead: space is needed for the original vectors, the quantized representation, the HNSW graph, and the related metadata.

So, why keep both? Because they play different roles, both at index time and at query time.

Quantized vectors

The quantized vectors are stored in the .veq file, while the metadata required to interpret them is stored in .vemq. [Reference]

Index Time

During indexing (in-memory buffer → flush (segment creation))

  • Each new document’s float vector is buffered in memory and, once the segment holds enough vectors, inserted into the in-memory HNSW graph.
  • Quantization needs statistics over the whole segment, in particular the segment centroid. Those are only known once the segment is complete. So neighbour selection during insertion uses full-precision float similarity.
    In Lucene, the quantized scorer hands off to its non-quantized delegate when it’s given plain float values rather than quantized ones.
  • At flush, Lucene computes the quantization parameters, writes .veq, and writes the graph that was built with floats as it is.

The result is that a freshly flushed segment’s graph edges were chosen with exact distances, but at query time that graph is navigated using quantized distances. That mismatch is harmless.

During merge

  • Lucene first writes the merged quantized vectors, re-quantizing from the raw vectors. The merged graph is then built with the quantized scorer.
  • When Lucene reuses the largest incoming segment’s graph as a starting point instead of rebuilding from scratch, those reused edges keep whatever precision they were originally built with. Only the newly inserted nodes are scored with quantized vectors.

In short: the graph is built with full precision at flush and with quantized vectors at merge.

Query Time

Quantized vectors are used during the HNSW search to quickly compare the query vector with vectors in the graph and compute approximate similarity scores. Their smaller representation reduces the amount of data that must be read and processed during graph traversal and makes vector comparisons faster.

Raw vectors

The raw vectors are stored separately in the .vec file, with their metadata stored in .vemf. [Reference]

Index Time

During Indexing

  • As described above, before a segment is flushed no quantized vectors exist, so the in-memory HNSW graph is built using the raw vectors. At flush, the raw vectors are also used to compute the segment centroid needed for quantization.

During Merge

  • Re-quantization during segment merges (automatic). Every time segments are merged, Lucene re-quantizes every vector from its original full-precision representation, rather than from an already-quantized one, avoiding the accumulation of additional quantization error across successive merges.

Query Time

Lucene keeps them for full-precision rescoring (optional): not enabled by default. For this purpose, Lucene provides RescoreTopNQuery.createFullPrecisionRescorerQuery(…). The typical pattern is to first run an approximate k-NN query using the faster quantized vectors. RescoreTopNQuery then rescores every candidate returned by that query using the original full-precision vectors, and keeps only the best N results.

So, while Lucene keeps the raw vectors available, you only get the potential accuracy benefit of full-precision rescoring if you explicitly use it.

Storage vs Precision

Keeping the raw vectors is therefore a design trade-off between storage and precision. However, if reducing storage is a priority, the raw vectors can be dropped, although Lucene does not currently expose this as a standard configuration option.
In this case, Lucene can reconstruct a float representation by dequantizing the quantized vectors, but of course the original full precision cannot be recovered. [Reference]

What about Apache Solr, Elasticsearch and OpenSearch?

Now that we have seen what happens under the hood in Lucene, let’s take a look at how Lucene-based search engines such as Apache Solr, Elasticsearch, and OpenSearch build on these core mechanisms and extend them with additional capabilities.

Since this is a Tips & Tricks post, we won’t go too deep into the implementation details. Instead, we will only focus on the key differences that are most relevant to what we have discussed so far.

Apache Solr

Solr follows Lucene closely, delegating quantized vector encoding and scoring to Lucene and therefore retaining both the quantized and raw vector representations.
When using a quantized vector field, first-pass k-NN retrieval and scoring operate on the quantized representation; full-precision rescoring is not performed automatically.

Solr provides several ways to re-rank first-pass results through its ReRank Query Parser. In the context of quantization, the most relevant is the vectorSimilarity() function. It computes exact similarity using the original full-precision vectors, but only for the candidate documents selected for re-ranking. To ensure that every candidate returned by the first-pass k-NN search is rescored, it is important that topK matches reRankDocs. This makes it possible to retrieve candidates efficiently using quantized vectors and then refine their ordering using full-precision scores. [Reference]

We have already covered how quantization works in Solr in dedicated blog posts, so if you are curious, please be sure to check them out:
– Scalar Quantization of Dense Vectors in Apache Solr
– Binary Quantization of Dense Vectors in Apache Solr

Elasticsearch

Elasticsearch also keeps the original full-precision vectors alongside their quantized representation. This is explicitly stated in the documentation:

“Quantization will continue to keep the raw float vector values on disk for re-ranking, reindexing, and quantization improvements over the lifetime of the data. This means disk usage will increase by ~25% for int8, ~12.5% for int4, and ~3.1% for bbq due to the overhead of storing the quantized and raw vectors.”

While Elasticsearch builds on Lucene’s vector quantization capabilities, it also adds its own implementations and features on top. Elasticsearch supports different quantization strategies, and provides its own approach to search and rescoring.
For int8 and int4 quantization, can be explicitly configured. With Better Binary Quantization (BBQ), Elasticsearch goes a step further and applies oversampling and full-precision rescoring automatically by default. For more info about it, please check the official documentation:

OpenSearch

OpenSearch is also built on Lucene, so raw vectors are stored together with their quantized representation, which is what makes rescoring against full-precision vectors possible when quantization is used. In this case, whether rescoring is applied automatically depends on the index configuration, including the mode and compression level. For more info about it, please check the official documentation:


Vector quantization is not just a question of storing smaller vectors.
When using Lucene-based vector search, it is important to understand that quantized and raw vectors can coexist in the index, each serving a different purpose during the search process.

Other posts you may find useful

Sign up for our Newsletter

Did you like this post? Don’t forget to subscribe to our Newsletter to stay always updated in the Information Retrieval world!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.