Sixteen Times Smaller

turbovec, Google’s TurboQuant as a vector index in Rust — what rotate-then-round does, what the library adds, what it costs in recall, who says RaBitQ got there first, and how it fares against FAISS on four CPU cores

Author

Behzad Samadi

Published

September 14, 2026

The last two articles in this section ran a language model and then a voice pipeline on a machine in the room, with nothing leaving it. A private assistant that reads your documents needs one more piece: an index of their embeddings that fits in the memory you have. Ten million 1536-dimensional vectors in float32 are 61 GB; ten million documents’ worth of chunks is a small company’s archive. turbovec is a Rust library, with Python bindings, that claims to hold “a 10 million document corpus” in 4 GB instead of 31, with no training step, and to search it faster than FAISS. It is four weeks past its 1.0 release, has 17,000 stars, and sits on top of the most argued-about algorithm of the year.

This article does three things. It explains the method — a random rotation, then rounding each coordinate — in six steps you can hold in your head. It reads the library at the source: what the API promises, what 1.0 committed to, how its own benchmarks were run. And it runs turbovec against FAISS on two real embedding sets, on the same four-core, GPU-less box the previous articles used, because the numbers in a README are the author’s numbers and yours will be different. The dispute between Google’s TurboQuant paper and the earlier RaBitQ work gets its own section, sourced from both sides.

NoteTL;DR
  1. Rotate, then round. TurboQuant multiplies every vector by one fixed random rotation, after which each coordinate has a known bell-shaped distribution whatever the data. Known distribution means the best 4 or 16 buckets can be computed once from the maths, so there is nothing to train: a vector is indexed the moment it is added.
  2. Sixteen times smaller at 2 bits, eight at 4. 1536 dimensions go from 6,144 bytes to 384 or 768. The compression is real and the README’s arithmetic checks; what it buys you in recall depends on the dimension and the bit width, and the honest way to use a 2-bit index is with a float32 rerank of the top few dozen candidates.
  3. It is a flat scan. No graph, no clustering: every query touches every vector. That is why there is no training and why deletes cost a microsecond — and why cost grows linearly with the corpus. At 100,000 vectors it is instant; at 100 million you would put an inverted file or a graph in front of it, which turbovec does not provide.
  4. The library is more than the algorithm. Stable ids with O(1) deletes, incremental crash-safe saves, an allowlist filter evaluated inside the SIMD kernel, drop-in stores for LangChain, LlamaIndex, Haystack and Agno, wheels for every platform, MIT. That is the part a RAG builder actually buys.
  5. TurboQuant is contested, turbovec is careful. The RaBitQ authors’ public reproduction finds RaBitQ matches or beats TurboQuant on recall and speed and says the paper’s timings could not be reproduced; Elastic and Qdrant have each published their own take. turbovec’s README credits RaBitQ for its unbiasing step, benchmarks against a stronger baseline than the paper did, and publishes every result file.

Rotate, then round

Every vector-compression method faces the same problem: the numbers in an embedding are not evenly spread. Some coordinates carry most of the variance, some almost none, and a scalar quantizer that rounds each coordinate to a few bits wastes its buckets on the quiet ones. Product quantization — the FAISS default since 2011 — solves this by learning a codebook from your data with k-means, which is why a FAISS PQ index has a train() step, why it needs a representative sample first, and why a corpus that drifts wants retraining.

TurboQuant’s move is to make the data look the same whatever it is. turbovec’s README lays it out in six steps, quoted and condensed:

  1. Normalize. Strip the length from each vector and keep it as one float; what remains is a unit direction.
  2. Rotate. “Multiply all vectors by the same random orthogonal matrix. After rotation, each coordinate independently follows a Beta distribution that converges to Gaussian N(0, 1/d) in high dimensions. This holds for any input data.” The rotation does not change distances or inner products; it only redistributes the variance so that every coordinate looks like every other.
  3. Calibrate, optionally. At finite dimensions the bell is not quite the ideal one. TQ+ — turbovec’s addition — “fits two scalars per coordinate — a shift and a scale” from about a thousand sample vectors, so the fixed buckets sit where the data actually is. One call, before adding; skip it and you have “plain TurboQuant”.
  4. Round. “Since the distribution is known, we can precompute the optimal way to bucket each coordinate. For 2-bit, that’s 4 buckets; for 4-bit, 16.” The Lloyd–Max boundaries that minimise squared error for a Gaussian are a table in the code, “computed once from the math, not from the data.”
  5. Pack. Two, three or four bits per coordinate. “A 1536-dim vector goes from 6,144 bytes (FP32) to 384 bytes (2-bit). That’s 16x compression.”
  6. Correct the bias. Rounding shrinks vectors slightly, so inner products come out low. turbovec stores one scalar per vector — ||v|| / ⟨u, x̂⟩ — and multiplies each score by it, “turning the inner-product estimator from downward-biased into unbiased at zero search-time cost.” The README credits this step to RaBitQ.

Search rotates the query once into the same frame and scores it against the packed codes with lookup tables that fit in SIMD registers — the FastScan trick FAISS introduced for 4-bit PQ, which turbovec’s README says its x86 kernel “adapts”. The Google paper’s own unbiasing device is different — a one-bit “Quantized JL” sketch of the residual — and turbovec does not use it; it takes the rotation-and-Lloyd–Max core from TurboQuant and the per-vector correction from RaBitQ. Hold on to that detail; it matters in the dispute.

What you should take from the six steps: there is no learned state except the optional thousand-vector calibration, so adding is encoding, and encoding is a rotation and a rounding. And the whole thing is exhaustive. Nothing narrows the search to a region of the corpus. That is a design choice with consequences both ways.

The library, read at the source

turbovec is “a Rust vector index with Python bindings”, MIT-licensed, written by Ryan Codrai, whose GitHub bio reads “Member of Technical Staff at Anthropic”. The repository was created on 26 March 2026 — two days after Google’s blog post on TurboQuant and the day the memory-chip stocks fell on it — and by 14 September had 17,083 stars, 1,466 forks and nineteen PyPI releases, the last of them 1.0.0 on 18 August 2026. Python 3.9 or newer; numpy the only dependency; wheels for macOS arm64, Linux x86_64 and aarch64, and Windows. pip install turbovec took two seconds here and 126 MB with FAISS beside it.

Two index types. TurboQuantIndex is positional: vectors live in slots, swap_remove deletes in O(1) by moving the last one in. IdMapIndex adds your own uint64 ids on top, remove(id) in O(1), and search(..., allowlist=ids). Both take bit_width of 2, 3 or 4 and a dim that “must be a positive multiple of 8 and ≤ 16384” — inferred from the first add if you omit it. Inputs must be contiguous float32; “other dtypes are rejected rather than silently converted”, which is the right call and the first thing that will bite you with an HDF5 full of float64.

from turbovec import IdMapIndex
import numpy as np

idx = IdMapIndex(dim=1536, bit_width=4)
idx.calibrate(sample)                       # ~1024 random rows; optional
idx.add_with_ids(vectors, ids)              # float32 (n, 1536), uint64 (n,)
scores, ids = idx.search(query, k=10)       # query: float32 (nq, 1536)
scores, ids = idx.search(query, k=10, allowlist=allowed_ids)
idx.remove(ids[0, 0])
idx.sync("docs.tvim")                       # incremental, crash-safe

What 1.0 committed to is the file. The changelog: “v7 is what turbovec reads and writes, and a file written by this release will be readable by later ones.” Older files are refused by version rather than misread, a convert tool brings v5 and v6 forward, and anything before v5 “can only be rebuilt from the source vectors” because a rotation change altered every byte. What v7 buys is sync(): “saving an index that has changed writes the rows that changed rather than the whole file”, one fsync, an atomic rename, and a removal that “rides the commit header” instead of rewriting the file.

The allowlist is the feature a RAG builder should look at first. The pattern is two-stage: SQL, BM25, an access-control list or a date range narrows the corpus to candidate ids; the dense search ranks within them. Most vector stores do this by over-fetching and dropping what is not allowed, which fails when the allowed set is small. turbovec’s kernel takes the allowlist itself: “blocks with no allowed slots are short-circuited before any LUT lookup or scoring work,” and you “always get up to k results from the allowed set — no over-fetching, no recall hit on selective filters.” The run below tests that claim.

The framework drop-ins are the second thing. Each replaces the framework’s own in-memory store with the same public surface: LangChain’s InMemoryVectorStore, LlamaIndex’s SimpleVectorStore, Haystack’s InMemoryDocumentStore, Agno’s LanceDb. “Swap the import and keep your pipeline.”

And the README’s own benchmarks are unusually well documented: every number comes with a script in benchmarks/suite/ and a JSON in benchmarks/results/, run on two named 8-vCPU cloud machines (Google Axion for ARM, Sapphire Rapids for x86), against FAISS IndexPQ for recall and IndexPQFastScan for speed with sub-quantizer counts matched to turbovec’s bit rate. The README also says, of its own choice of baseline, that FAISS PQ “is a stronger baseline than the custom u8-LUT PQ in the TurboQuant paper.” Its headline results: recall@1 within a point or two of FAISS PQ either way on the OpenAI sets, and behind it at 2 bits on low-dimensional GloVe; search 3.4× faster than FastScan at 4 bits and about 20–26% faster at 2 bits; single inserts in microseconds against FAISS’s milliseconds, and deletes in a microsecond against FAISS’s remove_ids, which repacks and takes up to a second at 100K.

The argument about who got there first

You cannot write about TurboQuant in 2026 without the other name.

RaBitQ — Jianyang Gao and Cheng Long, SIGMOD 2024, submitted in October 2023 — quantizes a vector to one bit per dimension after a random rotation, with a proven error bound, and was extended to more bits afterwards. It is what Elasticsearch and Lucene ship as BBQ, what Milvus 2.6 ships as IVF_RABITQ, and it is in FAISS, VectorChord, CockroachDB and turbopuffer (search). TurboQuant — Zandieh, Daliri, Hadian and Mirrokni of Google Research, arXiv April 2025, ICLR 2026 — proposes the same rotate-then-quantize shape with a Lloyd–Max scalar quantizer and a one-bit residual correction, proves near-optimal distortion “within a small constant factor”, and reports beating product quantization on recall “while reducing indexing time to virtually zero” (search). Google’s blog post on 24 March 2026 led with the KV-cache result — six times less memory at three bits, up to eight times faster attention on an H100 — and two days later Micron had lost about 14% and Samsung and SK Hynix 5–6% (search).

Gao’s response, in a Medium post that March, named three problems (search): the paper “described RaBitQ as grid-based PQ while omitting the core random rotation step in RaBitQ”; an ICLR reviewer asked for a fuller comparison and the final version moved RaBitQ “into the appendix”; and in January 2025 TurboQuant’s second author had “asked for help debugging his own Python version translated from their RaBitQ C++ implementation.”

Then the NTU group published a reproduction, with code — the one part of this dispute this article could read in full. Its findings, quoted from the repository: “In quantization accuracy, RaBitQ_prod matches or outperforms TurboQuant_prod across all tested bit widths.” “In nearest neighbor search, RaBitQ consistently achieves higher recall than both TurboQuant variants across all datasets and bit widths.” On speed, RaBitQ on the same A100 is 1.2–1.8× faster, and TurboQuant’s released code ran “up to approximately two orders of magnitude slower” than the paper reports, so those timings “could not be reproduced.” On the paper’s RaBitQ timings: “their experiments evaluated RaBitQ on a single-core CPU with multi-threading disabled, while evaluating TurboQuant on an A100 GPU” — attributed to correspondence with the TurboQuant authors. And a finding that cuts the other way for anyone reading the theory: “TurboQuant_mse consistently achieves higher recall than TurboQuant_prod, even though the latter is the variant designed for inner-product estimation.” On KV-cache quality the two methods “have comparable downstream accuracy, with no consistent winner.”

The vector-database vendors took sides in public (search). Elastic: optimized scalar quantization, the algorithm behind BBQ, “beats TurboQuant where production systems care most: throughput, ranking accuracy, and storage efficiency,” with symmetric kernels “10-40x faster” on an M2 Max, while conceding “TurboQuant still wins on raw reconstruction MSE, but that advantage comes mostly from the Hadamard rotation.” Qdrant shipped TurboQuant in 1.18 on 11 May as “an extended version of the algorithm with borrowed RaBitQ ideas”, with “similar recall to Scalar Quantization (SQ), using 2x less memory.”

Where does turbovec sit? Downstream of all of it, and — to its credit — saying so. Its bias correction is RaBitQ’s, and the README names the source. Its baselines are stronger than the paper’s, and the README says that too. It does not claim TurboQuant beats RaBitQ; it does not compare with RaBitQ at all. What it claims is narrower and checkable: this implementation, at these bit widths, against FAISS’s flat quantized indexes, on these machines. Which is what the next section checks on a different machine.

What ran here, on four cores

The machine is the four-core, GPU-less container from the previous articles — an Intel Xeon at 2.1 GHz with AVX-512 VNNI, the instruction set turbovec’s fastest x86 kernel wants — with 15 GB of RAM and no access to Hugging Face. Two real embedding sets came from a public mirror of the ann-benchmarks files: GloVe-100, word vectors at 100 dimensions (the low-dimensional regime the README calls harder), and DBpedia/OpenAI-1536, text-embedding-ada-002 vectors of Wikipedia entities (the regime the pitch is written for). 100,000 vectors and 1,000 queries each, normalised, with exact ground truth recomputed by brute force. Against turbovec at 2, 3 and 4 bits, calibrated and not, stood FAISS 1.15.0: IndexPQ with 256-entry codebooks and IndexPQFastScan with 4-bit codes, each at sub-quantizer counts matched to turbovec’s bit rate as the README does; scalar quantization at 8 and 4 bits; an HNSW graph over float32 for context; and exact float32 as the reference. Search is the median of five passes over the thousand queries, k = 64, one thread; every quantized method also gets a rerank column, where its top 64 are rescored against the float32 vectors, because that is how compressed indexes are used. The script is a hundred lines; the run note has every cell.

DBpedia/OpenAI, 1536 dimensions, 100,000 vectors.

Index Size Build Search, 1 thread Recall@1 Recall@10 After rerank
float32, exact 614 MB — 7.62 ms 1.000 1.000 —
turbovec 4-bit, calibrated 79 MB 2.9 s 0.79 ms 0.970 0.963 1.000
turbovec 4-bit 79 MB 2.7 s 0.77 ms 0.942 0.945 1.000
FAISS PQ, 8-bit codebooks, 4-bit rate 78 MB 87.7 s 33.5 ms 0.936 0.941 1.000
FAISS PQ FastScan, 4-bit rate 77 MB 18.3 s 2.09 ms 0.855 0.880 1.000
FAISS scalar 4-bit 77 MB 0.6 s 22.2 ms 0.864 0.889 —
turbovec 2-bit, calibrated 40 MB 2.8 s 0.98 ms 0.817 0.869 1.000
FAISS PQ, 8-bit codebooks, 2-bit rate 40 MB 52.7 s 16.1 ms 0.780 0.829 1.000
FAISS PQ FastScan, 2-bit rate 39 MB 8.6 s 0.97 ms 0.756 0.806 1.000
FAISS scalar 8-bit 154 MB 0.7 s 17.1 ms 0.987 0.992 —
FAISS HNSW, float32 642 MB 42.5 s 1.05 ms 0.944 0.965 —

GloVe, 100 dimensions (turbovec padded to 104), 100,000 vectors.

Index Size Build Search, 1 thread Recall@1 Recall@10 After rerank
float32, exact 40 MB — 1.51 ms 1.000 1.000 —
turbovec 4-bit, calibrated 5.7 MB 0.1 s 0.14 ms 0.843 0.876 1.000
FAISS PQ, 8-bit codebooks, 4-bit rate 5.1 MB 6.2 s 1.39 ms 0.791 0.852 1.000
FAISS PQ FastScan, 4-bit rate 5.0 MB 1.1 s 0.15 ms 0.741 0.815 0.999
turbovec 2-bit, calibrated 3.1 MB 0.1 s 0.18 ms 0.524 0.595 0.960
FAISS PQ, 8-bit codebooks, 2-bit rate 2.6 MB 3.2 s 1.14 ms 0.462 0.576 0.950
FAISS PQ FastScan, 2-bit rate 2.5 MB 0.5 s 0.10 ms 0.439 0.536 0.920
FAISS scalar 8-bit 10 MB 0.03 s 9.29 ms 0.975 0.981 —
FAISS HNSW, float32 67 MB 9.2 s 0.25 ms 0.944 0.922 —

Recall@1 is how often the true nearest neighbour is the top result; recall@10 is the share of the true top ten in the returned top ten; “after rerank” is recall@10 once the top 64 candidates are rescored in float32.

What the tables say:

  • The compression is exactly as advertised. 614 MB of float32 becomes 79 MB at 4 bits and 40 MB at 2 — 7.8× and 15.5×, the remainder being one scale per vector. The README’s headline “31 GB → 4 GB” is 10 million 768-dimensional vectors at 4 bits; at 1536 dimensions the same corpus is 61 GB in float32, 7.8 GB at 4 bits and 3.9 GB at 2.
  • On the embeddings it was built for, turbovec wins its own comparison. At 1536 dimensions and equal bytes, calibrated 4-bit turbovec beats FAISS PQ on recall (0.970 against 0.936 at @1) and beats it on speed by 42×, because the 256-entry lookup tables of 8-bit PQ do not fit in registers; against FastScan, the register-resident 4-bit PQ that is FAISS’s fast path, it is 2.7× faster and eleven recall points better. At 2 bits it is four points ahead of PQ and level with FastScan on speed. The README’s x86 figure of 3.4× at 4 bits is in the same range on a slower chip.
  • Calibration is worth having. TQ+ added 2.8 points of recall@1 on the OpenAI vectors and about a point on GloVe, for a thousand rows and a tenth of a second. The README says the gain is largest where the data is most skewed; ada-002 vectors are notoriously so.
  • Low dimensions are the hard case, and FastScan fights back. On GloVe-100, 2-bit recall@1 is 0.52 for turbovec — better than either FAISS PQ, but a coin flip — and FastScan is nearly twice as fast at that bit rate. At 4 bits turbovec is ahead on both. The Beta-to-Gaussian argument needs dimensions to work, and 100 is few.
  • The rerank closes every gap that matters. With 64 candidates rescored in float32, every method reaches recall@10 of 1.000 on the OpenAI vectors and 0.92–1.00 on GloVe. A 2-bit turbovec index plus a rerank is a 15× smaller index with the recall of the exact one, at the price of keeping the float32 vectors on disk and reading 64 of them per query.
  • Training time is the quiet win. FAISS PQ spent 53–88 seconds learning codebooks for 100,000 vectors at one thread; FastScan 9–18; HNSW 43. turbovec encoded the same vectors in under three seconds — about 28 µs each — and would take the next 100,000 at the same rate with no retraining.
  • 3-bit is a trap in this version. The 3-bit index was the same 79 MB as 4-bit, slower to search, and less accurate. The API doc explains why: “a byte holds only two codes” at 3 bits, so the packing is 4 bits wide anyway. Use 2 or 4.
  • Scalar quantization without a rotation is what turbovec replaces. FAISS’s 4-bit scalar quantizer at the same 77 MB got 0.864 recall@1 to turbovec’s 0.970, and took 28× longer to search. That gap is the rotation.
  • HNSW is the other design. Over float32 it is fast and accurate and eight times the memory; when it misses, it misses the vector entirely — its recall@1 did not improve from k = 1 to k = 8 — where a quantizer misranks and a rerank recovers. Real systems combine the two, and turbovec offers only the second half.

The allowlist. A 1% allowlist — a thousand allowed ids out of a hundred thousand — with exact ground truth computed over the allowed set alone. turbovec’s kernel filter returned recall@10 of 0.950 on the OpenAI vectors in 0.25 ms per query; fetching the top 1,000 unfiltered results and discarding the disallowed ones, the way most stores filter, got 0.865 in 5.65 ms. On GloVe: 0.899 in 0.05 ms against 0.834 in 5.37 ms. The claim holds: filtering inside the scan is both more accurate and, on a selective filter, much cheaper, because blocks with no allowed vectors are skipped before any scoring.

What did not run: anything at the README’s 10-million scale, so the “faster than FAISS” line is verified at 100,000 and extrapolated by arithmetic beyond it; any ARM machine; RaBitQ itself, whose reference library is C++ and was left for another day; and any GPU.

Four threads. The same run with turbovec and FAISS each allowed all four cores scales everything by three to four and leaves the ratios where they were: on the OpenAI vectors, turbovec 4-bit at 0.20 ms per query against FastScan’s 0.55 (2.7×), 2-bit level at 0.24 against 0.26, the exact scan at 2.1 ms and HNSW at 0.24; on GloVe, FastScan still about twice as fast at the 2-bit rate and level at 4. The allowlist search at 1,536 dimensions took 0.07 ms per query.

What it is not

A flat quantized scan is the simplest possible index, and its simplicity is exactly what the README sells: no training, no parameters, no rebuilds, microsecond adds and deletes, a file you can sync. The cost is that every query scores every vector. At 100,000 vectors that is a few milliseconds and nobody cares. At 10 million — the README’s headline number — a 4 GB scan per query is a different proposition, and the standard answer is to put something in front of the quantizer: an inverted file of a few thousand clusters (FAISS’s IVF, Milvus’s IVF_RABITQ), or a graph (HNSW, which Elastic and Qdrant put in front of their quantizers). turbovec has neither, and its author has not promised one. A paper called IVF-TQ (search) explores exactly that layer.

Nor is it a database. There is no server, no concurrency model beyond a Rust crate’s, no metadata store: the allowlist assumes the metadata lives somewhere else and hands over ids. That is a reasonable division of labour for a local RAG stack — SQLite for the documents, turbovec for the vectors — and the wrong shape for a multi-tenant service, whatever the Snowflake case study (search) managed with it.

And it is not RaBitQ, or Elastic’s BBQ, or Qdrant’s variant — all of which sit inside systems that also give you the graph and the storage. If you already run one of those, turbovec is a way to understand what your database is doing, not a replacement for it.

What to do this week

  • Install it and index what you already have. pip install turbovec, cast your embeddings to float32, IdMapIndex(bit_width=4), calibrate on a thousand random rows, add, sync. It is a fifteen-minute afternoon and you will know what the compression does to your recall — the only number that matters.
  • Use the rerank. Fetch 64 or 100 candidates from the 2-bit index and rescore them against the float32 vectors you kept on disk. Two bits plus a rerank is how every production system uses a quantizer, and it is the configuration where sixteen-fold compression costs almost nothing.
  • Try the allowlist before you build a filter. If your retrieval has a tenant, a date range or a permission check, pass the allowed ids instead of over-fetching. Measure recall against an exact search over the same subset.
  • Keep the ceiling in view. Below a few million vectors a flat scan on a laptop is fine. Above it, you want an IVF or a graph, and today that means a different tool.
  • Read the NTU repository. Whatever your view of the dispute, it is the best public example this year of how to check a paper: same datasets, both methods’ released code, ten seeds, error bars, and a README that says exactly what did not reproduce.

Sources

Research notes — the repository files read, the PyPI records, the run, and the pages that could not be fetched — are in the accompanying folder.