AI context reduction means feeding a coding assistant only the passages it needs instead of whole files. A widely shared tool claims a 98% cut. The number is believable — but reduction always has a price, and the price is usually detail. Here is how we do it, and where their approach beats ours.
How do you cut AI context by 98%?
The trick is less exotic than the headline suggests. A typical coding assistant reads far more than it uses: entire files pulled in because one function inside them mattered, directory listings, whole documents where two paragraphs were relevant. If you index that material ahead of time and hand back only the matching pieces on demand, the volume collapses — and a 90-plus percent reduction stops sounding like marketing. The tool making the rounds builds a local keyword index and serves matching fragments to the assistant instead of letting it read files directly. That is a sound idea and we do a version of it ourselves. The interesting question is never whether you can shrink the context. It is what you dropped, and whether the model needed it.
Why we run two indexes instead of one
Our AI development platform splits documents on their heading structure rather than on a fixed character count, and it refuses to cut in the middle of a fenced code block — a chunk that ends halfway through a function is worse than useless, because it looks complete. Each chunk is stored verbatim. Then we search two ways at once: a lexical ranking that catches exact identifiers, and a vector similarity search that catches paraphrase, where someone described a thing without using its name. The two result lists get fused into one ranking. Keyword search alone misses “how do we handle expired sessions” when the code says refresh_token. Vector search alone fumbles exact symbol lookups. Running both and merging costs very little and removes an entire class of silent retrieval misses.
The other rule we hold to: return the stored chunk, not a summary of it. A summary is a second inference between the source and the model, and it is exactly where a subtle detail — an off-by-one, a negated condition, an error branch — quietly disappears. Keeping the retrieval layer current and lossless matters more than making it clever.
The part where their approach beats ours
Now the uncomfortable bit, because a comparison where the author wins everything is not worth reading. On one axis their design is genuinely better than ours: it is lossless. It narrows what the assistant sees without rewriting any of it, so nothing is ever silently paraphrased away. One of our own compression paths is lossy — under pressure it can drop the middle of a long function body while keeping the signature and the ends. In the common case that is fine, and it only applies on our secondary execution path. But “usually fine” is not the same as correct, and when we ran this comparison we logged it internally as a quality issue rather than talking ourselves out of it. Finding out a competitor is right about something is the actual return on doing these reviews.
The idea we did not take, and why
The same tool leans on intercepting the assistant’s tool calls as they happen — hooks that fire before a file gets read and swap in a leaner payload. Elegant, and a poor fit for us. That design assumes you are running inside the assistant. We sit outside it: our platform is the orchestrator that launches a mature agentic coding CLI headlessly, hands it a task, and collects the result. There is no interactive session for a hook to attach to, so the same job has to be done earlier, when we assemble the prompt. This is a genuine architectural fork rather than a shortcoming on either side — where you sit in the stack decides which techniques are even available to you, which is a lesson we keep relearning.
Fewer tokens, or the right tokens?
Context reduction gets marketed as a cost story, and it is one — smaller prompts are cheaper prompts. But the reason we care is accuracy. A model handed forty files and one relevant function has to find the function first, and sometimes it does not; it anchors on something adjacent and confidently solves the wrong problem. Trimming context is really about raising the signal-to-noise ratio, with the bill as a side effect. That distinction matters when you choose a technique, because the cheapest possible context and the most useful possible context are not the same target, and optimising purely for the bill has its own failure mode. If a 98% cut removes the one paragraph that explained the edge case, you did not save money. You bought a bug at a discount.
Frequently asked questions
What is AI context reduction?
It is the practice of giving a language model only the passages relevant to the current task rather than whole files or documents. It is usually done by indexing the material in advance and retrieving matching fragments on demand, which lowers token cost and, more importantly, reduces the noise the model has to filter before it can work.
Is a 98% context reduction claim realistic?
Yes, for the right baseline. If the comparison is against reading entire files when only a few functions matter, cutting well over 90% is arithmetic rather than magic. The figure to interrogate is not the percentage but the method: whether the retained content is returned verbatim or summarised, and what happens when retrieval misses.
Does reducing context hurt code quality?
It can, if the reduction is lossy or the retrieval is imprecise. Dropping the middle of a function, or summarising a passage instead of quoting it, can remove the exact detail the model needed. Done carefully — verbatim chunks, structure-aware splitting, more than one retrieval strategy — accuracy generally improves, because there is less noise to get lost in.
Should you use keyword search or vector search for code retrieval?
Both. Keyword ranking wins on exact identifiers and error strings; vector similarity wins when the question is phrased differently from the code. Each misses cases the other catches, and merging the two ranked lists is cheap. Choosing only one is the most common avoidable cause of a retrieval miss.
