annex

what

A user describes an AI system. The agent returns the articles that may attach, cited to the text it read them from, and says so plainly when the law does not settle the question.

It does not determine whether a system complies. It finds the text a person needs to read.

question

I wanted to know whether retrieval still helped when a model could read the whole EU AI Act.

The whole Act comes to about 115,000 tokens, which fits inside the 131,072-token window one GPU could hold. That made the usual answer stop being obvious: if a model can read the whole law, is retrieval still worth building?

So rather than assume a vector database belonged in the design, I compared three arms on the same twelve questions: the whole Act in the context window, vector search alone, and vector search followed by a walk along the Act's own cross-references. All three shared the same answer prompt and the same grounding check, so any gap between them came from how each one found the text.

findings

Reading the whole Act found every article the answers needed, which is closer to a definition than a result, since that arm was handed every provision and could not miss one. The best retrieval arm found 0.83 of them.

Retrieval still earned its place on cost and on checking. It sent about a seventh of the tokens, and because it knows exactly which passages it sent, every citation in an answer can be checked by code. At the next corpus size up, the whole-document arm is not an option at all.

The cheapest change measured was not an architecture change. Replacing the embedding model moved every retrieval arm's recall by 0.20 to 0.25, with no new dependency, which is the kind of gain that gets missed by comparing pipelines rather than parts.

Traversal then recovered more linked obligations on top of the better embeddings, lifting recall from 0.61 to 0.83 on the original text. On the amended text it cost precision, because it also surfaces articles that belong to the chain but not to the literal question.

The conclusion is narrower than retrieval not working. Retrieval is not automatically the right answer when the complete, trusted source fits in context.

Arm	Recall, original	Recall, amended	Prompt tokens, original
Whole Act in the prompt	1.00	1.00	1 416 797
Search only	0.61	0.71	51 622
Search plus reference traversal	0.83	0.81	193 575

graph

The Act cites itself in ordinary sentences, phrases like "in accordance with Article 43" sitting inside running prose. Those citations are parsed into several hundred graph edges programmatically rather than inferred by a model.

That is what makes a multi-hop question answerable at all. Asking what follows from a system being high-risk walks from Article 6 to every one of Articles 8-15 and Article 43 in two hops, instead of hoping one retrieved passage happens to hold the whole chain.

refusal

The refusal is a branch in the pipeline rather than error handling. Where the retrieved and traversed text does not settle a question, the agent says so instead of extrapolating.

The clearest case is a threshold regulators have not defined. Whether a later modification to an already-deployed system counts as substantial enough to trigger full high-risk compliance is not settled by the text, so the agent declines to settle it either.

It is also the weakest thing I measured. Three of the twelve questions are ones the text does not settle, and across every arm and both versions the agent refused correctly on 5 of 18 chances. It refused wrongly on 1 of the 54 questions the text does settle.

freshness

A Digital Omnibus amendment in force since 27 July 2026 moved the deadline for standalone high-risk obligations from 2 August 2026 to 2 December 2027, so the Act has two versions and both are indexed.

Querying the unamended corpus returns a confidently wrong deadline. Querying the amended corpus returns the right one, with the citation change visible between them. That is a better argument for keeping a corpus current than any sentence about it.

limits

It names obligations and cites text. The compliance judgment stays with the reader.

The deployed page replays twelve questions recorded on both versions of the text rather than calling a model. The generation model needs 30GB of GPU memory and nothing free hosts that, so asking a new question means running the pipeline locally.