Count-Stable Permission-Aware Retrieval

The problem

Permissioned RAG is usually judged by one question: does it ever return a document the caller may not read? Answer "no" and it passes review.

That is the wrong question, and answering it correctly is not sufficient.

An engineering IC on a 35-document corpus may read 13 of them. They ask, in ordinary words, "redundancy planning next fiscal year". The system ranks all 35, takes the top 5, drops what the caller cannot read, and returns two.

No confidential text was disclosed. And the caller has just learned that three documents about redundancy planning exist and are being kept from them.

Absence is informative. The gap between what was asked for and what came back is a function of how many restricted documents matched — a fact about documents the caller cannot read, handed to them on every request.

What the leak measures

Fifteen queries, none adversarially worded. The leak does not need unusual input, which is what makes it a real problem rather than a curiosity.

The control is what makes it an oracle rather than noise. Total deficit across the same fifteen probes, by seniority:

Monotonically falling as permission rises, exactly zero for someone who reads everything. That is a reliable readout of what is hidden from you, available to anyone who can count.

The mechanism

naive:     rank all 35 → take top 5 → drop unreadable → return 2
                                                        └─ the deficit is the leak

enforced:  restrict to readable → rank those → take top 5 → pad to 5 → return 5
                                                            └─ count carries no information

Two changes, and the second is the one that matters. Filtering before ranking removes most of the deficit. But a principal with only three readable matches still receives three — so comparing counts with a colleague still leaks. Padding to a fixed width is what makes the count constant.

The invariant every test attacks: for any two principals issuing the same query, the observable result count is identical. Only the content differs.

Results

Enforcement improves recall by 7.7%, which was not the prediction — the thesis budgeted for up to 5% loss. Retrieve-then-filter costs quality precisely because restricted documents occupy top-k slots a readable document could have used. The gain tracks permission: the CEO loses nothing either way, the contractor gains most.

The two residual channels

The page template renders the headline limitation under its own "What does not work" heading, so this section is titled for what it actually contains: both surviving channels, in the order they were found.

A timing channel survives at median SNR 0.73, and the direction is counterintuitive — the heavily padded arm is faster, because padding appends pre-sorted documents rather than scoring more candidates. Had I assumed padding would be slower, I would have timed the wrong arm and reported a clean result.

The more serious one was found after all six criteria had passed, by sweeping the deployment. Padded results share no terms with the query. So a principal who receives five results and sees that none of them answer the question knows every document that did match is one they cannot read. That is the full original inference, restored: 15/15 exact, one request, no statistics.

Every earlier benchmark had measured an observer of the count — a colleague comparing notes, a proxy log. But the threat model said the attacker is the principal, and the principal reads the results. So the claim narrowed:

Count-stability defeats an observer of the count. It does not defeat the recipient of the results.

Still worth something: the colleague comparing counts, the access log that records counts but not payloads, the pipeline downstream of the payload. Not protection against the reader.