When you build a retrieval system, you probably evaluate it the same way everyone else does: hand a document and a question to a reader, see if the answer improves, score the document accordingly. That's static retrieval utility. But what happens when the "reader" is not a single-pass model — it's an agent that issues multiple queries, reasons across turns, and uses each document to decide what to search next?

A new paper from 2025 shows that static retrieval metrics and causal agent utility are nearly uncorrelated. The Spearman correlation is -0.026 across 23,322 document observations — essentially zero. About a third of the documents an agent reads are causally load-bearing while looking completely useless to a static reader. The researchers call them "bridge documents."

The Core Problem: Static vs. Causal Utility

Most RAG systems are trained and evaluated on a static idea of usefulness. You take a question, retrieve some documents, feed them to a reader model, and check if the answer is better than without retrieval. This works when a document is read in isolation.

But agents don't work like that. An agent issues a query, reads a document, then issues another query based on what it learned. A document might not contain the answer directly, but it might tell the agent what to search for next. Static evaluation misses this entirely.

The researchers measured this gap directly. They used a ReAct-style agent on HotpotQA, replayed 1,000 development questions, and for every document the agent read, they deleted it and re-ran the rest of the trajectory from that point. This gives a counterfactual: what would have happened without this document?

They built a Counterfactual Trajectory Utility (CTU) score from three deltas: final answer quality, next-query retrieval quality, and turn count. When they crossed CTU against Static RAG Utility (SRU), the correlation was essentially zero.

Bridge Documents: What They Are and Why They Matter

Bridge documents are the ones that look useless to a static reader but are critical for the agent. They don't contain the answer, but they contain information that helps the agent formulate better next queries.

Here's a concrete example. Suppose the agent is looking for the director of a film. The first query returns a page about the film that mentions the producer. The page doesn't contain the director (so static utility is low), but the producer's name lets the agent search for "who did [producer] work with on other films" — which leads to the director. The document was a bridge.

In the experiment, bridge documents made up 27.2% of all documents the agent read. The pattern held even when the researchers replaced the reader-based SRU with BM25 and cross-encoder proxies.

The Mechanism: Discriminative Entities

Why do bridge documents work? The researchers found the answer using Observable Entity Relevance (OER), a measure from prior work on entity ranking. They looked at entities that discriminate relevant from non-relevant candidates — entities that appear more often in documents that lead to correct answers.

These discriminative entities appeared in the agent's next query 4.02 times more often than entities found only in non-relevant documents (6.1% versus 1.5%, across 227,139 observations). A bridge document earns its keep by handing the agent a discriminative entity that redirects the search.

This is fundamentally different from how a static reader works. A static reader gets a document and produces an answer in one step. An agent gets a document, formulates a new query, gets the next document, and so on. Optimizing for the static reader ignores the second and subsequent steps.

The Three Deltas: How CTU Is Constructed

The Counterfactual Trajectory Utility score isn't a single number — it combines three deltas, each capturing a different way a document can matter.

The first delta measures final answer quality. After deleting a document from the agent's context, does the agent still produce the correct answer? If removing a document causes the agent to fail the question entirely, that document has high CTU on this axis. Many bridge documents score low here — they don't directly lead to the correct answer, they just redirect the search.

The second delta measures next-query retrieval quality. When the agent issues its next query after reading a document, how relevant are the results? If removing a bridge document causes the agent's next query to land on irrelevant pages, that document has high CTU on this axis. This is where bridge documents shine — they provide the vocabulary and context the agent needs to formulate useful follow-up queries.

The third delta measures trajectory length. Does removing a document make the agent take more turns to reach the answer, or fewer? Some documents are shortcuts: reading them lets the agent find the answer in two turns instead of five. Other documents are detours: they look relevant but lead the agent in the wrong direction, requiring additional queries to recover. The turn-count delta captures this navigational effect.

The key insight is that a document can score high on one delta and low on another. A bridge document might not affect the final answer directly (low delta 1), but it might improve next-query quality (high delta 2) and reduce trajectory length (high delta 3).

Why Static Metrics Dominate Despite This Gap

The disconnect between static evaluation and agent performance isn't accidental — it's historical. Static metrics like MRR and NDCG were developed for ad-hoc retrieval: a user types a query, gets ranked results, picks one. In that setting, document-level relevance is the right unit of evaluation.

When RAG systems emerged around 2020, researchers naturally adopted these same metrics. The retrieval component of RAG was just another retrieval task: given a question, find the right documents. The reading component was separate — you evaluated the reader on answer quality, the retriever on document relevance.

This decomposition worked when RAG was single-step: one retrieval, one reading, one answer. But agentic systems changed the architecture. Now the agent retrieves, reads, decides what to retrieve next, reads again, and so on. The boundaries between retrieval steps and reasoning steps blur. The "right" documents depend on what the agent plans to do next, not just on the original question.

The researchers didn't set out to prove static metrics wrong. They set out to measure the gap, and the data showed it was nearly zero. The implication is stark: optimizing a retriever for MRR in a RAG setting does not transfer to optimizing it for an agentic setting. You can have the best static retriever in the world and still have an agent that fails.

What This Means for Training Retriever Models

Most retriever models — BM25 variants, dense retrievers like ColBERT, fine-tuned embedding models — are trained on static relevance judgments. The training signal is: this document is relevant to this question. But for agents, the training signal should be: this document is useful for the agent's trajectory from this point forward.

The bridge document finding suggests that retrievers trained on static data will systematically underweight documents that provide navigational value. A document that doesn't answer the question but helps the agent formulate a better next query won't get positive training signal, even though it's causally load-bearing.

This doesn't mean static training is useless — bridge documents still need to be retrieved in the first place. But it means that the ranking among retrieved documents will be wrong: the retriever will put answer-containing documents at the top, while bridge documents (which might be more important for the agent's overall trajectory) will be ranked lower.

The practical implication is that agent builders should consider re-ranking retrieved documents based on trajectory utility, not just relevance. One approach: after the agent reads a document, evaluate whether the next query is more or less promising than before. If the next query looks worse, the document was a detour; if better, it was a bridge.

Practical Implications for Agent Builders

If you're building a multi-step retrieval system, optimizing for pass@k, MRR, or NDCG doesn't guarantee your agent will succeed. You need to evaluate documents not by what they say in isolation, but by how they affect agent behavior over time.

This requires counterfactual evaluation: remove a document from the agent's context and check whether the outcome changes. The CTU metric measures three aspects — final answer quality, next-query quality, and trajectory length — but you can start simpler. Just track whether removing a document changes the agent's final answer.

The researchers also found that the bridge document pattern persists across different proxy metrics (BM25, cross-encoder), so it's not an artifact of a specific reader model. The phenomenon is structural: it comes from the difference between single-step and multi-step retrieval.

For production systems, the takeaway is clear: don't assume your retrieval evaluation transfers. If you switch from a single-pass RAG pipeline to an agentic multi-step pipeline, you need new evaluation metrics that capture the causal role of documents in trajectories. Static pass@k will give you a false sense of security — your agent might be failing precisely because your retriever is optimizing for the wrong thing.

Comparison with Existing Approaches

Traditional RAG evaluation assumes documents are independent and additive. Each document contributes some information, and more documents = better answers. This assumption breaks down for agents because documents interact: one document changes what the agent searches for next, which changes what documents it retrieves, which changes the final answer.

Some recent work on adaptive retrieval (like Self-RAG or CRAG) acknowledges that not all documents are useful, but they still evaluate at the document level. The bridge document finding shows that even "useless" documents can be critical — just not in the way you'd expect from static evaluation.

FAQ

Why doesn't static RAG evaluation work for agents?

Static evaluation assumes documents are read in isolation and used immediately. Agents read documents sequentially, and each document affects subsequent queries. A document that doesn't contain the answer might still be critical if it helps the agent formulate a better next query. Static metrics miss this causal effect entirely.

What are bridge documents?

Bridge documents are documents that have low static utility (they don't improve the answer when read in isolation) but high causal utility (removing them changes the agent's trajectory and final answer). In the study, they made up 27.2% of all documents the agent read. They work by providing discriminative entities that appear in the agent's next query.

How should I evaluate retrieval for agents?

Use counterfactual evaluation: for each document the agent reads, remove it and re-run the trajectory. Measure whether the final answer, next query, or trajectory length changes. The CTU metric combines three deltas (answer quality, query quality, turn count), but you can start by just checking if the final answer changes.

Summary

Static retrieval utility and causal agent utility are nearly independent (Spearman ρ = −0.026 on 23,322 observations). Bridge documents — useless statically but critical for agents — make up 27.2% of retrieved documents. They work by providing discriminative entities that appear in the agent's next query 4.02× more often than non-discriminative entities. For developers, this means moving from static metrics to counterfactual evaluation when building agent systems with retrieval.

Research: Bridge Evidence: Static Retrieval Utility Does Not Predict Causal Utility in Multi-Step Agentic Search (arXiv 2607.15253), 2025.