Jev for RAG: reranking, passage filters, and citation checks

A retrieval-augmented generation system can fetch plausible passages and still give the answering model the wrong context. The candidate may be topically related but irrelevant to the exact query, contradicted by another passage, or carrying an instruction aimed at the model.

Jev can add bounded decisions between retrieval and generation. It can also check whether the final claims are supported by the cited context.

Where Jev fits in a RAG pipeline

A practical pipeline has several stages:

1. Retrieve a broad candidate set with lexical search, vectors, metadata filters, or a combination.
2. Ask narrow semantic questions about each candidate.
3. Keep, flag, or rerank passages in code.
4. Generate an answer from the selected context.
5. Compare important claims with their cited passages.
6. Route uncertain or contradicted claims for correction or review.

The first stage stays cheap and broad. Jev spends semantic evaluation on a smaller set where it can affect answer quality.

Rerank candidates by relevance

TypeSafe’s reranking cookbook starts with BM25 candidates and applies Jev relevance scoring. The published example reports improvements on its legal-query dataset.

Treat those numbers as results from that cookbook, not a guarantee for another corpus. Retrieval quality depends on chunking, candidate generation, query shape, domain language, and the relevance rubric.

A Score question can use ordered criteria such as:

• unrelated
• mentions the topic but does not answer the query
• partially useful
• directly answers the query

The application can sort by expected score, but it should retain the distribution. A passage split between unrelated and directly useful may need inspection rather than a confident middle interpretation.

Filter passages with separate questions

Relevance is only one property. TypeSafe’s RAG passage-classification cookbook separates checks that lead to different actions.

For each passage, ask questions such as:

• How relevant is this passage to the query?
• Does it contradict a premise in the query or another selected source?
• Does it contain instructions directed at the answering model?
• Does it contain sensitive information that should not enter the prompt?

Code can then decide:

• drop irrelevant passages
• preserve and label a useful contradiction
• quarantine likely prompt injection
• redact or block sensitive context

Collapsing these into one quality score loses the reason for the decision. A contradictory passage may be essential evidence, while an injected instruction may be unsafe despite looking relevant.

Verify citations claim by claim

Citation checking only works when the application supplies both the claim and the source context. Jev does not browse for the evidence on its own.

TypeSafe’s citation-check cookbook uses a closed classification such as supported, unsupported, contradicted, or fabricated. The exact labels should match the product’s correction workflow.

A useful state contains:

{
  "claim": "The policy allows refunds within 30 days.",
  "citation": "Customers may request a refund within fourteen days of purchase.",
  "document_title": "Refund policy"
}

The question should ask whether the cited text supports that precise claim. It should not ask whether the claim sounds generally plausible.

For long answers, split compound statements before verification. One sentence can contain a supported date, an unsupported reason, and a conclusion not present in the source.

Use confidence to decide what reaches the user

A low-confidence relevance result might remain in the candidate set if other passages cover the answer. A low-confidence citation check on a consequential claim should stop publication or trigger review.

This is another case where one threshold is inadequate. Set policies by stage and consequence:

Stage	Possible low-confidence response
Candidate reranking	keep lower in the list or retrieve more candidates
Prompt-injection screen	quarantine and inspect
Citation check for ordinary copy	regenerate or remove the claim
Citation check for legal, health, or financial content	require qualified review

Confidence does not replace source authority. A highly confident judgment about an unreliable source still leaves an unreliable source.

Keep retrieval state focused

Jev’s known limitations warn that large state full of irrelevant detail can hurt performance. Send the query, candidate passage, and only the metadata needed for that question.

Do not send an entire document when the pipeline already knows the cited span. If the surrounding context changes the meaning, include enough nearby text to preserve it.

For very large candidate sets, batch independent questions where the API and rate limits allow it. TypeSafe’s parallel-questions cookbook reports a large first-party speed and cost improvement for one document workflow. Reproduce the test with the request sizes and latency budget of the real system.

Evaluate with retrieval metrics and answer outcomes

A good evaluation separates stages:

• recall of the first-stage retriever
• ranking quality after Jev
• rate of useful passages incorrectly dropped
• prompt-injection detection errors
• citation support accuracy
• final answer correctness and citation coverage
• review volume and latency

A reranker cannot recover a relevant document that retrieval never found. A citation checker cannot fix a claim that has no source. Stage-level metrics show where the pipeline needs work.

For agent-wide routing and guardrails, read Jev for AI agents. For primitive selection, see Choice, Score, and Noul.

Sources

• Re-ranking
• Classifying RAG passages
• Double-checking citations
• Parallel questions
• Jev 1.13 jaggedness

FAQ

Can Jev rerank RAG results?

Yes. TypeSafe documents a reranking pattern that scores query-to-passage relevance after a cheaper first-stage retrieval step.

Can Jev verify citations?

Jev can classify whether supplied source context supports, contradicts, or fails to support a claim. It cannot verify a source that the application did not provide.

Does Jev replace embeddings in RAG?

It can supplement or replace parts of retrieval for some workloads, but a common design uses fast lexical or vector retrieval first and Jev for semantic filtering or reranking.