RAG responses prove their own origins — how the Claude Citations API structurally blocks citation hallucination

Honestly, I didn't expect citation handling in a RAG system to be this much of a headache. At first, I thought adding "you must cite [document title]" to the prompt would be enough. The results were dismal. Some responses included citations, some didn't, and some even confidently cited passages that didn't actually exist. That's when I realized this wasn't simply a matter of prompt quality.

The Citations API that Anthropic released in January 2025 fundamentally changes the approach. Instead of asking the model to produce citations, the API layer guarantees citation validity. Every citation returned must point to an actual location within the documents provided as input. Citation hallucination — where the model fabricates nonexistent passages — is structurally blocked. One thing worth noting upfront: what the Citations API eliminates is strictly "fabricated citations themselves," not the problem of "a bad retrieval step surfacing the wrong passage." That distinction is revisited later.

As of September 2026, it is available on active Claude models including Opus 5, Sonnet 5, and Haiku 4.5.

Why Prompt-Based Citations Break Down

A prompt that says "include sources in your answer" asks the model to do two things simultaneously — generate a good response and comply with a citation format. From the model's perspective, one of the two tends to suffer, and citation accuracy degrades especially when dealing with long documents, because the model cannot verify on its own whether a cited passage actually exists in the document.

The Citations API moves this verification responsibility to the API infrastructure. As the model generates a response, it internally tracks which passages it relied on, and the system pins those locations as character-level offsets. Developers don't need to implement extraction logic themselves; they simply read validated citation metadata from the response object.

Response Structure of the Citations API

The response content array contains text blocks, and citation information arrives via a citations property attached to each text block (based on the official cookbook using_citations.ipynb). The fields in each citation entry are as follows.

The fact that cited_text is not counted as output tokens is meaningful from a cost perspective. Compared to prompt-based approaches that repeat the original text verbatim in the response, token waste is eliminated.

Hands-On Integration

Basic Request Structure

Let's start with the simplest form: supplying two plain text documents as sources and asking a question. For the model ID, it's safer to check the official model list at the time of execution for the latest value. The example below uses snapshot IDs with a date suffix; you can also use aliases (like claude-sonnet-5 without a date).

import anthropic
 
client = anthropic.Anthropic()
 
sources = [
    {
        "type": "document",
        "source": {
            "type": "text",
            "media_type": "text/plain",
            "data": "Contract termination must be notified in writing at least 30 days in advance. "
                    "However, immediate termination is permitted in the event of a material breach by the other party.",
        },
        "title": "Terms of Service v3.2",
        "citations": {"enabled": True},
    },
    {
        "type": "document",
        "source": {
            "type": "text",
            "media_type": "text/plain",
            "data": "Refunds are only available if requested within 7 days of service use, "
                    "and refunds are restricted once a digital content download is complete.",
        },
        "title": "Refund Policy 2025",
        "citations": {"enabled": True},
    },
]
 
response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": sources + [
                {"type": "text", "text": "What are the conditions under which a contract can be terminated immediately?"}
            ],
        }
    ],
)
 
for block in response.content:
    print(block)

The response content array contains text blocks, and each text block has a .citations attribute attached. The response structure confused me at first too — the key insight is that whenever the model produces sentences backed by different sources, it splits them into multiple text blocks, each carrying its own citations.

Printing an actual response looks roughly like this (conceptual example).

TextBlock(
  text='Immediate termination is permitted in the event of a material breach by the other party.',
  citations=[
    Citation(
      type='char_location',
      cited_text='However, immediate termination is permitted in the event of a material breach by the other party.',
      document_index=0,
      document_title='Terms of Service v3.2',
      start_char_index=32,
      end_char_index=71,
    )
  ]
)

Parsing and Rendering Citations

In practice, you need code to parse citations and apply highlights in the UI. This takes the form of iterating over the citations attribute within each text block.

def render_response_with_citations(response):
    results = []
 
    for block in response.content:
        if getattr(block, "type", None) != "text":
            continue
 
        results.append({"kind": "text", "content": block.text})
 
        for citation in (getattr(block, "citations", None) or []):
            results.append({
                "kind": "citation",
                "cited_text": citation.cited_text,
                "document_index": citation.document_index,
                "document_title": getattr(citation, "document_title", None),
                "start_char": citation.start_char_index,
                "end_char": citation.end_char_index,
            })
 
    return results

The key is to access citations as a property of a text block, not to look for them as a separate block type. If you mistakenly add a branch like block.type == "citations", every citation silently disappears — a worst-case scenario. I strongly recommend printing response.content once before building anything.

Having the document_index and offsets makes it possible to link to the exact location in the original document, or highlight the relevant passage in a PDF viewer. This is the decisive difference from prompt-based citation — you can actually use character-level position information.

Using PDF Documents as Sources

In legal, medical, and academic domains, the original source is often a PDF. You can upload it via the Files API and reference it by file_id. For the exact parameter signatures, it's best to check the Anthropic Python SDK reference and the PDF support documentation.

# Conceptual example — actual signatures may vary by SDK version
uploaded = client.beta.files.upload(
    file=open("contract_v3.pdf", "rb"),
)
file_id = uploaded.id
 
pdf_source = {
    "type": "document",
    "source": {
        "type": "file",
        "file_id": file_id,
    },
    "title": "Contract 2026 Revised Edition",
    "citations": {"enabled": True},
}

On Amazon Bedrock, the Citations API and PDF support became generally available together in June 2025, and the same pattern applies.

Integrating with a RAG Pipeline

In a realistic RAG system, you don't have just one document. You pass multiple retrieved chunks as an array of document blocks, and Citations automatically links each claim to the chunk it was drawn from.

For the embedding model, it's worth choosing one from the Voyage AI official model list that fits your domain. As of September 2026, the voyage-3 family is available, with options like voyage-code-3 for code-specific use. The chunk count (K) varies by situation with no single right answer; many cases settle somewhere between 5 and 20 depending on document length, model context budget, and reranker quality. Rather than fixing it at something like "always 8," I recommend tuning it through offline evaluation.

def build_rag_content(query: str, retrieved_chunks: list[dict]) -> list:
    content = []
    for i, chunk in enumerate(retrieved_chunks):
        content.append({
            "type": "document",
            "source": {
                "type": "text",
                "media_type": "text/plain",
                "data": chunk["text"],
            },
            "title": chunk.get("title", f"Document {i + 1}"),
            "citations": {"enabled": True},
        })
 
    content.append({"type": "text", "text": query})
    return content

In production, I strongly recommend always populating the title field. With only a document_index, it's hard to tell which document is which when reading logs, and debugging time increases significantly.

Tradeoffs and Common Mistakes

Feature-by-Feature Comparison

Common Mistakes in Practice

When chunks are cut too small

Citations pin locations only within the provided documents. If chunks are only one or two sentences long, the citation range covers nearly the entire chunk, diluting the meaning of "where specifically did this come from." Chunking in semantically complete paragraph units is more natural.

When the title field is omitted

In logs, document_index: 3 becomes unidentifiable. It sounds minor, but the difference is tangible during production debugging.

When text blocks without citations are discarded

Not every response block has citations. General connective sentences like "Yes, according to what you mentioned..." may have empty citations. If your parser keeps only blocks with citations and discards the rest, the response appears fragmented. The safe approach is to always concatenate the text and layer citations on top as metadata.

Mistaking citations for a standalone block type

As mentioned earlier, citations are the .citations attribute of a text block. If you mistake them for a separate block type and set up the wrong branch condition, all citations are silently ignored while the code runs quietly — and the problem may go undetected for a long time.

Conflating "citation validity" with "answer correctness"

This is the most common misconception. The Citations API only guarantees that a citation the model produces actually exists within the document — it does not guarantee that the citation is the correct basis for the user's question. If retrieval goes wrong and an irrelevant chunk is surfaced, the model will confidently cite passages that genuinely exist within that chunk and produce a wrong answer.

Image Citations Not Supported: How I Worked Around It

The problem I spent the most time on throughout the project was image citations. A contract PDF contains tables, signature blocks, and attached images, and users wanted answers backed by visual elements like "the rate specified in the table on page 3." Since the Citations API only supports text citations, this wasn't covered out of the box.

The compromise I ended up with had two steps. First, during preprocessing, I converted tables and diagrams into captions, markdown tables, and OCR text, then inserted them inline into the document text. This way, Citations can cite that converted text. Second, I maintained a separate index mapping original page images to character offsets, and whenever a citation appeared, the UI displayed it alongside a thumbnail of the corresponding page. It's not perfect, but from the user's perspective it creates a visually connected experience of "which part of the original document this answer is looking at."

I don't know when the Citations API itself will directly address this problem. But I don't see it as a reason to delay adoption now. Text citations alone filter out a substantial amount of real risk, and the image portion can be supplemented by adjacent systems.

References

• Introducing Citations on the Anthropic API — Anthropic Official Blog
• Citations — Claude Platform Docs
• claude-cookbooks/misc/using_citations.ipynb — GitHub
• PDF support — Claude Platform Docs
• Files API — Claude Platform Docs
• About Claude models — Model IDs and Alias Rules
• Voyage AI Embeddings Model List
• Anthropic's new Citations API — Simon Willison
• Citations API and PDF support for Claude models in Amazon Bedrock — AWS Announcement