Prompt Chain Drift Forensics

Why this exists

When the last step in a chain gives a bad answer, trace the output back one step at a time. Find where a fact went missing, changed, or first appeared before you edit a prompt.

How to use this protocol

1. Save the original input, each step’s output, and the bad final result.
2. Start with the final step and work back through the steps below.
3. Check what each step got and what it sent on.
4. Match the first break to one of the eight patterns.
5. Use the table at the end to choose whether to fix that step or rebuild the chain.

The forensics workflow

Start by running the bad input three times. Save the model, settings, prompt version, tool results, and where it ran. Check where the runs first differ. Use what you find to pick the next test.

Save what each step got and what it sent on. Use fake data or a log with limited access. Leave out secrets and any personal details the check does not need.

Start at the final wrong output. Ask:

• Was the wrong content invented at this step, or passed through from the previous step?
• If passed through, the fault is upstream. Go back one step.
• If invented at this step, the fault is here. Stop and inspect.

Repeat until you find the step that introduced the error.

At the step where the error first appeared, inspect the input it received. The fault may be in the step itself (the prompt is wrong) or in the boundary (the previous step delivered something this step could not handle).

Use the catalog below. The named pattern tells you what to fix and what not to fix.

Step boundary inspection

At each handoff, check four things:

1. Completeness. Did all expected fields make it across? Missing fields can encourage guessing unless the next step rejects incomplete input or preserves an explicit unknown.
2. Fidelity. Was data reformatted in a way that lost information? "$1.2M revenue" compressed to “large company” loses the number.
3. Provenance. Can you tell what came from the original input versus what was generated by an earlier step? If not, hallucinated facts will be indistinguishable from given facts.
4. Confidence markers. Did hedging language survive? If step one said “Unverified: assumed Q3” and step two rephrased it as “Q3”, the hedge is gone and the chain has laundered uncertainty into a fact.

A handoff that passes all four checks is clean. A handoff that fails any one of them is suspect.

Eight chain-specific failure patterns

Pattern 1 - Compression loss

Detail present in early steps disappears by the final step. Numbers become adjectives. Specific names become categories.

Compare the level of specificity at each handoff. Loss usually happens at one specific step.

A summarization step has no instruction to preserve named entities, numbers, or quotes verbatim.

At the offending step, add: “Preserve all named entities, numerical values, and direct quotations exactly. Do not paraphrase these.”

Pattern 2 - Hallucinated bridge

The final output asserts something not present in the original input. Working backwards, the assertion appears for the first time at one specific step.

The asserted fact does not appear in the input to the originating step but does appear in its output.

The originating step had thin context and was instructed to produce a complete output. It bridged the gap with invention.

Add an explicit gap-handling rule to that step: “If a required field cannot be derived from the provided input, return ‘unknown’ for that field rather than inferring.”

Pattern 3 - Format laundering

A step changes the format of upstream output. Hedging, citations, or structured fields collapse into prose.

Compare the structured fields entering the step versus leaving it. Lost structure means the step rewrote rather than transformed.

The step was given a free-form output instruction without a preservation rule.

Specify which structures must be preserved across the boundary. Use a template the step fills in rather than a free composition.

Pattern 4 - Confidence loss

An early step appropriately hedged a claim. By the final output, the claim is asserted as fact.

The hedging language exists at step N but not at step N+1.

A later step drops words like “may” or “might.” Check whether its prompt tells it to sound sure.

“Preserve all uncertainty markers from the input (‘possibly’, ‘unverified’, ’estimated’). Do not rephrase these into assertions.”

Pattern 5 - Cascade contamination

A small error at step one becomes a large error by step three because each downstream step treats the previous step’s output as ground truth.

The error grows in magnitude across steps rather than persisting at the same level.

No step has a verification or sanity-check rule. Each one trusts the input.

Insert a checkpoint step or add a self-check to one downstream step: “Before continuing, verify the input is internally consistent. If a numerical claim is implausible, flag rather than process.”

Pattern 6 - Schema drift

The chain works most of the time but fails when the upstream step returns a slightly different shape (extra field, missing field, list instead of object).

The failing step receives input that is structurally different from what it was tested with.

The upstream step produces a flexible shape and the downstream step has a rigid expectation.

Either lock the upstream step’s output schema explicitly with an example, or add a normalization step at the boundary.

Pattern 7 - Invisible truncation

The final output reflects only part of the original input. Long inputs produce results as if the input were shorter.

An intermediate step’s input is full-length but its output reflects only the beginning or the end.

The step hit a token or context limit and silently dropped the middle. Models rarely surface this.

Add an explicit completeness check: “Confirm in your response that you addressed every section of the input. If you did not, list what was skipped.”

Pattern 8 - Implicit ordering assumption

The chain works on input formatted one way and fails on equivalent input formatted differently.

Reordering fields, sections, or list items in the original input changes the final output.

A step is implicitly relying on field order, list order, or position in a document. Changing the order breaks the implicit contract.

Make the addressing explicit: “Look for the section labeled ‘Requirements’.” not “Look at the second section.”

Rebuild versus patch matrix

After identifying the originating step, decide whether to patch the prompt at that step or restructure the chain.

Hand-off note

Pair this artifact with Three-Step Prompt Chain Map for chain design and Prompt Chain Validation Checklist for pre-launch verification. The map is the chain you build. The checklist is the chain you ship. The forensics protocol is what you reach for when the chain you shipped is producing wrong output and you need to find the cause before you change anything.

Fictional Example

In this made-up case, a follow-up email adds a rollout the client never named.

The visible failure. Step 3 produced: “As discussed, we will move ahead with a phased rollout starting in October.” Nothing like that appeared in the call.

Step 1 - Reproduce. Three runs against the saved transcript. The phrase appeared in two of three, worded differently each time. Recorded model, prompt versions, and the fact that retrieval is not part of this chain, so external state is ruled out early.

Step 2 - Capture intermediates. Step 1’s extraction and step 2’s selection were both logged. Without them this would have ended as “step 3 is hallucinating, tighten step 3” - which would have been wrong and would have held for about a week.

Step 3 - Walk backwards.

The originating step was 1, three steps away from where the problem was visible.

Step 4 - Inspect the boundary. The quote field read "…start small and see how it goes before committing to everything." That is a real sentence from the transcript. The quote-match validation passed, correctly. What failed was the mapping from that sentence to the phrase “phased rollout” - the model had translated a hedge into a project-management term and then attached an honest citation to it.

Step 5 - Match a pattern. Not compression loss; nothing was dropped. This is the inverse: confidence inflation at the boundary. Step 1’s job is to record what was said, and it recorded what was meant - using vocabulary the client never used. Downstream, “phased rollout” reads as a decided approach, because that is what it means to everyone who reads it.

Fix, at step 1 only. “Use the speaker’s own words in the what field. Do not translate a description into a term of art. If the speaker did not name a thing, do not name it for them.” Step 3, the visible offender, was not touched - it had correctly rendered the input it was given.

Verification. The three saved runs now extract what: "start small before committing to everything", which step 3 renders as a sentence that sounds exactly as tentative as the client did.

How to read this. The bad line showed up at step 3. Reading back through each saved step showed that step 1 turned a guess into a plan. Fix that step first. It kept the quote, but it lost the speaker’s doubt.

One limitation. This check needs what each step wrote. If you don’t have those records, set up logging and wait for the error to happen again. The last answer alone leaves you guessing where it started.