Evidence Based Conclusions

In intelligence analysis, a conclusion without evidence is just an opinion -- and opinions do not survive peer review, congressional oversight, or the harsh light of being wrong. This lesson covers the evidence chain, source evaluation frameworks, how to distinguish facts from analytical judgments, and the costly lessons of what happens when evidence standards slip.

Every analytical judgment sits atop a chain of evidence. If any link in that chain is weak, missing, or fabricated, the entire conclusion is compromised. Understanding this chain -- and making it visible to your reader -- is what separates professional intelligence from punditry.

The Four Links

• Raw Data: The unprocessed information as collected -- a signal intercept, a satellite image, a human source report, an open-source article.

• Processed Information: Raw data that has been translated, decrypted, formatted, or contextualized. The intercept becomes a transcript; the imagery becomes a labeled photograph.

• Analyzed Evidence: Processed information that has been evaluated for reliability and relevance, then integrated with other sources. This is where the analyst adds value.

• Analytical Judgment: The conclusion drawn from the evidence, stated with appropriate estimative language and confidence levels.

Your reader should be able to trace backward from your conclusion through each link. If they cannot, your evidence chain has a gap.

The evidence chain is like a game of telephone, except the stakes are national security and you are not allowed to mumble. Every link must be clear, documented, and auditable.

When reviewing your own work, try the "prove it" test: for each analytical judgment, ask "What is my evidence?" If you cannot point to specific, cited information, the judgment is unsupported and needs either evidence or a caveat.

Not all evidence is created equal. A corroborated signals intelligence intercept carries different weight than a single-source rumor from an untested informant. Source evaluation frameworks give you a systematic way to assess what your evidence is actually worth.

The Admiralty (NATO) Rating System

The most widely used framework in Western intelligence is the Admiralty system (also called the NATO system), which evaluates two dimensions independently:

Source Reliability

• A -- Completely reliable
• B -- Usually reliable
• C -- Fairly reliable
• D -- Not usually reliable
• E -- Unreliable
• F -- Reliability cannot be judged

Information Credibility

• 1 -- Confirmed by other sources
• 2 -- Probably true
• 3 -- Possibly true
• 4 -- Doubtful
• 5 -- Improbable
• 6 -- Truth cannot be judged

A rating of A1 means a completely reliable source providing confirmed information -- the gold standard. A rating of F6 means you know nothing about the source and cannot verify the information -- proceed with extreme caution.

Beyond the Admiralty System

Some agencies use the CARI framework (Credibility, Accuracy, Reliability, and Integrity) or variations tailored to specific intelligence disciplines. The key principle is the same: never treat all sources as equal. A conclusion built on three E5 sources is not "well corroborated" -- it is three bad sources agreeing with each other.

Three unreliable sources confirming each other is not corroboration. It is a book club for rumors.

One of the most critical skills in evidence-based writing is clearly separating what you know from what you think. Facts are observable and verifiable. Judgments are analytical conclusions drawn from facts. Your reader must always be able to tell which is which.

The Language of Facts vs. Judgments

Fact

"Satellite imagery from 12 March shows three mobile launchers at the eastern test facility."

Observable, verifiable, sourced.

Judgment

"The presence of mobile launchers at the test facility indicates the regime is likely preparing for a missile test within 30 days."

Analytical conclusion with estimative language.

Dealing with Conflicting Evidence

Real intelligence rarely points in one direction. When evidence conflicts, you have three obligations:

• Acknowledge the conflict openly. Do not cherry-pick the evidence that supports your preferred conclusion and ignore the rest.

• Explain why you weigh some evidence more heavily. Source reliability, recency, corroboration, and access all factor into this judgment.

• Present alternative interpretations. If the conflicting evidence supports a different conclusion, state it and explain why your primary assessment is stronger.

Use explicit signpost language: "The evidence indicates..." for facts, "We assess that..." for judgments, and "An alternative interpretation is..." for competing hypotheses. This transparency builds trust with your reader.

No discussion of evidence-based intelligence is complete without acknowledging the most consequential evidence failure in modern intelligence history: the 2002 National Intelligence Estimate (NIE) on Iraq's weapons of mass destruction.

What Went Wrong

The Iraq WMD case study is a textbook example of multiple evidence failures converging:

• Source fabrication: Key intelligence came from "Curveball," a single human source whose reporting was fabricated. His information was not independently verified before being used to support high-confidence judgments.

• Confirmation bias: Analysts interpreted ambiguous evidence in ways that confirmed the pre-existing assumption that Iraq possessed WMD, rather than considering alternative explanations.

• Groupthink: Dissenting views within the intelligence community were present but were marginalized in the final product rather than given appropriate weight.

• Analytical leaps: The gap between evidence and conclusion was bridged by assumption rather than data. "We haven't found proof they destroyed their stockpiles" became "they still have stockpiles."

The Lasting Reforms

The Iraq WMD failure led to fundamental reforms in how the intelligence community handles evidence:

• Mandatory inclusion of confidence levels and sourcing transparency in NIEs.
• Formal requirements to present dissenting views and alternative analyses.
• Structured analytic techniques (SATs) to combat cognitive biases.
• The creation of the Office of the Director of National Intelligence (ODNI) to improve coordination.

Treat every piece of evidence as if it will be scrutinized in a post-mortem investigation -- because someday it might be. Document your evidence trail, flag your assumptions, and make your confidence level explicit. Your future self (and your institution's credibility) will thank you.