Threat modelling in practice: facts, hypotheses, conclusions

Most teams have written a threat model at some point. Few still trust the one they have. The document was accurate the week it was written, then the system changed: a queue was added, a role was widened, a vendor was replaced. Nobody could tell which parts of the model those changes invalidated, so the whole thing quietly stopped being read.

The root cause is usually structural. A typical threat model mixes three kinds of statement in the same paragraphs: things the authors observed ("the upload bucket is private"), things they suspected ("an attacker could upload a crafted image"), and things they decided ("we accept this risk until Q3"). A reader cannot tell them apart, and neither can the next author.

Separate the model into three layers, each pointing at the one below.

• Facts are observable statements about the system, each with a source and a date: the endpoint is public; the worker runs as this role; the role can read this bucket. A fact can be checked, and when the system changes, a fact can become false.
• Hypotheses are threats that the facts make possible: what an attacker could do, under which conditions, and what would confirm or refute it. A hypothesis names the facts it depends on.
• Conclusions are decisions: mitigate, accept, transfer or avoid, with an owner, a date and, for accepted risks, an expiry. A conclusion names the hypotheses it answers.

This structure gives you something most threat models lack: a way to know what to update. When a fact changes, every hypothesis that cites it, and every conclusion built on those, is due for review. Nothing else is.

Start with what you can observe. Infrastructure code shows networks, identities and data stores. Application code shows entry points, authorisation checks and outbound calls. Pipeline definitions show who can deploy. Cloud configuration shows what actually exists, which is not always what the code says.

Record each fact with where it came from and when. Just as important, record what you do not know: "we have not established whether the reporting service can reach the payments database" is a fact about your knowledge, and it is worth more than an optimistic assumption. Unknowns are where hypotheses should concentrate.

Adam Shostack's four questions are a reliable frame: what are we working on, what can go wrong, what are we going to do about it, and did we do a good job? The second question is where structure helps most.

• Walk each trust boundary crossing and apply STRIDE: spoofing, tampering, repudiation, information disclosure, denial of service and elevation of privilege.
• For the most valuable assets, build attack trees backwards from the attacker's goal.
• Use known adversary behaviour, such as MITRE ATT&CK techniques, to check that hypotheses reflect what attackers actually do.
• Write abuse cases next to user stories for new features.

A good hypothesis is specific and testable: it names the facts it relies on and what observation would refute it. "Attackers could compromise the system" is not a hypothesis. "A crafted image could exploit the thumbnail library and give an attacker the worker's role, which can read every tenant's uploads" is.

For each hypothesis, decide what evidence would settle it and gather that evidence: check the role's permissions, read the library's advisories, test whether the worker is sandboxed. Then assess likelihood and impact with a method your organisation already uses, such as the approach in NIST SP 800-30, and decide.

Write the conclusion so a stranger can audit it. "Mitigate: run the thumbnail worker in a sandbox with no network access, and scope its role to a per-tenant prefix. Owner: platform team. Done when: the role policy and sandbox configuration are merged." Or, for an accepted risk: who accepted it, why, with which compensating controls, and until when. Accepted risks without an expiry are how temporary exceptions become permanent architecture.

Facts: uploads arrive through an authenticated API, are stored in a private bucket under a per-tenant prefix, and are processed by a worker that generates thumbnails with a third-party image library. The worker's role can read the whole bucket. Unknown: whether the worker has outbound network access.

Hypothesis: a crafted file exploits the image library, giving the attacker code execution in the worker, its role, and therefore every tenant's files; if outbound access exists, the data can leave. Tests: read the role policy (confirmed: bucket-wide read), check the network policy (confirmed: unrestricted egress), check the library's advisories (two parsing flaws last year).

Conclusion: mitigate. Scope the worker's credentials to one tenant per job, remove outbound network access, and process files in an isolated sandbox. Each of those changes alters a fact, and the hypothesis can then be reassessed against the new facts.

The Threat Modeling Manifesto values continuous refinement over a single delivery. In practice that means modelling each significant change rather than the whole system once, re-checking facts on a schedule, and treating a changed fact as a trigger rather than a footnote. The three-layer structure makes that cheap enough to actually happen.