Model governance proof trail
Defend a model-driven decision to the examiner who asks eighteen months later.
A regulated firm publishes a marketing claim. An ML model output feeds the call to substantiate it. Months later, internal audit, model validation, and an examiner want to know why that output, on that date, was justified. This page walks one such decision, end to end, in synthetic form: the messy evidence in, the deterministic proof trail out, and how an outside reviewer re-derives the disposition offline without trusting the firm's own tool.
The scenario.
Meridian Ridge Asset Management (synthetic) runs a marketing-review workflow. Before a performance claim goes out, an ML peer-ranking model scores where a strategy sits against its peer group, and that output feeds the decision to substantiate the claim or hold it for a human. The claim under review reads: the Meridian Ridge Quantitative Equity strategy achieved top-decile risk-adjusted returns among its peer group over the trailing five years.
The honest reviewer of that decision sits outside the desk that made it. Internal audit tests the control. Model validation, as a second line, has to re-derive the output. An examiner, under the firm's own obligations, can ask for the basis of the claim long after the model, the reference data, and the reviewer have all moved on. The firm applies its own regime here: the SEC Marketing Rule expectation to substantiate a claim on demand, and model-risk governance in the SR 11-7 tradition. Those are the firm's obligations, not a Forge credential. Forge supports them and produces the defensible proof trail they call for. It does not certify compliance, and the business judgment stays with the firm.
| The decision | Given the model output and the firm's encoded marketing-review policy, substantiate the claim for distribution, hold it for human review, or do not substantiate it. |
|---|---|
| Control depth | Gate and evaluate. Forge checks the evidence against the encoded policy and returns a disposition. It is not enforcing inside a protected action boundary here, and it is not a network control of any kind. |
| What is pinned | The model version (peer_ranking_model v4.2.1), the policy version and its hash, a substantiation confidence threshold, a source-reliability floor, and a data-freshness window. All bound into the proof trail. |
| Who re-derives it later | Internal audit, model validation, and the examiner, each working from the exported package and the public key, without trusting the firm's running system. |
The messy evidence in.
These are the artifacts the workflow actually hands over. AI and natural-language extraction normalize and pull the values before anything is signed, entirely outside the signed decision path. Every extracted value is carried with its source document and a content hash, so a reviewer can see exactly where each fact came from. The load-bearing weakness is visible before any rule runs: one substantiation source is a peer-universe snapshot that is older than the policy freshness window.
Peer-ranking model output
peer_ranking_model v4.2.1 returns a trailing five-year peer percentile of 8, inside the top decile. Normalized outside the signed path.
Feature vector fed to the model
Sixty months of strategy and benchmark return series plus the peer universe identifier that the model scored.
Peer universe reference (stale)
The peer set the percentile is measured against. This snapshot is older than the policy freshness window, which is what caps confidence.
Risk-adjusted-return methodology
The approved memo defining how risk-adjusted return is computed for the claim, on a Sharpe basis.
Read the shape of it before any policy runs: the model leans toward substantiate, but the peer set it depends on is stale, and the whole claim rests on a reference snapshot no one refreshed. The point of the run is to make that weakness explicit, attributed, and reproducible, rather than to smooth it over.
The proof trail out.
This is the product: a deterministic, replayable proof trail, not just a signed blob. The customer's policy is applied, the model version is pinned, and the same canonical input reproduces the same selected_action and the same replay_key every time. Fields carry the real DecisionRecord names. Values are illustrative and synthetic.
The policy, applied to these exact inputs, does not clear the claim on its own. The leading substantive outcome is below the auto-substantiate threshold and one substantiation source falls below the reliability floor, so the record routes to a human. This is not "the model said yes."
The leading outcome is SUBSTANTIATE at 0.71. The policy requires 0.80 to clear a claim without a human, so the selected_action is the hold, not the argmax. The posterior is shown, not summarized away, so the reviewer sees exactly how close the call was.
- Stale peer universe (e3). The percentile is measured against a snapshot older than the freshness window. Confidence is capped pending refresh. If refreshed peer data moves the strategy out of the top decile, the substantive outcome flips and the claim cannot be substantiated as written.
- Leading outcome below threshold. SUBSTANTIATE sits at posterior 0.71, under the 0.80 auto-substantiate bar. The margin is recorded, not rounded up.
The model-validation sign-off boundary is explicit, not implied.
- Forge does not make the call. The record carries a disposition and the proof. A human at the boundary keeps the decision. Here the record sits in PENDING, routed by role, because the encoded policy tripped both its confidence gate and its reliability floor.
- Model validation, as a second line, re-derives the output. A supervisory principal signs the claim off or holds it. If the reviewer supplies a refreshed peer universe or a stronger substantiation source, that is recorded as a new signed amendment that references the original. The first record is never altered, and the stale snapshot stays in the chain, still weighted 0.45, still visible.
- This is calibration, not training. Reviewer feedback runs through an explicit, customer-controlled path. Forge does not learn from it silently and does not mutate the decision logic. Confidence and reliability are traceable, reproducible calculations an auditor re-derives from the recorded inputs, not qualitative reviewer flags.
- The reviewer who must be satisfied sits outside the desk. Internal audit, model validation, and the examiner all need to trust a decision they did not make. If the only account of why the claim was justified comes from the same team that produced it, that is not independence. You cannot build your own independence.
Verify it yourself, offline.
An external reviewer receives the exported package and the public key, and verifies it on their own machine, with no connection to Forge and without trusting Forge's running system. They re-derive the disposition and the replay key themselves. They do not have to take the firm's word for any of it.
$ forge verify ./meridian-ridge-claim-package/ # illustrative record dr_9c31f7a02e6b845d signature (Ed25519) ............................ VALID key_id ......................................... forge-signing-key-2026-02 canonical_input_hash recomputed from inputs .... MATCH sha256:c4a90b7e…11f0 ruleset_hash present in bundle ................. MATCH sha256:7b1d5a2f…e93a model_ref pinned in bundle ..................... MATCH peer_ranking_model v4.2.1 evidence content hashes (4/4) .................. MATCH deterministic replay ........................... REPRODUCED selected_action = HOLD_FOR_HUMAN_REVIEW confidence = 0.71 replay_key = rk_4d8f2a91c7e0b3d6 (MATCH) RESULT: package is authentic and reproduces the determination from the recorded inputs.
Forge signed it, and it was not altered in transit.
The Ed25519 signature proves the record came from Forge and that its bytes were not changed on the way to the reviewer. That is the integrity check. It is deliberately not the headline.
Full field-level tamper-evidence comes from replay.
The reviewer recomputes the canonical input hash, re-derives selected_action and replay_key from the recorded inputs and the pinned model and policy, and rechecks every evidence content hash. A single changed byte anywhere fails to reproduce. Because no generative model sits in the signed path, replay is exact, not approximate.
| What VERIFIED proves | The record is authentic and unaltered after signing. selected_action, confidence, and replay_key reproduce exactly from the recorded inputs, the pinned model version, and the encoded policy. The stale peer-universe source was surfaced and attributed, not hidden. The decision routed to a human by role. |
|---|---|
| What VERIFIED does not prove | That the 0.80 threshold, the reliability floor, or the top-decile claim itself is correct. That the peer universe data is accurate. That any legal or regulatory conclusion follows. That judgment stays with the firm, its counsel, and its examiners. Forge proves the decision was justified under the encoded policy and the recorded evidence. It does not certify that the policy or the evidence was right. |
What a proof trail contains.
Every field is populated on every run, so a reviewer never has to reconstruct one later.
- record_id A stable identifier for this decision record.
- selected_action The disposition the policy produced. Substantiate, hold for human review, or do not substantiate.
- confidence The decision confidence, tied to the policy threshold that governs whether a human is required.
- posterior The distribution over the candidate substantive outcomes, shown in full rather than reduced to a single number.
- evidence_chain Each item carried with its source and a truncated sha256 content hash, including the model output and version, the input features, and the substantiation sources.
- uncertainty The held uncertainty, attributed to specific evidence, with the sensitivity that would flip the outcome.
- human_review boundary The boundary, the status, the role the record routed to, the reason, and the note that the human keeps the decision.
- replay_key The handle that lets a reviewer reproduce the identical record from the identical input.
- signature and key_id An Ed25519 signature and the key identifier that verify the record offline.
Start with one decision surface.
You do not have to re-plumb model governance to see whether this holds up. Pick one high-consequence surface, a marketing-claim substantiation, a model-change approval, or an agentic override that no human currently logs. We encode the policy, point one real decision at the engine, and produce the independently verifiable proof trail so your validation team and your examiner can re-derive it. That first bounded run starts as a scoped Artifact Discovery. Forge Orbital is at design-partner stage. The engine and API are live, tested, and pilot-ready.