mantis-calibrate

Risk Calibrator (/mantis-calibrate)

System Goal

Risk Analysis Expert. Evaluates confirmed findings against a rigorous risk matrix, taking into account successful reproduction and production viability to produce a final risk score (1-10).

Command Definition

• Command: /mantis-calibrate
• Description: Calibrates the risk level of findings based on evidence and impact.

Input/Output Contract

• Reads:
  • workspace/findings/*.json (all finding files to load full pipeline state).
  • workspace/kb/THREAT_MODEL.md (if exists, to check threat boundary overrides and asset criticality).
  • workspace/.mantis_state.json (to track the current loop pass, and to read active_snapshot — {snapshot_id, snapshot_pinned, root} — for finding provenance and history stamping; absent ⇒ degraded/today's behavior).
• Writes:
  • Updates finding files in-place with scoring/calibration fields (impact_score, likelihood_score, availability_tier, inferred_exposure, attacker_position, mantis_risk_score, priority, sanity_triage_applied (may begin with STALE_EVIDENCE when the STALE-EVIDENCE guard suppresses heuristics), calibration_checklist (entries may carry STALE_EVIDENCE: reasons), outrage_commentary, executive_summary). Appends a history entry with snapshot provenance (see history JSON template in the body).
  • Reusable helper script workspace/helpers/append_calibrate.py.
• Preconditions:
  • Confirmed or raw findings must exist in workspace/findings/.
• Idempotency Guarantee:
  • Updates findings in place by overwriting existing keys with the calculated score. Running multiple times on the same inputs yields identical outputs, with no duplicated entries.

Instructions

Convert the raw security findings and their empirical results (repro/patch) into a prioritized, actionable risk report.

Execute the calibration as follows:

Locator resolution (findings-only). Calibrate never needs to read target source to compute a score, but some heuristics below may re-inspect code; resolve the code root the same way every stage does. Block A is inlined below:

LOCATOR RESOLUTION (before reading ANY target code or artifact):
0. ROLE: If this skill NEVER reads target source (report, calibrate, reflect),
   you are a FINDINGS-ONLY stage: skip steps 2-6; still read active_snapshot from
   state for provenance/annotation; NEVER stop merely because a code root is unset.
1. Determine CODE_ROOT, in this priority order:
   a. If --target_root is passed on THIS invocation, CODE_ROOT = --target_root.
      It is AUTHORITATIVE and OVERRIDES SNAPSHOT_ROOT and the state fallback
      (used when a caller hands you a prepared tree, e.g. a patched shadow).
   b. Else if --snapshot_root (or SNAPSHOT_ROOT) is passed, use it.
   c. Else read state_root/workspace/.mantis_state.json (state_root from
      --state_root if passed, else ./workspace/... relative to the current dir)
      -> active_snapshot.root / .snapshot_id / .snapshot_pinned.
   d. Else (no arg AND no readable active_snapshot): CODE_ROOT = current directory,
      treat snapshot_pinned = false (MODE-OFF). Do NOT stop.
2. SENTINEL CHECK (only if snapshot_pinned is true AND you did NOT take path 1a):
   verify CODE_ROOT/.mantis_snapshot_id exists and equals SNAPSHOT_ID. If missing
   or different -> STOP "snapshot sentinel mismatch". (A --target_root tree (1a) is
   deliberately mutated and is sentinel-EXEMPT.)
3. PATH FIELDS:
   - SNAPSHOT-RELATIVE (read under CODE_ROOT): code_paths entries; plan target_files
     that are file paths. Strip ONLY a trailing ":<digits>". A code_paths entry
     containing "://" is a URL/endpoint, NOT a file read. A code_paths entry that is
     NOT of the form <existing-path>:<integer> is a non-source LOCATOR
     (symbol/offset/endpoint): only check that the artifact/symbol exists; skip ALL
     line-range and line-existence logic.
   - STATE-RELATIVE (read/write under state_root/workspace, NEVER prefix CODE_ROOT):
     kb_references, repro_file_path, reattack_file_path, helper scripts, report
     files, and all state/findings JSON.
4. Never WRITE under CODE_ROOT when snapshot_pinned is true. Any command that
   compiles, generates, or writes artifacts MUST run in a PRIVATE SHADOW copy
   (mktemp -d from CODE_ROOT), never with cwd=CODE_ROOT. Read-only inspection may
   cd into CODE_ROOT.
5. VCS-METADATA CARVE-OUT: history-log extraction and any VCS diff/blame command
   run in the LIVE repository root (which still has .git/.hg/.repo), NOT CODE_ROOT
   (the snapshot copy strips VCS metadata). Do NOT stop merely because CODE_ROOT
   lacks .git/.hg/.repo.
6. Every shell command uses ABSOLUTE paths and sets its own working directory on
   that call. Do NOT assume the working directory persists between calls.

[!NOTE] CURRENT-PASS CHECK (defensive; the binding guarantee is on the harness per mantis-pipeline-adapter Scenario 2): if active_snapshot is present AND active_snapshot.pass != state.pass_number, treat the snapshot as STALE for this pass — STOP "stale active_snapshot: pass mismatch" or degrade as HALT (snapshot_pinned effectively false: no authoritative verdicts, Block B NOT_MATCHED, reproduce not_attempted). This catches a custom harness that preserved active_snapshot across the Stage 15 pass increment without re-pinning. The reference meta-agent re-pins every pass, so this check never fires there. Block B itself cannot detect this (it is snapshot_id-only, not pass-aware).

Snapshot provenance & STALE-EVIDENCE guard (mechanical; do this before scoring):

P0. Read active_snapshot ({snapshot_id, snapshot_pinned, root}) from state_root/workspace/.mantis_state.json (Block A step 0/1c). NEVER stop if it is absent — calibrate is a FINDINGS-ONLY stage (Block A step 0).

P1. MODE (single decision for the whole run — branches on active_snapshot presence, 3-state model): - active_snapshot is ABSENT in state (no --sync was requested — MODE-OFF = today's default) -> MODE = MODE-OFF. Score exactly as today: run every heuristic as written, do NOT compute PROVENANCE, do NOT emit STALE_EVIDENCE, and do NOT emit any HALT/PINNED banners. (Backward-compatible default path — byte-for-byte today's behavior.) - active_snapshot IS present AND snapshot_pinned is not exactly true (HALT mode — the tree raced or could not be pinned) -> MODE = HALT. SNAPSHOT_ID = active_snapshot.snapshot_id (a live: id). Compute PROVENANCE per P2 below and fire the STALE-EVIDENCE guard conservatively (same as PINNED for guard purposes), because findings' locators may be stale. Authoritative verdicts (VERIFIED_SECURE, failed_to_reproduce, DUPLICATE, FALSE_POSITIVE, NON_VIABLE) are forbidden this pass. - snapshot_pinned == true -> MODE = PINNED. SNAPSHOT_ID = active_snapshot.snapshot_id. Full provenance + STALE-EVIDENCE guard as below.

P2. In MODE = PINNED or HALT, compute PROVENANCE per finding F using Block B's compare rule: - F.discovery_commit missing OR empty OR the literal "MIXED" -> NOT_MATCHED. - F.discovery_commit != SNAPSHOT_ID (exact string compare, no fuzzy) -> NOT_MATCHED. - F.discovery_commit == SNAPSHOT_ID -> MATCHED.

P3. A per-finding heuristic is STALE for F when MODE is PINNED OR HALT AND either: - PROVENANCE(F) == NOT_MATCHED, OR - the finding's code_paths target file does NOT exist under CODE_ROOT. (Resolve CODE_ROOT via Block A steps 1 & 3; strip a trailing :<digits>; read ONLY the pinned root, NEVER the live tree. If you cannot resolve CODE_ROOT at all, treat the file as absent -> STALE.)

P4. When a heuristic is STALE for F, do NOT apply it; keep the conservative (pre-adjustment) score; and ensure the literal token STALE_EVIDENCE is the FIRST token of sanity_triage_applied (add it once, before any UNKNOWN warnings and any fired-rule list). The four guarded heuristics and their exact STALE handling are: - Dead-code 0.2 multiplier (Section 2, "Asset Criticality & Reachability"): do NOT apply the 0.2 reduction; use the multiplier you would otherwise have. - repro_failure (Section 3 rule): do NOT force-LOW; set calibration_checklist.repro_failure.outcome = "UNKNOWN", reason beginning "STALE_EVIDENCE: ". - vague_code_paths (Section 3 rule): do NOT force-LOW; set calibration_checklist.vague_code_paths.outcome = "UNKNOWN", reason beginning "STALE_EVIDENCE: ". - static_confirmation trace-lift (Section 3 rule, its "valid external stack trace/sanitizer…" exception): do NOT apply the trace-lift; KEEP the static HIGH cap in force (likelihood_score <= 3, 0.8 Hazard multiplier, NOT CRITICAL). Set calibration_checklist.static_confirmation.outcome = "APPLIES", reason beginning "STALE_EVIDENCE: trace-lift suppressed; trace/crash-log may predate the active snapshot; ".

P5. NON-SOURCE finding: if the finding's code_paths entry is a non-source LOCATOR (contains "://", or is not of the form <path>:<integer> — per Block A step 3: a symbol / offset / endpoint), SKIP all source-only heuristics for it — dead-code, file-path/imports/caller-hierarchy exposure inference, vague_code_paths, and static trace re-inspection — and score from the finding's DECLARED metadata (attacker_position, privileges_required, production_viability, repro_status, threat model). A non-source skip is NOT flagged STALE_EVIDENCE (it is normal, not drift). Non-source findings default inferred_exposure to "INTERNAL" (0.8) unless the finding/threat-model declares otherwise.

1. Load Full Pipeline State:

   • Read all JSON files from the workspace/findings/ directory. Because the pipeline appends data to each finding file at each stage, these files provide the complete picture of each finding's journey (including its id, reproduction status, and production viability).
   • Missing Fields Fallbacks: If any finding is missing viability, or reproduction fields (such as chained findings), apply the following fallback defaults before scoring:
     • If production_viability is missing, treat it as "CONDITIONAL_VIABLE".
     • If repro_status is missing, treat it as "not_attempted".
   • Snapshot provenance (only when MODE is PINNED or HALT): for each finding, compute PROVENANCE (MATCHED / NOT_MATCHED) per preamble step P2 and record it in scratch. This value gates the STALE-EVIDENCE handling in Sections 2 and 3 (preamble steps P3–P4). In MODE == MODE-OFF, skip this — score as today.
   • Read workspace/kb/THREAT_MODEL.md from the Knowledge Base (if it exists) to evaluate component exposure, trust boundaries, asset criticality, and any custom Calibration Overrides (e.g., specific threat positions or caps that should be lifted or customized for the project).
   • Batch Processing: If there are more than a few findings to calibrate, split the task into batches (a few findings at a time). If you have the ability to invoke subagents, delegate each batch to a subagent to process in parallel, then aggregate the results. Each batch/subagent inspects ONLY the pinned snapshot: pass --snapshot_root=<active_snapshot.root>, --snapshot_id=<active_snapshot.snapshot_id>, and --state_root=<workspace parent> so every subagent resolves the same CODE_ROOT via Block A. Any code inspection a batch performs (exposure inference, dead-code, static-trace verification) MUST read through that CODE_ROOT (the pinned copy), NEVER the live tree, and MUST honor the STALE-EVIDENCE guard (preamble P3–P4). Findings-only fields (score, priority, history stamp) are computed from the finding JSON regardless.

2. Calculate Risk Score (1-10): For each unique finding file, calculate the actual technical risk score in a matrix form based on the following formula components, where Hazard = Impact + Likelihood:

   • Impact (1-5): Evaluate impact using the CIA triad (Confidentiality, Integrity, Availability) while strictly considering Blast Radius.
     • 5: Complete, systemic loss of Confidentiality (full data breach, leak of root cryptographic/HSM master keys) or Integrity (system compromise, e.g., clear Remote Code Execution (RCE) by an unprivileged attacker who isn't already in an effective position to execute code). MUST NOT be used for attackers who already have execution privileges.
     • 4: Substantial loss in one or more areas. This includes systemic Availability loss (total outage of a major service) or major data exposure.
     • 3: Moderate loss (e.g., partial data exposure, temporary or partial system disruption).
     • 2: Minor loss (e.g., minor information leak, localized disruption). A vulnerability whose blast radius is limited to affecting only a single user's own data MUST NOT be scored higher than 2. Exception: If the action lacks non-repudiation (allowing the user to plausibly deny the action to commit fraud or blame others), or triggers side-effects affecting other users/system stability, it should not be downgraded.
     • 1: Negligible impact on CIA, mostly a cosmetic issue. Findings of the type "the code is fragile", "lack of defense-in-depth", or purely theoretical hygiene issues MUST have an Impact score of 1, ensuring they are rated LOW at most.
     • Security Control Bypass (Upgrading): If the vulnerability directly bypasses a core security control (e.g., authentication, authorization, cryptographic signature verification) or defeats the primary security purpose of a library (e.g., a library meant to secure keysets allows attacker control), elevate the Impact score to at least 4 (or 5 if it leads to systemic compromise), even if the immediate technical impact seems localized.
     • Note on Privileges Required & Lateral Movement:
       • If the finding requires HIGH privileges (e.g., administrative privileges, admin-to-super-admin escalation) or only allows lateral movement/pivoting between internal components from an already compromised state, cap its individual Impact score at 2, unless the exploit results in escaping the container boundary (to the host node) or cross-tenant escalation.
       • If the finding requires LOW privileges (e.g., standard authenticated user), cap its individual Impact score at 3 (unless it leads to systemic compromise of other tenants/users, OR it directly bypasses a core security control/library purpose, in which case it can be higher).
       • These caps apply to individual findings. If successfully chained into an Exploit Chain (Super Finding) by the chaine

...