Postil

postil plan: config dry-run

Review configuration is usually tuned by trial and error against live pull requests, for weeks. postil plan replaces that loop with a deterministic preview: apply a candidate config to reviews that already happened and see exactly what would change.

How it works

Every completed review leaves an envelope: the full set of findings the model produced, including ones suppressed by your current thresholds. postil plan re-applies the candidate config's filters — severityThreshold, minConfidence, ignore globs, maxFindings, gate.failOn — to those stored envelopes.

No model calls are made. The plan is exact for filtering changes, free to run, and instant.

Usage

# store envelopes as you review, then preview a candidate config
postil review --staged --output-json > .cache/envelopes/r1.json
postil plan --envelopes .cache/envelopes --config .postil.candidate.yaml

postil plan: replaying 3 stored review(s) under candidate config (.postil.candidate.yaml)

  r1.json: 5 -> 3 finding(s); gate: passing (unchanged)
      would suppress: src/api/users.ts:88 [warn] broad except swallows errors
      would suppress: src/api/users.ts:140 [info] redundant null check
  r2.json: 4 -> 2 finding(s); gate: FAILING -> passing
      would suppress: src/billing/charge.ts:51 [error] retry has no upper bound
      would suppress: generated/schema.ts:12 [warn] unused import
  r3.json: 2 -> 2 finding(s); gate: passing (unchanged)

Summary: 4 finding(s) would be suppressed; 1 gate outcome(s) would change.

What it answers

  • "If we raise minConfidence to 0.75, how many of last month's comments disappear — and were any of them ones we acted on?"
  • "If the gate also fails on warn, how many merged PRs would have been blocked?"
  • "Does ignoring generated/** hide anything that ever produced an error-severity finding?"

Limits, stated plainly

postil plan re-filters what the model already said. It cannot predict findings a different reviewer.focus or model would have produced — those change the model call itself. For filter and threshold changes (the overwhelming majority of config churn) the preview is exact; for model changes it tells you so instead of guessing.

Envelope format details are in the envelope schema reference.