OPER-149
Precheck reconciliation drift monitor — D-26 next action 2
Band
MEDIUM
Suggested
Justin Cooke
Body
feature_id: FEAT-studio-dispatcher
decision_memo: docs/pdt/2026.09.09-d26-premature-done.md
blocked_by: OPER-148
filing_template_version: 1
## Context
Shankar's DISSENT mitigation on PDT memo D-26 (`docs/pdt/2026.09.09-d26-premature-done.md`): even with the provenance guard shipped, the precheck is basically an EVAL of "did this ticket actually complete?" and evals need drift detection. Add a nightly reconciler that diffs the precheck-DONE set against the merged-via-webhook DONE set and emits a metric.
If the two diverge (precheck reconciled a ticket to DONE but no webhook-driven completion was ever recorded for the same branch, or vice versa), the guard is either too tight (missing real completions) or too loose (still hitting stale identifier matches). Either way it needs human triage.
Alert threshold per D-26: any 7-day window where reconciliations differ by more than 5% of dispatched-to-DONE volume.
## Acceptance Criteria
- New file `packages/dispatcher/src/precheck-drift-monitor.ts` exports `computePrecheckDrift({ from, to, prisma })` returning `{ precheckDoneCount, webhookDoneCount, divergentTickets: Array<{ identifier, precheckAt, webhookAt }>, driftPercent }`.
- New cron route `apps/web/src/app/api/cron/precheck-drift/route.ts` runs the monitor nightly over the last 7 days; when `driftPercent > 5`, emit a `precheck.drift.above_threshold` alert via the existing metrics/notification path.
- Unit tests cover: (1) zero drift; (2) precheck-only completion (webhook missed) → included in divergentTickets; (3) webhook-only completion (precheck skipped it correctly) → included in divergentTickets; (4) driftPercent calculation across a mixed set.
- Vercel cron config adds the new route on a nightly schedule (00:30 UTC).
## Non-goals
- Auto-remediation of divergent tickets — this is an ALERT, not a fixer.
- Historical backfill of divergence data older than the monitor's first run — only forward-looking.
## Related
- Depends on the provenance-guard ticket (D-26 next action 1) shipping first. Running this monitor before the guard lands would flag today's known bug on every ticket and drown the signal.Attachments
Loading attachments…
Comments
Loading comments…