Concepts
Incident memory
Repeats cost no LLM call, new analyses start with what was concluded before, and your corrections are tested.
AlertINT remembers what it concluded, so a nightly disk-fill or a noisy quota doesn't cost a fresh analysis every time.
| Mechanism | When | Effect | Extra LLM cost |
|---|---|---|---|
| Collapse | The same condition re-fires while its Situation is open | Counted, not re-analyzed | None |
| Recall | A new Incident matches a past analysis | The past Finding is given to the model as a hypothesis | None |
| Verdicts | You corrected a past Finding | The next triage must test your correction | None |
| Near-match classifier | A past Finding is one label away | Optional check: "same condition?" | ~250 tokens, off by default |
Collapse, recall, and verdicts are always on.
Collapse: repeats cost nothing#
A re-fire attaches as an occurrence to the analyzed Incident when all of these hold:
- the group key is identical
- the owning Situation is still open (not yet Recovered or Recovery unconfirmed)
- it fires within the attach window (default 30 min) of the last occurrence; each re-fire slides the window
Some re-fires are marked as an escalation on the occurrence: a severity rise, a new alert type, a sudden faster cadence, 100 occurrences, or 4 hours since the last analysis. They still attach (see Limits).
What you see: the Situation's recurred ×N count goes up. At ×5, ×10,
×25, ×50, ×100, then every ×100, one quiet thread reply is added (see
Slack). stdout gets one occurrence line.
After the Situation ends: a re-fire opens a new Incident and a new, linked Situation, and is triaged normally.
Recall: new analyses start with the past#
When a new Incident's group key matches a past analysis (within
lookback_days), the model gets a short memory section:
| Recalled | Not recalled |
|---|---|
| Source Incident and analysis time | Whole past Findings |
| Confidence and root-cause statement | Raw alert labels |
| Recurrence count and cadence |
- A hypothesis, not evidence. It is labelled as such, and it can't lift confidence: a re-fire with only annotations stays under the metadata-only cap.
- Self-correcting. Each analysis rules on the recalled cause: confirms, refutes, or silent. A cause refuted twice is demoted.
- Error status check. If the past Finding named an application error, AlertINT checks it once: resolved-but-firing reads as a regression, ignored reads as known-tolerated.
Verdicts: your corrections are tested#
A correction captured over MCP (alertint_incident_capture_verdict) is not
taken as fact. On the next triage of that group, its evidence runs in the
verification round, and the model must rule:
| Ruling | Result |
|---|---|
supported |
Your cause is adopted; confidence comes from the evidence |
contradicted |
Not adopted; the Finding says what contradicted it |
unverifiable |
Adopted as the leading hypothesis, confidence capped at 0.6, dated |
| Evidence never fetched | Treated as untested; the Finding says so |
- A confirmation ends the steering: the Finding was right.
- Corrections never expire by date. Live evidence can retire them; the calendar can't.
- Notes (
alertint_incident_annotate) are for the next person. They appear in the Situation's thread and in MCP (operator_history), and never enter the prompt.
Near matches (optional classifier)#
Recall needs an exact group key. When a past Finding differs by one label
value (same cluster and namespace, different service), it's shown as a
weak "one label off" hint. The classifier is one small Haiku call (~250
tokens, about $0.0003) that asks: same condition?
memory.classifier.mode |
Behaviour |
|---|---|
off (default) |
No call |
shadow |
Call runs, verdict goes to the audit log only; the prompt is unchanged |
on |
A matched verdict tags the hint "probably related" |
A timeout or unreadable reply never counts as a match.
When to switch shadow → on: after precision ≥ 0.90 over ≥ 100
matched verdicts, measured on your own audit log:
SELECT
SUM(json_extract(v.payload_json, '$.verdict') = 'confirms') AS confirmed,
SUM(json_extract(v.payload_json, '$.verdict') = 'refutes') AS refuted,
ROUND(1.0 * SUM(json_extract(v.payload_json, '$.verdict') = 'confirms')
/ NULLIF(COUNT(*), 0), 2) AS precision
FROM audit_log c
JOIN audit_log v
ON v.kind = 'incident.memory_recalled'
AND json_extract(v.payload_json, '$.recalled')
= json_extract(c.payload_json, '$.candidates[0]')
AND json_extract(v.payload_json, '$.verdict') IN ('confirms', 'refutes')
WHERE c.kind = 'memory.classifier_verdict'
AND json_extract(c.payload_json, '$.verdict') = 'matched';
Check it#
-
What the model saw:
alertint_get_incidentreturns amemoryblock with the same facts the analysis got. -
Analyses avoided:
SELECT COUNT(*) FROM audit_log WHERE kind = 'incident.occurrence_attached' AND json_extract(payload_json, '$.trigger') = 'none'; -
How recalled causes held up (healthy memory confirms far more than it refutes):
SELECT json_extract(payload_json, '$.verdict') AS verdict, COUNT(*) AS n FROM audit_log WHERE kind = 'incident.memory_recalled' GROUP BY verdict;
Configure#
Under memory:
| Field | Default | Does |
|---|---|---|
attach_window_minutes |
30 |
Attach window: a re-fire within this time of the last occurrence attaches |
judgment_ceiling_hours |
4 |
Marks an escalation this long after the last analysis |
occurrence_cap |
100 |
Marks an escalation after this many occurrences |
lookback_days |
90 |
How far back recall and recurrence counts reach |
classifier.mode |
off |
off / shadow / on. Quote the value: bare off and on are YAML booleans. |
classifier.timeout_seconds |
10 |
Classifier call timeout |
Limits#
- Escalations don't re-analyze. An escalation is recorded on the occurrence, but it doesn't start a new analysis while the Situation is open.
- Keys are verbatim. Grouping on volatile labels (a pod name, a job id) means a condition rarely repeats.
alertint validatewarns about labels that look volatile.