Log and stack-trace summarization

A customer attaches a log because the log is what they have. Nobody on the support side wants to scroll nine thousand lines to find the four that matter. Summarization does that first read. It returns a short list of candidate causes, ordered, with the exact line ranges behind each one. An agent should be able to click a hypothesis and land on the lines that produced it.

The ranking is a set of hypotheses. None of them is a finding until something confirms it. A log shows what a system printed, which is not always what happened. Timeouts print as connection errors. Retries print as repeated failures. A summary saying the database was down, when the log only shows connection refusals, has crossed from reading into asserting.

This step also does not write a fix. Producing the corrected setting or patch is a separate drafting job with its own review. Keeping summarization read-only keeps its mistakes cheap. A wrong hypothesis costs a few minutes. A wrong patch can cost the customer an outage. The two jobs share a ticket, not a step.

A model summarizing an unfamiliar log guesses at structure. Give it the shape instead. Which services write to which file. How timestamps are formatted and what timezone they use. Which log levels the product actually uses, and which ones are noise by convention. Where request identifiers appear. Those identifiers are how one user's failure gets separated from the rest of the traffic.

The error catalog matters more than the model choice. Most products have a finite set of error strings with known meanings. Load them, with the usual cause and the usual fix noted for each. A summarizer that can match a line against a known code writes a grounded hypothesis. One without that list writes a paraphrase that sounds informed. The catalog is a support engineering artifact, and it needs an owner who updates it after releases.

Redaction runs before the model sees anything. Logs carry tokens, email addresses, internal hostnames, and sometimes full request bodies. Strip them at ingestion. Set a retention period for the raw file that is shorter than the retention period for the summary. These handling rules overlap with what the intake parser is allowed to store, and both should follow the same policy.

Bound the window. A log covering three days will bury the failure under normal traffic. Ask the customer for the window around the incident, or cut it by timestamp when the ticket names one. When no timestamp exists anywhere, say so and summarize nothing.

Logs mislead through volume. The most repeated error is usually a downstream effect, printed once per retry. The line that matters often appears once, earlier, and at a lower level. Ranking should favor the earliest anomaly in a causal chain over the most frequent one. Frequency belongs in the output as a count, not as a reason to promote a candidate.

Every hypothesis needs three parts. The claim, stated in one sentence. The line ranges that support it. The check that would confirm or kill it. That third part turns a summary into something an engineer can work through in order. Hypotheses without a check tend to get argued about rather than tested.

Say when the log does not contain the answer. Truncated files, missing debug levels, and logs from the wrong service are all common. The correct output there is a statement of what is absent and what to request. Silence about a gap is worse than an empty result. The reader assumes the file was searched and came back clean.

The scenario below is invented. A vendor sells electronic signature software to law firms. An administrator reports that a bulk send of 240 agreements stopped after 61 documents. She attaches a 9,000-line application log covering forty minutes. The support engineer has the ticket, the log, and no reproduction yet.

The summarizer returns three hypotheses. The first says the PDF render worker ran out of memory on document 62. It cites an out-of-memory line at 04:12 and a worker restart four lines later. The second points at a template with an unusually large embedded image, citing a size warning on the same document. The third names rate limiting from the storage service, citing 118 throttle responses across the run.

The ranking puts the throttle responses last despite their count. They start at 03:58, well before the stop, and the run kept working through them. The memory line sits first because it is the earliest event with no normal explanation. Each hypothesis carries a check. For the top one, the check is the render worker's memory metrics for that minute.

The engineer opens the cited lines and finds the summary accurate. The memory hypothesis still is not a finding. It becomes one after the metrics show the worker hitting its limit at 04:12. The same template also has to fail on its own in a test send. Confirming that second half is a probe against a test account, not another pass over the log.

The costly errors here all look competent. Invented citations are the worst of them. A model may reference line 4,120 for a claim that line does not support. Validate every citation against the file. Drop any hypothesis whose quoted text is not where the summary says it is.

Framework noise crowds out product code. Stack traces are mostly library frames, and a summarizer can spend its ranking on those. Weight frames from the product's own namespaces higher. That is what an engineer does when scanning for the first familiar package name.

Known errors get reframed as new ones. A line with a documented meaning gets a fresh explanation in the model's own words. The engineer then loses the link to the existing runbook. Matching against the catalog first prevents most of this.

The same explanation gets written on twenty tickets. Summarization works inside one file and cannot see that the queue is full of the same failure. Noticing that requires retrieval of how similar issues were resolved before and a view across the corpus.

Nothing from the summary belongs in the cause field until a person checks it. The rule is simple to state and easy to skip under load: open the cited lines, read them, and decide. A hypothesis copied into an escalation without that check arrives with the model's confidence. It arrives without any of the caveats.

Keep the label visible wherever the text travels. Suspected and confirmed should be separate fields on the ticket, not adjectives inside a sentence. Fields survive copying into a Jira description. Adjectives do not. Anyone reading the escalation later can then tell what was tested.

Broader gathering stays a different step. When the log answers only part of the question, the account and status systems hold the rest. Pulling those is a wider fact-gathering run. The engineer decides which one the ticket needs.

Review the misses monthly. Take tickets where the recorded cause differs from the summary's top hypothesis, and read what the log actually showed. Those cases usually point at a missing log level or an error string nobody added to the catalog. Both are cheap to fix once named.