Reproducibility probe agent

Some tickets can be settled by running the thing. A customer says a call returns an error. The fastest way to know is to make that call. A probe agent takes the reported steps and runs them against a test account or sandbox. It records what came back. The result is reproduced, not reproduced, or blocked because something needed was missing.

Reproduced is the strongest evidence available at this stage. It turns a customer report into an observation the team made itself. The record holds a timestamp, a request identifier, and the response body. That is a finding. Everything before it, including a confident reading of the logs, is still a hypothesis.

Not reproduced means something narrower than people assume. It means the failure did not occur under the conditions the probe used. It does not mean the customer is wrong. Plan tier, data volume, region, browser, and account age all change behavior. The probe should list the conditions it ran under. The gap between those and the customer's real setup then stays visible.

The environment question comes before anything else. A probe that runs against production data can create orders, send email, or charge a card. Point it at a sandbox with seeded accounts, or at a test tenant inside production that no customer shares. Write down which one it uses. That answer changes how much a not reproduced result is worth.

Define the allowed action set explicitly. Read calls in every case. Writes only inside the sandbox, and only for the object types listed. No calls that touch a real customer's records, even to read, without an approval attached to the ticket. Rate limits and a call ceiling belong in the same document. A retry loop against a payment gateway becomes an incident of its own.

The probe also needs an input contract. Steps to perform, in order. The expected result. The observed result. The account or object type involved. Version and region where they matter. When the ticket lacks the steps, the probe should stop and say which field is empty. Collecting those details is cheaper at submission, through questions asked while the customer is still typing.

Set a hard stop on runtime. A probe that hangs on a slow dependency should return blocked and name the dependency. Retrying until someone notices helps nobody. A blocked result is useful. It tells the engineer that the environment, not the product, is in the way today.

The first run should mirror the report as closely as the sandbox allows. Same endpoint, same parameters, same sequence. Resist the urge to clean up the steps. A customer who sent a malformed date is describing the actual failure, and a corrected date hides it. Run the report as written before running the version that makes sense.

When the first run does not fail, the second run changes one thing. Plan tier. Record count. Region. Locale. Each change is a separate run with its own record. A probe that alters four variables at once and then fails has explained nothing. The value of this step is the list of conditions that trigger the problem and the conditions that do not.

Every run leaves an artifact. Request, response, status code, duration, and the environment identifier. Attach them to the ticket rather than pasting a summary sentence. An engineer reading the ticket next week needs to see the actual response, not a claim that the call failed. Those artifacts also feed the search for tickets that were solved the same way.

What follows is a constructed case. A ticketing platform sells seats for concerts and sports through venue websites. A venue reports that the seat map fails to render for one upcoming show. Every other show on the same account works. The ticket includes the event identifier and a screenshot of an empty map area.

The probe runs four times. Run one calls the seat map endpoint for the reported event with a read-only sandbox token. It returns a 500 after eleven seconds. Run two calls the same endpoint for a different show at that venue and returns a normal map. Run three requests the raw seating chart for the failing event and succeeds. Run four requests the pricing overlay for that event and fails with the same error.

The result is reproduced, and the failing surface is narrower than the ticket said. The seat map is not broken. The pricing overlay for that one event is. The probe records four runs, four responses, and the environment it used. It does not name a cause. A price rule with an open end date is a plausible explanation. It stays in the hypothesis field until someone reads the event configuration.

The engineer takes it from there. The narrowing keeps the escalation from being filed as seat maps are down. That wording would pull three teams into a problem living in one pricing record. Pulling the rest of the account context is a separate fact-gathering run across billing and configuration.

A probe that passes is the most dangerous output in this stage. It reads as proof that nothing is wrong. Usually it means the sandbox differs from the customer's account in a way nobody wrote down. Treat not reproduced as a request for more conditions, never as a reason to close.

Environment drift builds quietly. Sandboxes fall behind on version, seed data ages, and feature flags get set differently. Stamp every probe result with the build and the flag state. A result from a sandbox two releases old says little about today's product.

Side effects escape the sandbox. A probe that sends a notification, books inventory, or hits a third party in test mode can still reach a real person. Review the action list whenever the product adds an integration. Anything that can leave the system needs a named owner's approval.

One ticket at a time hides scale. A probe confirms this customer's failure and says nothing about the other nine reports sitting in the queue. Counting them is the job of clustering across tickets, and the cluster is what justifies an engineering escalation.

Probing touches systems, so somebody has to own the permission. The support engineer on the ticket can run sandbox probes without asking. Anything that touches a live customer account needs the account owner's agreement, recorded on the ticket, including read-only checks.

Interpretation stays human. The probe reports what happened. Whether the result confirms the working hypothesis is a judgment. Partial reproductions are common, and they need a person to read them. Write the conclusion in the engineer's words, with the run identifiers cited underneath.

A confirmed reproduction is also the raw material for the handoff. Engineering wants the steps, the environment, and the responses, not a narrative. Packaging that evidence is the filing step at the end of the workflow, and it goes much faster when the probe artifacts are already attached.

Later, the same machinery gets reused for a different question. Once a fix ships, someone has to run the failing call again and see it succeed. That check belongs to the verification step after resolution. Same tooling, different claim: one asks whether the problem exists, the other asks whether it is gone.