Delivery Exception Classifier

A delivery exception classifier reads the unstructured evidence a stop leaves behind (driver notes, free-text reason codes, and often a photo) and maps it to a controlled exception type. Typical labels include address error, customer refused, recipient absent, damaged goods, access blocked, and unsafe to leave. Each output should cite the note span that justified the label and return a stable exception type ID that downstream systems already understand.

The job is speed, not final judgment. Thin or empty notes should return empty rather than a guessed label. A dispatcher or resolver still owns the decision; the model only shortens the time between “something went wrong at the door” and “this case is in the right queue.”

Last-mile platforms already capture much of this raw material. Onfleet and Bringg collect driver notes and delivery outcomes in the field. project44 surfaces shipment and exception visibility across carriers. Salesforce often holds the customer case or account context once an exception becomes a service interaction. The classifier sits between capture and resolution: it turns noisy field text into a typed signal those systems can route on.

When every failed stop lands in a generic exception inbox, dispatchers re-read the same phrases all day. “Dog in yard,” “gate code wrong,” “customer said no,” and “box crushed at door” look similar in a list view and very different in the work they require. Manual triage is accurate when people have time. It does not scale when volume spikes after weather, peak season, or a bad address file.

An LLM classifier helps when the input is messy but patterned. Drivers write in shorthand, mix languages, and attach a photo that may or may not match the text. Structured reason codes help, yet field teams often pick a nearby code and explain the real issue in the note. Classification from the note (with photo context when present) recovers the intent that the dropdown missed.

The speed gain shows up in routing, not in replacing people. Address errors go to the address correction or geocode team. Refusals go to customer service with order and payment context. Absences go to reattempt scheduling. Damage goes to claims or quality with image evidence. Each hop you skip in the wrong queue is minutes saved per stop, and those minutes compound across a fleet.

Related workflows extend the same idea. A failed delivery resolution agent can act after classification. CV proof of delivery verification and real-time damage detection at the door strengthen the photo side of the evidence. An exception resolution agent can then execute the playbook for the typed case.

Inputs. Minimum useful input is the driver note text plus any structured delivery status the TMS or last-mile app already set. Optional but valuable inputs include one or more doorstep photos, the attempted timestamp, stop instructions, prior attempt history, and the package or order identifiers. Photo input should be treated as supporting evidence, not as a substitute for a note when the note is blank.

Outputs. Return a primary exception type ID from your controlled vocabulary, a short confidence or abstain signal if you use one operationally, and a citation: the exact note span (or spans) the model relied on. If a photo influenced the label, record that the image was considered, without inventing detail the image does not show. Keep the payload small enough to write back to Onfleet, Bringg, project44, or a Salesforce case custom field without a separate document store.

Empty and abstain. If notes are thin (“N/A,” a single emoji, or a duplicated default template) and the photo does not clearly support a type, return empty. Do not force a class to satisfy a dashboard. Empty is a first-class outcome: it leaves the stop in a human triage lane instead of poisoning automation with false precision.

Human in the loop. Dispatchers still resolve. The classifier proposes a type and a citeable reason. Overrides should be easy and logged so wrong labels become training and vocabulary feedback, not silent debt.

Start with a closed exception taxonomy your resolvers already use. Map each type to an owner team, SLA, and next action. Align IDs with whatever Onfleet tasks, Bringg exceptions, project44 milestone codes, or Salesforce case record types you already report on. Renaming types in the model without updating those systems creates a second, unofficial taxonomy.

Prompt or fine-tune against real notes from your network, not generic parcel language. Include regional shorthand, apartment access phrases, and carrier-specific status language. Require span citation in the response schema so operators can audit a sample every day without opening raw model traces.

Wire classification at the moment the stop is marked failed or exceptioned, or on a short debounce after the note and photo upload complete. Write the type ID and citation back to the delivery task and, when a case exists, to Salesforce. Use that typed field as the routing key for queues, not the free text.

Measure speed as time from exception capture to correct-queue assignment, plus reclassification rate and empty rate. A fast wrong label is worse than a slower empty. Track override reasons by type so you can fix vocabulary gaps (for example, splitting “access” into gate code versus building closed) without growing an unbounded label set.

Keep vendor boundaries clear. Onfleet and Bringg remain the system of record for the stop. project44 remains the visibility layer when the exception must surface to shippers. Salesforce remains the customer conversation layer. The classifier is a decision aid on top of those systems, not a parallel exception database.

LLMs over-interpret sparse notes. A photo of a porch with no package can look like a successful leave-at-door or like a theft risk depending on prior context the model does not have. Prefer empty over speculative labels when evidence is weak.

Photos introduce privacy and quality issues. Blurry images, night shots, and faces or house numbers in frame need the same retention and access controls you already apply to proof-of-delivery media. Classification should not require storing a second unbounded image archive.

Status codes and notes can conflict. If the app says “delivered” and the note says “refused,” escalate to a human rather than auto-resolving the contradiction. The classifier can flag conflict; it should not silently pick a side.

Multilingual fleets need explicit coverage. A model strong on English driver notes may fail on Spanish or Polish shorthand common in a market. Test per region before turning on auto-routing.

Finally, remember the outcome is speed of routing to the correct resolver. Success is fewer minutes in the wrong queue and clearer audit trails via cited note spans. Success is not unsupervised closure of customer-impacting exceptions. Pair classification with resolution agents and human override, and keep empty as the honest answer when the note is too thin to trust.