Skip to main content
DoneThat

AI Adoption GuideResearchDecide

Probabilistic decision register

Capture decisions as probability distributions, not binary

Research processScopeSourceTriageShallow AssessDeep AssessDecideHand Over

By Don, DoneThat’s AI coach · updated

A register entry holds a decision, a probability, and the condition attached to it

Committees record decisions as yes or no. The yes everyone argued about and the yes that took four minutes look identical in the minutes. A probabilistic register keeps the difference.

Each entry carries five fields. The decision as made, the person who made it, the date. A probability that a named outcome occurs by a named date. And the condition the decision-maker would want revisited if it changed.

The probability is not a score of the candidate. It is a statement of how confident a person was, at a moment, about one specific thing. That is a narrow claim and it needs to stay narrow. Registers fail when the number starts meaning quality.

Be honest about what this buys in the short term, which is very little. The first year produces a table nobody reads. The value arrives when outcomes land and someone compares them with the recorded confidence. Adopt it as a small habit, not as a program.

Define the outcome the probability refers to, or record nothing

A probability with no attached outcome is a mood ring. Seventy percent confident about what, measured how, by when. Without those three, the entry cannot be checked and should not be stored. An empty confidence field is more honest than a number with no referent.

Write the outcome as a sentence somebody could verify without asking the author. "The assay reaches the agreed sensitivity threshold in the validation study" works. "The program goes well" does not. Keep one outcome per entry, even when the decision rests on several.

Multiple outcomes get multiple entries under the same decision. Splitting them is the whole benefit, because confidence is rarely uniform. A committee can be highly confident the science works and doubtful the supply chain holds. One blended number destroys that information.

Decide the scale before the first entry and never change it mid-register. Percentages are fine. So are ranges, and so is a small set of labeled bands. What matters is that an entry from three years ago means the same thing today.

State what to do when nobody will commit to a number. Record the refusal and the reason, rather than pressing for a figure. A committee that cannot state its confidence has told you something useful about the evidence.

Elicit the number from the people who decided, and store the reasoning with it

The number comes from humans, not from a model reading the assessment. A model that infers confidence from the tone of a memo has invented an opinion. Ask the decision-maker directly, at the moment of the decision, in one question. Delay of even a week reshapes the answer around what has happened since.

Capture the reasoning in the same breath. One or two sentences on what drives the confidence, and what would move it. That text is the field people actually return to. A bare number tells a future reader nothing they can act on.

Where the committee split, record the spread rather than a consensus figure. Two members at thirty and three at seventy is not five members at fifty-five. The spread is a signal about the evidence, and averaging discards it.

Language models are useful in a narrow supporting role here. They can draft the outcome sentence from the assessment and propose a resolution date. They can check that an entry has all five fields. They should not supply the probability.

Meeting capture makes elicitation cheaper. Where a session already produces positions and the assumptions underneath them, the outcome sentences are often sitting in that output. The assumption people argued about is usually the thing worth putting a number on.

Register three diagnostic assay decisions with different shapes of doubt

The following is hypothetical, with invented figures. A diagnostics organization's portfolio committee is advancing assays into validation. Amara Boateng chairs the committee and owns each decision. Three decisions are made in one session.

The first is a clear yes on a respiratory panel. The entry records ninety percent confidence that the panel meets its sensitivity threshold in validation. Resolution lands in nine months. The reasoning field says the assay has already passed an equivalent internal study. Nothing about this entry is interesting, and it takes two minutes.

The second is a yes that took an hour. A rapid antigen format has strong laboratory data and an untested manufacturing route. Two entries are recorded rather than one. Performance in validation sits at eighty percent; manufacturing yield at target cost sits at thirty-five. The condition names a supplier qualification step due in four months.

The third is a no, and it is registered anyway. A multiplex platform is deferred. The entry records sixty percent confidence that a competing approach makes it unnecessary within two years. Registering rejections is what makes the register worth keeping.

Four months later the supplier qualification fails. The condition on the second decision fires and the committee revisits it. Nobody had to remember why. That single event is most of the return on the register.

False precision, drifting definitions, and probabilities used as cover

Numbers look more considered than they are. A committee that says sixty-three percent is not more informed than one that says roughly sixty. Round to a coarse scale and resist the pull toward precision. Precision invites arithmetic that the underlying judgment cannot support.

Definition drift breaks the register slowly. Year one asks about technical success; year three quietly asks about commercial success. The entries look comparable and are not. Store the outcome definition inside each entry rather than in a shared template that gets edited.

Cheap entries become the norm if nobody enforces the fields. Fifty percent with no reasoning is a way of appearing to comply. Reject entries with an empty reasoning field the same way you would reject a blank decision date.

The worst use is defensive. A committee that assigned forty percent and proceeded can later claim it flagged the risk. That is only true if the number changed something at the time. Require any entry below a stated confidence to carry a mitigation or an explicit acceptance.

Low participation makes the register a personal diary. If one member records confidence and the rest do not, the entries describe that member. Either the committee records together, or the field says whose number this is.

The named decision-maker supplies the number and owns the call

A probability is a description of a decision, never a replacement for one. Amara approves the assay, and the register records that she did. No threshold in the register can approve or block anything on its own. A rule that rejects everything below a confidence line has converted a judgment into a gate.

Low confidence is a legitimate reason to proceed. Cheap, reversible, and informative options deserve a yes at forty percent. Expensive, irreversible ones may deserve a no at eighty. The register captures the confidence; the decision-maker weighs it against cost and reversibility.

Never let the number travel without its outcome sentence. A confidence figure lifted into a summary slide becomes a claim about the candidate. Where material is repackaged for different readers, briefs whose caveats cannot be hidden keep the definition attached.

The register only pays off if entries resolve. Somebody has to close them, on a date, against a stated source. The scoring machinery for that already exists in a registered forecast resolved at a fixed horizon. Resolved entries then feed tracking of systematic over- and under-confidence per rater.

Keep the register where the next committee will find it. Entries stored in a shared drive nobody indexes are entries nobody reads. A searchable store across rounds of decision artifacts is where these belong. The point is that a successor can ask how sure the last committee actually was.

Is this worth automating for you?

Whether this pays back depends on how much time it takes your team today. Most teams estimate that from memory, and the estimate is usually wrong in one direction or the other.

DoneThat reconstructs where the time actually went, with no timers to forget, so you can measure the baseline before committing to a project and check the gain afterward.

Measure the baseline first