A replenishment team uses an AI assistant to recommend weekly stock orders. Planners review the proposals, adjust a few quantities, and move on to the next cycle. Stores handle shortages and excess stock, while purchasing staff negotiate with suppliers. Several months later, each group has accumulated experience, but nobody can readily connect the original reasoning to the consequences it produced.
The planners see aggregate sales and inventory figures. They rarely see which assumptions failed in particular orders, which shortages were caused by delivery problems, or which interventions prevented a larger loss. They can become faster at processing recommendations without becoming better at judging when those recommendations need to change.
Experience improves judgment only when there is something useful to learn from it. Repetition, confidence, and responsibility do not guarantee that the work supplies accurate feedback. An organisation needs to examine which consequences are observable, who receives them, and what conclusions those observations can support.
Define what the decision was trying to achieve
A stock-order decision balances several concerns: availability, working capital, storage, spoilage, and the supplier’s delivery constraints. A single outcome measure cannot automatically establish whether the decision was good. Strong sales might coexist with costly excess inventory, while low inventory could reflect efficient planning or missed demand.
Record the objective and important assumptions before the outcome is known. For a particular order, that might include expected demand, the delivery window, an upcoming promotion, and the consequence of running out. The record should be short enough to maintain and specific enough to interpret later.
Kahneman and Klein’s analysis of intuitive expertise emphasises the predictability of the environment and the opportunity to learn its regularities. The point is not that every business decision has a learnable pattern. Where relationships are unstable or feedback is weak, experience may provide less support for confident intuition than the number of decisions suggests.
For the planners, ask which judgments can be assessed repeatedly under reasonably comparable conditions. Predicting the effect of a routine promotion may offer different learning opportunities from anticipating a unique disruption. Keep that distinction visible when evaluating expertise.
An outcome is evidence, not a verdict
A well-supported order can turn out badly after an unforeseeable delivery failure. A poorly reasoned order can turn out well because demand unexpectedly increases. Assessing the decision solely through its realised outcome encourages the wrong lesson in both cases.
Separate information available at the time from information learned afterwards. Examine whether the planner considered relevant evidence, applied the agreed constraints, and identified uncertainty. Then examine what the outcome reveals about the assumptions or method. The two assessments can disagree without either being meaningless.
Suppose a planner increases an order because a promotion is expected, but the promotion is cancelled after the order is committed. The excess stock does not alone establish that the planner should have ordered less. It may reveal a need to connect promotional changes to purchasing before commitments become difficult to reverse.
Likewise, an urgent manual transfer that prevents a shortage can hide a planning error in the final availability figure. Preserve relevant interventions so the team can distinguish a robust initial decision from a successful recovery. Otherwise the recovery work can keep the underlying weakness invisible.
Notice what the data cannot reveal
If a store stocks 40 units and sells all 40, the record does not establish that demand was exactly 40. Customers who could not buy may leave no equivalent sales transaction. The observed result reflects the stocking decision as well as demand.
Research on demand estimation from censored observations addresses this problem directly. Its relevance here is that the action can limit what is subsequently observed. Feeding recorded sales back as if they were complete demand can reinforce an incorrect understanding of the market.
Other feedback channels are selective too. Complaints reflect people who complained, while reopened cases reflect problems that returned through the recorded route. These observations can reveal important failures without representing every affected case. Absence from an exception queue should not automatically become a positive quality label.
AI-assisted processes can introduce another selection step when the system decides which cases deserve review. Inspecting only flagged orders can teach the team about flagged cases while leaving unflagged errors unexplored. Include some review of ordinary accepted decisions and document the limits of the sample.
Connect records across the actual handoffs
A decision record does little if the outcome cannot be linked to it. Preserve identifiers across ordering, supplier confirmation, delivery, and store handling. Where a quantity changes, record who changed it, when, and the relevant reason. This allows a later review to examine the sequence rather than compare two disconnected totals.
The review needs participants who understand the different parts of the process. A store manager may know why stock was moved, while purchasing knows why the supplier substituted an item. Neither perspective alone necessarily explains the complete outcome.
Choose cases for the learning question rather than presenting every incident in a large meeting. A set of routine decisions, known errors, and successful recoveries can help expose different mechanisms. Retain unresolved explanations instead of manufacturing a single root cause to complete the discussion.
The operational connection is similar to the one examined in a control-theory view of automation bias. Here, the focus is how people improve their future judgments through that connection, rather than only how a current process is corrected.
Give feedback a form people can use
A monthly percentage can establish that performance changed without showing a planner what to do differently. Useful feedback often needs a case, a relevant comparison, and an explanation of what the evidence supports.
| Feedback received | What it can support | What remains unresolved |
|---|---|---|
| Total stockouts increased | Investigating a service-level change | Which decisions or delivery failures contributed |
| A specific assumption was wrong | Revising that assumption or checking its source | Whether the error generalises to other cases |
| An override improved one outcome | Examining the override’s reasoning | Whether the same rule helps across comparable cases |
| A case required recovery work | Finding a concealed initial weakness | Whether the recovery itself introduced other costs |
Deliver the lesson while the evidence can still be reconstructed. Some consequences take months to emerge, so a process may need an early check of assumptions and a later outcome review. Label those stages accurately. An early indicator is useful without pretending to be the final result.
Make learning actionable through another attempt. A planner can apply the revised method to a comparable case or a controlled replay and explain what changed. Reading an incident summary may inform them, but it does not establish that they can use the lesson in a new decision.
Protect uncertainty from performance incentives
If the review rewards confidence and punishes visible mistakes, staff have a reason to write vague assumptions, avoid documenting uncertainty, or attribute every bad result to external events. That weakens the evidence available for learning even when the organisation runs frequent review meetings.
Agree how records will be used and distinguish an honest, supported judgment from a departure from the required process. Accountability still matters, but an outcome label should not stand in for reconstructing what someone knew and did. The review needs enough specificity to identify a correction without making retrospective certainty the standard.
Metrics that distort judgment can also shape what gets reported. If manual recoveries count only as delays, the team may conceal work that reveals an important planning weakness. If overrides are treated as failures, staff may stop recording useful disagreement.
A healthy learning process should be able to say that the evidence is insufficient. Some outcomes have several plausible explanations, and some comparisons are not credible. Preserve that uncertainty and identify what additional observation would help. Inventing a lesson is another way for a review to fail.
Maintain judgment as the environment changes
A successful lesson can become obsolete when a supplier changes its lead times, customer behaviour shifts, or a new policy alters the decision’s constraints. Record the scope of the lesson and revisit it when relevant conditions change. Repeatedly applying a once-useful rule is not necessarily continued learning.
Automation may help by connecting records, identifying cases, and surfacing earlier assumptions. Check its interpretation against the sources, especially where the same system both recommends the action and evaluates its success. A generated retrospective explanation should remain an account to examine.
Maintaining organisational expertise includes funding this review work. Without time to reconstruct cases and try a changed approach, the feedback loop may exist on paper while daily processing remains unchanged.
The goal is a working relationship between decision, evidence, and revision. People should be able to explain what they expected, what happened, what remains uncertain, and what they will test next. That gives experience a chance to improve judgment without treating every outcome as an unambiguous lesson.
