An accounts team introduces an assistant that proposes how invoices should be coded. Accepting a recommendation takes a few seconds. Checking it can require opening the purchase order, finding the relevant contract, and asking a project owner which budget should carry the expense.
The team remains responsible for accuracy, but its daily workload assumes that most recommendations will be accepted quickly. A reviewer who investigates an uncertain case processes fewer invoices, while a mistaken allocation may not be discovered until the monthly reconciliation. The process rewards speed immediately and reveals some of its costs later.
This is one route by which reliance on automation can become attractive even to conscientious staff. It does not mean checking is always uneconomic or that employees deliberately disregard quality. It means the organisation needs to account for the effort required to verify an answer, as carefully as it accounts for the time saved in producing one.
Generation and verification are different kinds of work
A system can produce a proposed allocation without retrieving everything a reviewer needs to justify it. The reviewer may then have to reconstruct the evidence and the applicable rule. An explanation that sounds plausible can be much quicker to read than the source material is to check.
The relevant comparison is therefore between complete workflows. Measure how long it takes to reach an adequately supported decision, including evidence retrieval, interpretation, corrections, and escalation. Timing only the moment when the assistant displays its answer leaves much of the human work outside the calculation.
Verification can also be narrower than repeating the original task. A reviewer may be able to check a decisive contract clause or reconcile a known total. Where the system can expose those points clearly, review becomes cheaper and more informative. Where the proposal contains many interdependent assumptions, verifying it may require substantial reconstruction.
A 2025 study by Lee and colleagues surveyed 319 knowledge workers about generative AI use. Higher confidence in AI was associated with less reported critical-thinking effort. The study also described work shifting towards verification and integration. It measured reported experience and associations, not a causal law that AI use reduces everyone’s ability to think.
Workload can make the promised review impossible
Suppose a team receives 200 cases a day and meaningful checking takes an average of six minutes per case. Full review requires 1,200 minutes, or 20 reviewer-hours, before administration and interruptions. These are illustrative planning assumptions, not research estimates.
If the organisation allocates eight hours of review capacity, a policy requiring every case to receive that level of checking does not fit the plan. Staff may perform shallower checks, accumulate a queue, or work longer. Calling the process human-reviewed does not resolve the mismatch.
Observe the work to refine the assumptions. Some cases may be checked quickly, while a smaller number require lengthy investigation. Include that variation and the effect of escalations. A plan based only on the median case can leave the team repeatedly short of time when several difficult cases arrive together.
The design choices include reducing unnecessary evidence retrieval, narrowing the automated task, reserving specialist capacity, or changing which cases require which checks. The right combination depends on the consequences of error. Merely instructing reviewers to be more vigilant leaves the workload unchanged.
Distinguish personal incentives from organisational costs
A fast approval may help a reviewer meet a daily target even when it creates later reconciliation work for another team. The cost has moved between people and periods. A business case that counts the first team’s saved minutes while excluding the second team’s corrections can report a misleading gain.
The allocation of credit and responsibility matters too. If agreement is treated as efficient work but every override requires extensive justification, the process places a higher immediate cost on disagreement. If staff are blamed for any accepted error while receiving no time to investigate, they face incompatible demands.
HSE’s guidance on managing human failures identifies workload, time pressure, interface design, and other performance-influencing factors. Its setting is workplace safety. The transferable point is to examine the conditions of the task rather than treating motivation or a reminder as a complete explanation of performance.
For the invoice team, investigate which records are difficult to obtain, what happens to a case marked uncertain, and how an investigation affects the reviewer’s workload assessment. Those details give managers something they can change. They also help distinguish overreliance from a reasonable decision to accept a well-supported recommendation.
Reduce the cost of a useful check
Bring relevant evidence into the review path without replacing it with another unsupported summary. An extracted contract clause should link to its source and identify the version used. A proposed coding category should expose the rule connecting the expense to that category. Missing evidence should remain visible.
Keep the checking task focused. Instead of asking whether an entire generated explanation looks right, the interface might ask the reviewer to establish which project incurred the expense and whether the selected account matches the current coding policy. These are questions with identifiable evidence and an operational consequence.
An explanation can still be useful for orientation. The distinction is whether it helps the reviewer perform a check or merely gives the proposal a more persuasive presentation. Review the effect through actual cases, including incorrect recommendations, rather than assuming that adding text improves scrutiny.
Research by Buçinca, Malaya, and Gajos found that cognitive forcing designs reduced overreliance relative to simpler explanation-based designs in their experiment, but participants rated the more effective designs less favourably. The study illustrates a possible trade-off between checking and immediate ease of use; it does not identify one interface suitable for every task.
Allocate scrutiny without trusting the system’s confidence alone
Different cases can justify different levels of review. An uncertain allocation that can be corrected internally may need a different process from a payment instruction whose effects are difficult to reverse. Define the consequences and the checks that can reduce them before deciding how to distribute effort.
| Source of review cost | Design question |
|---|---|
| Searching for evidence | Can the relevant source be reached directly? |
| Reconstructing policy | Is the applicable rule and version identifiable? |
| Resolving exceptions | Is there a staffed route for uncertain cases? |
| Documenting disagreement | Does the record capture useful evidence without needless repetition? |
| Correcting downstream effects | Can the operation be held or reversed before costs spread? |
Do not let model confidence be the sole route into review. A system can be confidently wrong about a recurring case type. Include sampling and other triggers grounded in known consequences, unusual inputs, changed policies, or missing evidence. Sampling accepted recommendations can expose errors that would otherwise never enter the exception queue.
Any selective review policy leaves residual uncertainty. State what the checks cover and what evidence supports the allocation of effort. For consequential decisions, the organisation may need to redesign the task or restrict automation if effective verification cannot be provided at the planned scale.
This connects to meaningful human oversight: responsibility requires usable information, time, competence, and authority. Increasing the number of nominal approvals does not necessarily increase those resources.
Evaluate the value of the complete process
Assess cost per adequately resolved case, while keeping quality and consequences visible. Include time spent reviewing, correcting, investigating complaints, maintaining the integration, and resolving delayed failures. Separate genuine savings from work transferred to another team or to the person affected by the decision.
For the invoice example, a misallocation might require a supplier to explain the same expense repeatedly while payment remains unresolved. That effort belongs in the assessment even if it never appears in the accounts team’s timesheet. Collect evidence from people downstream of the reviewed decision, including those asked to supply missing information or dispute an outcome. Also retain constraints that cannot sensibly be reduced to an average processing cost: some actions require a particular verification step regardless of whether skipping it appears faster across the queue.
A pilot can compare review arrangements using representative cases and a separately established reference assessment. Record errors caught, errors missed, time used, escalation demand, and the consequences of incorrect action. An arrangement that is faster and finds fewer important errors may require a different decision from one that saves time by making evidence easier to check.
Keep acceptance rate out of the role of quality label. Accurate automation may deserve frequent acceptance, but routine acceptance can also conceal a weak review process. Metrics that distort judgment are particularly relevant when employees learn that disagreement makes their reported performance worse.
Review these conditions after volume increases or the task changes. A process that was viable during a small pilot may become a different job when case arrivals double. The organisation should revisit its staffing, routing, and promised safeguards at that point.
The economic question is how to make warranted reliance achievable within the work. Automation can reduce effort, including verification effort, when it exposes the right evidence and supports useful checks. Those gains need to be demonstrated across the complete process, with enough capacity to investigate the cases where accepting the recommendation would be a mistake.
