A business leader receives two proposals. One promises to predict which customer orders will arrive late. The other offers an assistant that drafts messages to customers when delays occur. Both use AI, but they change different parts of the service. The first needs evidence about forecasting and available interventions; the second needs reliable order information, suitable wording, and controls over what gets sent.
Understanding this distinction is more useful than treating AI as a single investment category. Leaders need enough technical literacy to ask what a system does, alongside enough knowledge of the business to decide whether that capability addresses a worthwhile problem. The investment decision then concerns a complete service, including the people, data, controls, and maintenance needed to make the output useful.
Understand the Capability Being Purchased
Artificial intelligence is a broad term for machine-based systems that infer how to produce outputs such as predictions, recommendations, content, or decisions. The OECD’s explanation of its updated AI-system definition recognises variation in autonomy and adaptation. It does not imply that every AI product learns continuously or operates without people.
For a business proposal, identify the concrete capability. Classification assigns categories; prediction estimates an outcome; generation produces candidate material; optimisation searches for a plan against objectives and constraints. A product may combine these with conventional software. Its AI label does not tell you which part is responsible for the claimed benefit.
Ask what is included in the purchase. A model API, a managed application, and a fully operated service leave different responsibilities with the customer. Someone still needs to establish data access, assess suitability, support users, handle failures, and manage changes. Procurement should make the division explicit rather than relying on a general promise of automation.
The introduction to AI systems explains the underlying concepts. A leader does not need to reproduce the training algorithm, but should be able to describe the proposed input, output, action, and evidence of suitability in ordinary language.
Choose a Problem With an Available Response
For the late-order model, ask what the organisation can do with an earlier warning. It might prioritise a warehouse task, obtain a replacement item, or give a customer a more useful estimate. If none of those actions is available, a more accurate forecast may have limited operational value.
Define the problem using the current process. Which delays are avoidable? How often do staff discover them too late? What does the resulting failure cost customers and the organisation? A measured baseline helps distinguish an attractive demonstration from an improvement worth pursuing.
Compare with feasible alternatives. Better stock records, supplier reporting, clearer escalation, or a conventional rule may solve part of the same problem. AI may still be useful alongside those improvements. The business case should identify the contribution expected from each change rather than treating all subsequent progress as a model benefit.
Consider opportunity cost at the portfolio level. A promising pilot may require the same integration team needed to repair an unreliable order feed. Funding both on paper does not create delivery capacity. Leaders should compare the whole commitment of scarce people, budget, and managerial attention with other work the business needs.
Review overlapping proposals together. Several departments may be purchasing assistants that require the same customer records, identity controls, and support capability. Shared infrastructure can reduce duplicated work, but a single platform can also concentrate dependency and delay projects with different requirements. Ask which capabilities should be shared, which need independent evaluation, and who pays for the continuing service. Portfolio oversight should reveal these commitments before separate pilot budgets turn into an unfunded common operating burden.
Decide What Evidence Would Change the Investment
A useful evaluation has a decision attached to it. The team might be seeking permission for a limited internal trial, a customer-facing release, or expansion to another region. These stages require different evidence. A successful demonstration can justify further investigation without establishing readiness for unrestricted production.
For the forecasting proposal, test the horizon at which staff can still intervene and the information available at that time. For the drafting assistant, test whether accepted messages use current records and avoid unsupported promises. Include difficult, incomplete, and out-of-scope cases rather than selecting only examples the supplier handles well.
Specify acceptance conditions before viewing the results. Some are improvement targets, such as less handling effort. Others are requirements, such as preventing unauthorised disclosure. A weighted score should not allow a large productivity estimate to cancel a blocking security or legal issue.
The AI readiness assessment connects evidence with a proposed stage of use. Its purpose is to identify what needs to be demonstrated next, rather than award the organisation a general maturity label that conceals differences between use cases.
Fund the Operating Model Alongside the Product
Estimate implementation with the teams who will deliver it. Work may include source-data preparation, identity controls, integration, user interfaces, validation, evaluation, and changes to procedures. The distribution of effort varies; there is no dependable universal ratio between model work and data work.
Recurring responsibilities include monitoring the service, supporting users, responding to incidents, and assessing material changes. Some models may need retraining; others may remain suitable without frequent updates. A change in performance should be investigated before assuming that retraining is the appropriate remedy.
For the late-order service, a business owner needs to manage the intervention process, technical owners need to maintain the data and model path, and support staff need a response when the recommendation is unavailable or disputed. Name these responsibilities and give them the resources required to perform them.
Include an exit route. The organisation may need to export case history, replace a supplier, or continue work without the AI feature. A service that saves time while making replacement prohibitively difficult creates a dependency that belongs in the approval decision.
Separate Value From an Attractive Calculation
Time released is not necessarily expenditure removed. If the assistant saves staff several hours, the benefit might be additional capacity, faster service, or lower overtime. Each has a different route to financial value. Identify who will make the changes needed to realise the proposed benefit.
Measure review and correction as well as generation. A draft produced instantly can still take longer to verify than a message written from an approved template. Conversely, a tool that leaves headcount unchanged may improve customer outcomes enough to justify its cost. The comparison should follow accepted work and the business objective.
The AI ROI guide distinguishes these benefit types and includes costs beyond licensing. Use ranges where inputs are uncertain, and identify the assumptions that would reverse the decision. A precise percentage should not obscure weak evidence about adoption, workload, or quality.
Avoid treating industry research as a default multiplier. A study of one task and workforce can establish that a particular gain occurred under those conditions. It does not establish the return for another application with different data, users, review requirements, and commercial arrangements.
Keep Risk and Responsibility Specific
Examine the consequences of a wrong output, a disclosure, an unavailable service, and an unauthorised action. The customer-message assistant needs different controls depending on whether it only suggests text or can send a binding offer. Authority matters alongside predictive or generative quality.
NIST’s AI RMF Core connects governance, context, measurement, and management across the lifecycle. For a business leader, the practical questions include who approves the use, which limits apply, how failure will be detected, and who can restrict or stop the service.
Legal assessment should concern the actual deployment, data, sector, jurisdiction, and affected people. In New Zealand, the Privacy Commissioner’s AI guidance explains how existing information-privacy requirements apply when AI uses personal information. A new technology label does not remove those obligations.
Assess suppliers on the evidence and access they provide. Ask about model updates, data retention, permitted use, incident assistance, performance information, and contractual responsibilities. Vendor certifications can support due diligence, but they do not establish that the organisation’s particular process is appropriate.
Approve a Bounded Commitment and Review It
An approval should state the intended use, scope, evidence reviewed, operating owner, limits, and conditions for reassessment. Record what remains uncertain and how the next stage will address it. This makes it possible to proceed with a useful bounded trial without presenting every open question as settled.
Give employees and affected users an appropriate role in evaluation. Staff can identify missing context and unrealistic review workloads; customers may reveal problems with the service that internal metrics miss. The workforce-preparation article examines how participation, training, and role changes support that work.
After release, review accepted quality, operating effort, costs, and relevant outcomes. If the forecast improves but interventions do not happen, examine the process before purchasing another model. If the assistant creates unsupported promises, narrow its authority while investigating the failure. Expansion should follow evidence about the complete service.
Business leaders should care about AI because it can change what the organisation can predict, produce, and coordinate. Their role is to connect that capability with a worthwhile purpose and a maintainable operating arrangement. The resulting decision should be understandable enough to fund, evaluate, and revisit when the evidence changes.
