An engineering company wants an assistant that helps staff find maintenance instructions. The demonstration looks straightforward: enter a question, receive an answer, and follow a link to a manual. To make that useful at work, the company must decide which manuals are current, which employee can access them, how the right passage is retrieved, and what happens when the answer is incomplete.
Each of those decisions belongs to a different part of the application. A stronger language model might improve the explanation while leaving an outdated manual in the search index. A faster server might reduce delay while leaving access restrictions unenforced. Understanding the components helps the team locate the actual problem before choosing a remedy.
This article follows those components through a document assistant. It complements an introduction to AI capabilities by explaining how a model becomes part of an operational application.
Data Has Different Roles in the Same System
Training data contributes to the model’s learned behaviour. Evaluation data helps assess performance. Runtime data supplies the information used for a particular request. These roles may involve different sources, owners, and permissions; treating them as one pool obscures important choices.
The engineering company may use a model trained by an external provider and supply maintenance manuals only when staff ask questions. In that arrangement, the manuals need not become training data. Whether the provider retains them or uses them for another purpose depends on the service’s configuration and terms, which the company should verify separately.
Operational data also needs interpretation. Two manuals may describe similar equipment with different safety procedures. An old document may remain useful for a retired machine while being unsuitable for its replacement. A document’s date alone may therefore be insufficient to determine whether it applies. Equipment identifiers, revision status, and intended use need to travel with the content.
Source ownership matters because the assistant cannot resolve every contradiction reliably. If a maintenance bulletin supersedes a manual, someone needs to record that relationship. Otherwise retrieval may return both documents and leave the model to reconcile a disagreement that should have been settled in the source information.
Models Turn Representations Into Outputs
A model is a computational structure with parameters that determine how inputs are transformed into outputs. Different tasks use different model families. Decision trees can split records using learned conditions; neural networks use layers of mathematical operations. A large language model is one kind of neural model, rather than a name for all AI.
Inputs usually need preparation. A forecast may use numerical and categorical features. A language application converts text into tokens. Search systems may use embeddings: numerical vectors whose relationships help represent similarity for a task. Google’s guide to obtaining embeddings explains how these representations can be learned.
Representations are useful without being complete descriptions of meaning. Two passages may be close in an embedding space because they discuss similar equipment, while differing on the one restriction relevant to the question. Retrieval quality needs testing against actual information needs, especially where near matches could be misleading.
The model’s output is also only part of the application result. A generated paragraph, a classification score, and a structured set of extracted fields need different validation. The document assistant should preserve the link between its answer and the applicable source passages so that a staff member can inspect the evidence.
Training and Adaptation Change Different Things
Training fits model parameters to an objective. For a neural network, optimisation commonly uses gradients to adjust those parameters. Backpropagation computes gradients through the network; the optimiser uses them to update the weights. These mechanisms describe how learning proceeds, rather than establish that the chosen objective captures everything the organisation values.
The engineering company does not necessarily need to train a model. It could start with a pretrained service and test whether instructions and document retrieval produce acceptable results. That route still requires engineering and evaluation, but avoids assuming that a custom training programme is the first step.
Fine-tuning continues training an existing model, generally using data selected for a more specific purpose. Prompt design changes the instructions and context supplied during use. Updating a document index changes what can be retrieved. Google distinguishes fine-tuning from prompt engineering; confusing them makes it harder to understand what a proposed change will affect.
For the assistant, outdated maintenance advice first calls for an investigation of source currency and retrieval. Fine-tuning the model would not automatically remove an obsolete manual from the index. The appropriate intervention depends on where the failure entered the process.
Retrieval Connects the Model to Current Sources
Retrieval-augmented generation, usually shortened to RAG, finds relevant information and supplies it as context for generation. A typical document application prepares content for search, retrieves candidate passages for a question, and asks the model to answer using those passages. Microsoft’s RAG design guide treats preparation, retrieval, and evaluation as separate design concerns.
The engineering company needs to test each stage. Has the relevant document been ingested? Did splitting it into passages separate a warning from the instruction it qualifies? Did retrieval select the applicable equipment revision? Did the model preserve the warning in its answer? Looking only at the final paragraph makes these failures harder to distinguish.
RAG can provide access to information that was not present in model training, but it does not guarantee a correct answer. Search can miss the right material, sources can disagree, and generation can introduce unsupported content. A citation is useful only when it supports the claim being made and refers to a source the user is permitted to inspect.
Compute Shapes the Experience and the Economics
Compute includes the processing, memory, storage, and network resources used to train and run the system. A hosted service packages some of those resources behind an API. It still imposes operational constraints such as request limits, response times, and availability dependencies.
The assistant’s cost includes document preparation, search, generation, and repeated calls when a response needs correction. Longer source passages may improve context for some questions while increasing processing time and cost. The team should measure the result for its workload rather than assume that the largest available model or longest prompt is always preferable.
Capacity testing should resemble the intended use. A response that arrives promptly for one engineer may slow considerably when many staff start work at the same time. Queues, caching, and smaller models may help, but each changes behaviour. A cached answer must still apply to the current document version and the current user’s permissions.
The useful financial measure is tied to accepted work: for example, cost per question answered with sufficient evidence. Cost per model call leaves out failed searches, retries, and the time engineers spend checking or correcting the response.
The Application Determines What Happens Next
The application layer receives requests, checks identity, retrieves permitted information, calls the model, validates outputs, and presents results. It is also where the organisation implements decisions about approvals and escalation. These responsibilities should have identifiable owners rather than be left implicitly to the model provider.
For the document assistant, permissions should apply before protected material reaches the model. Asking the model not to reveal a restricted passage is weaker than ensuring the user cannot retrieve that passage through the application. If the assistant later gains tools that can schedule maintenance, authorisation for those actions needs separate consideration.
Output handling needs equivalent care. A generated equipment identifier should be checked against a recognised record before it becomes part of a work order. An answer intended as guidance should not silently become an instruction that changes operational state. The legacy integration guide develops these boundaries in more detail.
Evaluate the Connections as Well as the Components
An evaluation set for this assistant should include questions staff genuinely ask, ambiguous equipment references, superseded instructions, restricted documents, and questions the available manuals cannot answer. Review should examine source selection, factual support, completeness, and whether the response helps the engineer take the appropriate next step.
When the team changes a model, prompt, retrieval setting, or source collection, it should repeat the relevant checks and retain a record of what changed. Monitoring then needs to reveal problems that emerge during use, including missing documents, repeated corrections, and access failures. Google’s production ML guidance places the model within this broader system of data, serving, testing, and monitoring.
The team also needs a way to isolate a failure. A slow answer may originate in document search, model processing, or a source system waiting for access checks. Recording timings and errors at those boundaries helps the operator investigate without assuming every problem belongs to the model.
Operational ownership should follow those boundaries too. A document owner maintains authoritative content, an application team manages retrieval and permissions, and a service owner coordinates incidents. In a small organisation, the same person may hold several responsibilities. The essential point is that a reported defect reaches someone who can investigate and change the relevant component.
A component view makes improvement more precise. If the source is missing, maintain the source collection. If retrieval chooses the wrong revision, improve selection. If the model misstates an accurate passage, investigate generation and review. A dependable AI application comes from making these connections work together and retaining enough evidence to know which one needs attention.
