AI Centre of Excellence and Federated Models: Where Decisions Should Sit

A company establishes an AI centre of excellence to help departments choose tools and develop their first applications. Demand grows. The central specialists now divide their time between supplier reviews, training, platform support, and delivery requests. Business teams begin buying their own tools because they cannot obtain help quickly enough.

Moving delivery into departments may relieve the queue, but it creates further questions. Who maintains shared integrations? Which controls can a department change? Who pays for the platform? A new organisation chart will leave those questions unresolved unless it comes with a workable allocation of decisions and responsibilities.

An AI centre of excellence concentrates expertise or coordination in a central group. A federated model distributes responsibilities across business units while retaining some shared standards, capabilities, and oversight. Neither term specifies exactly who develops applications, approves risk, or operates services. Those details determine how the model works.

Distinguish a Federated Operating Model From Federated Learning

The phrase federated AI has two meanings. A federated operating model concerns organisational design: business units own use cases, product roadmaps, and local delivery within shared guardrails. Federated learning concerns technical architecture: participants train on local data and contribute updates towards a shared model rather than pooling their raw training data. An organisation can use either approach without using the other.

The federated learning research by McMahan and colleagues describes learning a shared model by aggregating locally computed updates while training data remains distributed. In a typical arrangement, a coordinator sends a model to participating devices or institutions, they train locally, and the coordinator aggregates their updates. That is a different decision from assigning application ownership to business units.

Keeping raw data local does not by itself guarantee privacy or compliance. NIST explains that individual model updates can expose information and describes secure aggregation as one way to protect those updates. A technical federated-learning proposal therefore needs its own security, privacy, and evaluation design. The remainder of this article focuses on organisational responsibilities.

Separate the Responsibilities Before Comparing Models

AI work includes several responsibilities that can sit in different places: setting organisational policy, selecting shared platforms, maintaining data access, evaluating systems, delivering applications, approving use, supporting production, and managing suppliers. Treating all of these as one centralisation decision obscures useful combinations.

A central team might operate a model gateway and evaluation service while departments own their applications and outcomes. Another organisation might concentrate development in a shared team while business units retain process ownership. Both arrangements could describe themselves as having a centre of excellence.

Begin with a real use case. Suppose a retail group wants an assistant for customer-service staff and an application supporting warehouse planning. The two share identity services, supplier relationships, and some security controls. They differ in source data, failure consequences, users, and the people who can judge whether the output is useful.

The operating model should preserve access to that local knowledge while avoiding unnecessary duplication of capabilities that are genuinely shared. That requires examining each responsibility rather than assuming one reporting structure is best for the whole lifecycle.

A Centre Can Concentrate Expertise and Reduce Repeated Work

A central group can be useful when experienced people are scarce or several departments need the same foundation. It can establish approved service patterns, negotiate supplier terms, develop common evaluation methods, and help teams learn from early deployments.

An AI centre of excellence is usually cross-functional: it brings together data science, AI engineering, security, risk and compliance, and business leadership, either as permanent members or through assigned participation. Its remit can include enterprise AI strategy, tooling standards, model and application registries, vendor strategy, reusable infrastructure, and workforce training. The intended benefit is coordinated investment and more consistent governance, with business outcomes and return on investment tracked across the portfolio. Establishing a centre does not by itself ensure regulatory compliance or financial returns.

Centralised, Embedded, and Hub-and-Spoke Structures

The centre’s mandate can be delivered through different structures. In a centralised arrangement, one shared team performs most development and provides common services. In an embedded or distributed arrangement, specialists sit within business units and coordinate through shared policies and professional networks. In a hub-and-spoke arrangement, a central hub maintains strategy, standards, platforms, and specialist support while local spokes deliver and operate business applications.

A centre of excellence and federation can therefore coexist. The centre can supply the common capabilities that make local delivery practical, while business units retain responsibility for their use cases. Specify the work each part performs instead of treating the labels as mutually exclusive choices.

For the retail group, a shared identity integration could prevent each department from inventing its own access mechanism. A reusable logging service could help investigators connect application events with supplier calls. Central support can also make specialist review available to departments that could not sustain it independently.

These benefits depend on the centre’s mandate and capacity. A team responsible for delivery, policy, training, support, and every approval may have more responsibilities than it can perform well. Requests then compete without a clear basis for prioritisation.

Define which services the centre provides and how teams obtain them. A documented integration pattern may be available for direct use, while an unfamiliar deployment requires specialist assistance. Publish ownership, support expectations, and the evidence needed for review so departments can prepare useful requests.

The centre also needs continuing contact with operational work. A common evaluation template can help organise evidence, but it will not establish whether a warehouse recommendation is feasible during a shift. Local process owners and users need a role in defining and assessing the outcome.

Federation Depends on Local Capacity and Clear Boundaries

A federated model can place delivery and operational decisions closer to the people affected by them. A customer-service team can adapt an assistant as policies change, while warehouse specialists can investigate planning exceptions without waiting for the same central delivery queue.

That autonomy requires capability. A department needs people who can maintain the application, interpret evaluation results, handle incidents, and obtain specialist help. Giving it ownership without time, funding, or technical support can transfer responsibility without creating the means to fulfil it.

Federation also needs common boundaries. The organisation may require shared identity controls, supplier assessment, data-handling rules, and incident reporting. Departments can then make defined decisions within those conditions, with a route for proposals that fall outside them.

Without regular communication and comparable evidence, local autonomy can become standards drift. Departments may duplicate integrations, apply different audit practices, or adopt unapproved tools to avoid slow shared processes. Maintain a shared inventory, exchange evaluation and incident findings, and review repeated local needs for opportunities to reuse work. Oversight can be lighter for established low-risk patterns while more consequential changes receive specialist review.

For example, a business unit might be authorised to release changes to an internal drafting assistant after completing an established evaluation. Connecting that assistant to a tool that sends customer messages would require further review because it changes the application’s authority. The agent governance article develops controls for that transition.

Delegation should be recorded at the level of actual decisions. A statement that departments own innovation does not tell anyone whether they may sign a supplier agreement, accept a privacy risk, or alter a shared platform. Unclear boundaries tend to surface when a deadline or incident makes them difficult to resolve calmly.

Allocate Decisions According to Their Consequences

Shared consequences are a reason to retain a shared decision. A change to an organisation-wide identity service affects many applications. A supplier agreement may create commitments across departments. A control exception may expose information belonging to another unit.

Local consequences can support local authority where the team has the necessary capability. The wording of a service assistant’s draft templates may sit with customer operations, subject to applicable content controls. The decision to retire an unused warehouse feature may sit with its business and service owners.

The following is an illustrative allocation rather than a universal model:

For each row, identify who makes the final decision and what happens when responsibilities overlap. Consultation is valuable, but a group of consulted teams does not automatically provide an accountable owner.

The voluntary NIST AI Risk Management Framework can inform the risk-management responsibilities, while ISO/IEC 42001 addresses an organisational AI management system. Neither selects the organisation chart for a particular business. Their practices need to be assigned to people who can carry them out.

Funding and Assurance Need to Support the Design

A shared platform can appear inexpensive to departments while its central budget absorbs growing usage and support. Alternatively, charging every experiment the full cost of foundational services may discourage uses that would have benefited from shared infrastructure. Make those funding choices explicit.

Separate experimental costs from the continuing cost of an approved service. A department seeking production ownership should understand what it must fund, what the centre provides, and how shared costs change as usage grows. Otherwise an apparently successful pilot can arrive at handover with no funded operator.

Assurance should also reflect the distribution of authority. Where a central team builds an application, more consequential releases may warrant review by people able to challenge its assumptions independently. In a federated arrangement, departments should provide comparable evidence without forcing every application through an identical assessment.

Exception handling is particularly important. Record the control involved, the reason for the exception, its owner, any compensating measures, and the review date. Repeated exceptions may show that a shared pattern needs improvement or that a department’s proposed use exceeds its current capability.

Metrics should reveal service quality and decision delays. Track how long reviews wait, why submissions return for correction, whether shared components are used successfully, and who resolves production problems. A count of applications delivered will not explain whether the operating model is sustainable.

Change the Allocation as Capability Develops

An organisation can begin with concentrated expertise and delegate more delivery as departments develop the necessary skills. That transition should follow evidence of capability: named owners, reliable deployment practices, useful evaluations, incident handling, and an understanding of the process being automated.

Move responsibilities in bounded steps. A department could first maintain prompts and evaluation cases within an existing service. It might later take responsibility for releases under defined controls, then operate its own application using shared infrastructure. The appropriate sequence depends on its people and the consequences of the use case.

Keep specialist support available after delegation. Local teams will still encounter unfamiliar suppliers, new legal questions, or changes that affect other departments. Federation works better when escalation is a supported part of the design rather than evidence that the department has failed to be independent.

The allocation can also move in the other direction. If multiple units repeatedly build similar connectors, a shared service may be justified. If a department cannot support a consequential application, some responsibilities may need to return to a group with the required capability while the gap is addressed.

Review the model through actual work. Follow a proposal from funding to deployment and an incident from detection to resolution. Identify where authority was clear, where evidence was missing, and where a handoff added avoidable delay. That produces a better basis for adjustment than a general preference for centralisation or autonomy.

The AI adoption strategy establishes which outcomes warrant investment. The operating model determines who can deliver and sustain them. A useful arrangement lets teams make the decisions they understand, provides shared capabilities where those reduce repeated work, and leaves no ambiguity about who owns the resulting service.