Once a company commits to an agent program at scale, the next question is who decides what gets built. The two organizational models are the center-of-excellence (COE) and the federated model. The COE concentrates agent-building inside a single team that owns every production agent; the federated model distributes agent-building across the business units that own the underlying work, with a thin platform team providing shared rails. Both have respectable lineages — the data-team-vs-decentralized-analytics debate of the 2010s is the closest analog — and both have produced wins and disasters. In 2026, the operating consensus has shifted decisively toward federated-with-shared-rails. The COE has not disappeared, but its role has changed: it is the platform team, not the build team. The COE-as-bottleneck failure mode killed enough programs in 2024-2025 to put the model on the defensive everywhere. This lesson lays out the two models in detail, names the failure modes of each, walks through the 2026 winner pattern, and gives the strategist the diagnostic for picking which model fits the specific company they are operating inside.

The Two Models, Defined

The terms get used loosely. Pinning them down before the comparison matters.

The center-of-excellence model

In a COE model, a single central team owns the building of agents. The team typically reports to a senior executive (CIO, CTO, chief AI officer, or chief data officer in some structures) and has its own engineers, product managers, designers, and operations staff. Business units (sales, support, finance, marketing, operations) bring requests to the COE; the COE evaluates, prioritizes, builds, deploys, and operates the agent. The COE owns the agent for its entire lifecycle.

The COE model has a long lineage. Most enterprise IT functions have organized centrally for decades because central IT can enforce standards, capture economies of scale, and act as a single point of accountability for cross-cutting concerns (security, compliance, identity, data governance). The COE-for-AI is the same instinct applied to agents.

The federated model

In a federated model, business units build their own agents using shared rails provided by a central platform team. The sales team builds the sales agent. The support team builds the support agent. The finance team builds the finance agent. Each team has its own engineers, citizen developers, or vendor partners who do the actual building. The platform team provides eval, observability, guardrails, identity, and the MCP allowlist (the layers from the previous lesson), plus policy and standards. Agent teams pick their own orchestrator within the platform's bounds, write their own prompts, own their own roadmap, and run their own on-call.

The federated model has its own long lineage. Most modern engineering organizations have decentralized service ownership for two decades — each team owns its services from cradle to grave, supported by a central platform team that provides the substrate. The federated-for-AI model is the same instinct applied to agents.

The hybrid that fools people

There is a third configuration that gets called "hybrid" or "hub-and-spoke" and that often fools the strategist into believing they have solved the COE-vs-federated tension when they have not. In this configuration, a central team builds most agents but business units can build "small" ones. The risk is that the central team gradually absorbs everything (because building agents is what they exist to do) and the business units' agents wither (because their teams are not staffed to maintain them). The hub-and-spoke either drifts back to a pure COE or drifts toward federated; the in-between is not a stable equilibrium.

This lesson treats hub-and-spoke as a transitional state, not a destination. The strategist who picks hybrid often does so to avoid the political work of picking one of the two real models. The political work has to happen anyway.

When The Center-Of-Excellence Model Works

COE is not dead. There are organizational shapes where COE is the correct answer and federated would be wrong. The strategist needs to name them.

Small companies (under 500 employees)

A company with two hundred employees does not have business units that can absorb agent-building capacity. There is one team building everything, regardless of what you call it. Federated is a fiction at this size. The COE — which here is often just "the AI team" of three to eight people — builds, runs, and supports every agent. The COE label is fine; the substance is correct.

Highly regulated industries with mandatory single-point accountability

Some industries (large-bank consumer lending, parts of healthcare, parts of insurance) require a single accountable function for AI decisions. The regulator wants one phone number. The audit trail must converge on one team. The model risk management function must approve every model in production. In this regulatory context, the COE is the model; trying to federate creates a compliance nightmare that the regulator will eventually unwind for you.

Even here, the COE often federates the building work to embedded engineers in business units while retaining centralized approval and oversight — a structure that looks more like federated-with-strong-rails than classic COE. But the central team owns the regulatory relationship and the official accountability.

Companies where AI capability is highly differentiated

If the company's agents require deep specialist skill (reinforcement learning for trading, multi-agent simulation for logistics, specialized fine-tuning for medical imaging), the rare specialists must be concentrated to be productive. Spreading three world-class ML engineers across five business units leaves no team with the critical mass to ship anything ambitious. The COE concentrates the talent; the business units consume the output.

Programs in their first year

Almost every agent program starts as a COE whether you call it that or not. The first three to five agents are built by the same small team because the company is figuring out what an agent program even is. This is correct. The decision about whether to stay COE or transition to federated arrives later — typically at the 10-agent threshold from the previous lesson, when the central team's capacity is becoming a bottleneck.

When The COE Model Fails

The COE failure modes are mature, well-documented, and have produced enough wreckage to be named.

The bottleneck

The most common COE failure. The central team's intake queue grows faster than its delivery capacity. Business units wait nine months for an agent that takes the COE three weeks to build because they are in the queue behind nineteen other agents. The waiting business units lose interest, find workarounds (often unsanctioned shadow agents built in n8n or Zapier outside the COE's view), and stop bringing the COE the next round of requests. The program looks productive (the COE is shipping) but is actually starving (the demand is going elsewhere).

The bottleneck failure is structurally identical to the central-IT-as-bottleneck failure that drove the federated-engineering revolution in the 2010s. The same dynamics, the same outcomes, the same political collapse when business units route around the central team.

The distance-from-the-work problem

The COE's engineers are technically excellent but do not understand the work the agent is supposed to do. The sales agent is built by an engineer who has never sold; the support agent is built by an engineer who has never handled a ticket; the finance agent is built by an engineer who has never closed a quarter. The agents work in demo and fail in production because they encode a model of the work that the actual practitioners would never recognize.

The fix in a COE is embedded subject matter experts. The fix in a federated model is the business unit owning the build. The federated structure is closer to the work by design.

The single-vendor lock-in

The COE makes one decision about orchestration framework, model provider, and platform that applies to every agent. The decision is reasonable when made; it ages poorly across three years. The voice agent the company wants to build does not fit the framework. The new model from Anthropic costs half as much but the COE has standardized on OpenAI. The MCP server the partner needs is incompatible with the COE's identity model. The COE's standardization, which was a strength at scale, becomes a constraint that produces second-best agents across the board.

The political collapse when something fails

The COE is the named accountable team for every agent. When one agent fails publicly (a hallucination in front of a customer, a security incident, a privacy violation) the political damage lands on the COE. The other business units, which were customers of the COE rather than owners of the agent, are insulated from the blast. The COE absorbs the blame and the COE's executive sponsor loses cover. Repeat this twice and the COE is in survival mode, every decision is risk-averse, and the program stops shipping anything ambitious.

The reorg cycle

Because COEs are politically expensive when they fail, they get reorged. The COE under the CTO becomes the COE under the CDO, which becomes the COE under a new chief AI officer, which gets absorbed into the data office, which gets restructured into the digital office. Each reorg costs six months. The program never accumulates the institutional memory it needs because the team chart keeps changing.

The bottleneck, the distance from the work, the single-vendor lock-in, the political fragility, and the reorg cycle. Each one has killed agent programs in 2024 and 2025. A strategist who recommends COE in 2026 must explain how the recommendation avoids all five — and the answer is rarely satisfying.

When The Federated Model Works

The federated model has been the default for general software engineering for fifteen years. The argument for federated agents is largely the same argument adapted to a new substrate.

The business units are closer to the work

The sales team understands sales. The support team understands support. The finance team understands finance. When the agent-builders are the people closest to the work, the agents encode the work correctly. The eval cases are real cases, not synthetic ones. The edge cases are recognized as edge cases. The success metric is what the business unit cares about, not what an engineer thought they should care about.

The decisions are local

The sales team can decide its sales agent runs in Relevance AI because they are already familiar with it. The support team can decide its support agent runs in a custom LangGraph build because they need the orchestration flexibility. The finance team can decide its reconciliation agent is built in n8n because that is what their citizen developers already use. The local decisions are made by the people who will live with them, not by a central team optimizing for averages.

The on-call is owned

When the sales agent breaks at 2am the page goes to the sales team's on-call, not to a central team that does not understand sales workflows. The team that owns the agent owns the incident. This is the same property that made decentralized service ownership the dominant engineering model — the team that builds and runs the service has the right context to debug it.

The platform team is bounded

The platform team in a federated model has a clear, bounded role: provide the shared rails. They do not build agents. They do not own agents. They do not become the bottleneck. Their success metric is the velocity of the agent teams using the platform, not the volume of agents the platform team has shipped.

When The Federated Model Fails

Federated has its own failure modes. The strategist who picks federated must name these too.

The capability gap

Some business units do not have the engineering capability to build agents. Marketing might have one technical person; finance might have none. Telling these units "build your own agent" produces either no agent at all (the work does not happen) or an agent built badly (no eval, no observability, no guardrails). The federated model assumes a baseline capability that some units do not have.

The federated answer is one of three patterns: (a) embed platform engineers into capability-poor units for the first agent or two, (b) allow capability-poor units to consume agents built by neighboring units that have capability, or (c) use no-code citizen-developer platforms (Lindy, Relevance AI, n8n) that lower the building bar enough for the unit to participate.

The inconsistency problem

Twelve teams build twelve agents in twelve different shapes. The user experience varies. The eval rigor varies. The cost-per-run varies. The security posture varies. The CIO sits down with the twelve dashboards and cannot answer the simple question: "are our agents working?"

The federated answer is the shared rails — the platform's eval, observability, and guardrail substrate makes inconsistency at the agent level surface as consistency at the metric level. A federated program with strong rails has twelve different-looking agents and one consistent dashboard.

The duplication problem

Three teams build similar agents because they did not know the other teams were building them. The marketing team's content QA agent and the support team's content QA agent are eighty percent the same code. The platform team's job includes a tool catalog and a "what agents exist" registry; duplication that survives that registry is intentional rather than accidental.

The cross-cutting work nobody owns

Some agent work is genuinely cross-cutting and does not naturally fit inside a single business unit. The "answer questions about company policy" agent is needed by every team. The "summarize the last week's customer feedback" agent crosses sales, support, and product. In a pure federated model these agents do not get built because no single unit owns them.

The federated answer is a small central team — sometimes called the "shared agents" team or the "horizontal agents" team — that owns the cross-cutting work. This team is part of the platform organization but is not the platform team; their charter is explicit and small.

Federated-With-Shared-Rails: The 2026 Winner Pattern

The pattern that the most successful agent programs in 2026 are converging on is a specific shape of federated. The shape is opinionated enough to be reproducible.

What the platform owns

The five layers from the previous lesson: evaluation, observability, guardrails, identity, and MCP allowlist. The platform also owns the tool catalog, the agent registry (what agents exist, who owns each, what they do), the standards documentation, and the agent intake form and risk tiering process. The platform's deliverable is the substrate that every agent runs on.

What the business units own

The agent itself: its prompts, its orchestration choice (within platform-supported options), its tool selection (from the platform's allowlist), its eval set (within the platform's harness), its user experience, its rollout plan, its on-call. The business unit owns the agent for its full lifecycle, including its sunset.

The horizontal agents team

The shared-agents team owns the cross-cutting agents that do not fit inside a single business unit. This team is small (two to five engineers in a medium-sized company), reports into the platform organization, and has a deliberately narrow charter (we build agents that serve more than one business unit and have no obvious natural owner).

The standards and policy function

A small central function — sometimes one person, often part of the platform team — owns the standards and policy. What is the agent risk tiering system? What are the eval threshold requirements for production? What is the incident response playbook? What is the kill plan template? These are policy questions whose answers belong centrally; the agent teams consume the answers, not write them.

The governance forum

A standing meeting (typically monthly or quarterly depending on program scale) where agent owners from across the business units, the platform team, the security team, the data office, and the legal/compliance function review the agent portfolio. New agents are presented for awareness, not approval. Existing agents are reviewed against the standards. Incidents are debriefed. Kills are announced. The forum is the connective tissue across federation; without it, the units drift into incompatible practices.

A concrete sketch at scale

A 5,000-person company with twenty production agents in 2026 looks something like this. Twelve business units have built agents (some have built more than one). The platform team is eight engineers. The shared-agents team is three engineers building four cross-cutting agents. The standards function is one half-time strategist. The governance forum meets monthly. Total people fully dedicated to the agent program: thirty-five to fifty across the org, of whom twelve are in central functions and the rest are embedded in business units. The COE equivalent for the same workload would be a forty-to-sixty-person central team, with all of the failure modes above, plus six months of organizational politics to get the headcount approved.

The Diagnostic: Picking The Right Model For Your Company

The strategist needs a way to pick between COE, federated, and the transitional hybrid. The following diagnostic gets the answer right in most cases.

Company size

Under 500 employees: COE (which is just "the AI team"). Federated does not have business units to federate to.

500 to 2,500 employees: hybrid is acceptable as a transitional state, with federated as the target.

Over 2,500 employees: federated with shared rails; the COE failure modes are nearly inevitable at scale.

Regulatory environment

Heavily regulated industries with mandatory single-point accountability: keep formal accountability central but federate the building work. The structure looks like COE on the organization chart and federated in the building model.

Lightly regulated industries: federated outright.

Capability distribution

If the business units have engineering or citizen-developer capability: federated works well.

If the business units have no technical capability: federated still works, but the central team has to embed engineers into capability-poor units for the first agents. The capability gap is solved by capability-building, not by reverting to COE.

Existing organizational pattern

Companies that decentralized their software engineering (each team owns its services, supported by a platform team): federated agents will feel native.

Companies whose engineering is still centralized (one big central engineering team builds everything): COE for agents will feel native at first, then hit the same scaling walls the central engineering team is already hitting. The strategist should consider whether the agent program is the wedge that finally federates the engineering organization, or whether the agent program inherits the constraints of central engineering.

The political reality

The strategist's recommendation has to land in a specific political environment. If the CIO has just announced a new AI center of excellence, recommending federated requires careful positioning. If the CIO has just been burned by a centralization failure, federated is the easy sell. The strategist reads the room, makes the recommendation that fits the room, and uses the failure modes from this lesson as the language to defend the recommendation against the inevitable challenges.

The Transition From COE To Federated

Many programs start as COE — correctly, in their first year — and need to transition to federated as they scale. The transition is the most politically delicate move in the program's life. It needs to be done deliberately.

The trigger

The transition trigger is the same set of signals from the previous lesson: the central team is hitting capacity, the queue is growing, business units are starting to build shadow agents, the failure modes are appearing. When three of the COE failure modes are visible (bottleneck, distance-from-the-work, vendor lock-in, political fragility, reorg cycle), the transition is overdue.

The first move: turn the COE into the platform team

The COE does not disappear. It transitions. The COE becomes the platform team that provides the substrate for the federated agents. The engineers who were building agents now build the eval substrate, the observability layer, the guardrail libraries, the identity registry, and the MCP allowlist. The deep technical skill the COE accumulated is exactly what the platform team needs. The reframe is from "team that builds agents" to "team that makes agent-building possible."

The second move: identify the first federated builders

One or two business units are selected to be the first federated agent builders. These are units with both motivation (they have a high-value use case they want owned locally) and capability (they have engineers or citizen developers who can build). The platform team embeds support staff with the first federated builders for the first agent. The first federated agent is the proof point that the model works.

The third move: codify the standards

The standards and policy that the COE was applying tacitly now get written down. The agent risk tiering system, the eval thresholds, the incident response playbook, the kill plan template. The business units that are now building need to know the rules; rules that existed only in the COE's heads do not transfer.

The fourth move: rebalance the headcount

Over four to eight quarters, the COE's headcount shifts. Engineers transition from agent-building to platform-building or to embedded roles in business units. New agent-builder headcount goes to the business units. The total headcount may grow modestly but the central-to-federated ratio shifts dramatically. By the end of the transition, the central team is half or less of what the COE was, and the building capacity has grown three to five times because each business unit is now contributing.

The political timing

The transition cannot happen during a crisis. Trying to reorg the program in the middle of an incident response is a disaster. The transition needs a stable quarter, a healthy program, and an executive sponsor who understands why the change is happening. The strategist times the transition to a moment of program health, not a moment of program stress.

The Anti-Patterns Of Federation

Even teams that pick federated correctly can execute it badly. The anti-patterns are worth naming.

Federation as abdication

The platform team announces federation and then disappears. There are no shared rails. There is no governance forum. There is no standards documentation. The business units are left to figure it out themselves. The result is chaos — twelve agents in twelve incompatible shapes, no consolidated reporting, no consistent security posture. This is not federation; this is abandonment.

Real federation requires the platform team to be visibly present and visibly useful. The shared rails have to actually exist. The governance forum has to actually meet. The standards have to actually be enforced.

Standards-as-suggestions

The platform writes standards but does not require them. The standards become documents nobody reads. The federated agents drift into whatever shape each team felt like building. The platform's eval substrate is bypassed because following it is optional. The platform's MCP allowlist is bypassed because someone added their own MCP server outside the registry. Standards that are not enforced are not standards.

The fix is to make the platform's services the path of least resistance. The platform's eval substrate is opt-in but the alternative is building your own from scratch. The platform's MCP allowlist is the only set of MCP servers with credentials managed automatically. The platform's identity registry is the only identity that works in production. The carrots are real; the alternatives are real work.

The platform that is everywhere and nowhere

The platform team is responsible for so many things that it cannot meaningfully focus on any of them. Eval, observability, guardrails, identity, MCP, tool catalog, governance forum, training, evangelism, incident review, vendor management, FinOps. The platform team is over-extended; nothing gets done well.

The fix is to scope the platform's deliverables explicitly. The five layers plus the tool catalog and the standards. Other things (training, evangelism, FinOps) are owned by other teams that the platform supports. The strategist who writes the platform team's charter precisely is the strategist who prevents this anti-pattern.

The federation that drifts to fiefdoms

Each business unit builds in isolation. The sales team's agents only talk to other sales agents. The support team's agents only talk to other support agents. Cross-cutting work that needs multiple teams' agents to cooperate cannot get done because no team has authority across the boundary. The federation has become a set of silos.

The fix is the governance forum and the shared-agents team. The forum keeps the units talking. The shared-agents team builds the cross-cutting agents. Together they create the connective tissue that prevents the fiefdom drift.

Key Takeaways

• The two organizational models are the center-of-excellence (COE) and the federated model. The hybrid that gets called "hub-and-spoke" is a transitional state, not a destination.
• The COE works in small companies, heavily-regulated industries with single-point accountability, companies needing concentrated specialist skill, and programs in their first year.
• The COE failure modes are the bottleneck, the distance from the work, the single-vendor lock-in, the political fragility when something fails, and the reorg cycle.
• The federated model works when business units are closer to the work, decisions can be local, on-call is owned, and the platform team is bounded.
• The federated failure modes are the capability gap in some units, the inconsistency problem, the duplication problem, and the cross-cutting work that nobody naturally owns.
• Federated-with-shared-rails is the 2026 winner pattern. The platform owns the five layers plus tool catalog and standards. Business units own their agents. A shared-agents team handles cross-cutting work. A governance forum keeps the federation coherent.
• The diagnostic: under 500 employees, COE; over 2,500 employees, federated; heavy regulation, formal accountability central but building federated; lightly regulated, federated outright.
• The transition from COE to federated is the most politically delicate move in a program's life. The COE becomes the platform team; first federated builders are selected from motivated, capable units; standards are codified; headcount rebalances over four to eight quarters.
• Federation anti-patterns: federation as abdication (no shared rails), standards-as-suggestions (not enforced), platform that is everywhere and nowhere (over-scoped), federation that drifts to fiefdoms (no connective tissue). Each has its own fix.