Strategic AI Architecture

The Model is Not the System

Late on a Tuesday evening, a bank customer disputes a $35 overdraft fee. The bank’s AI service agent reviews her account history, retrieves the fee-waiver policy, concludes that she qualifies, reverses the charge, and documents the exchange. Two minutes later, the customer closes the chat, satisfied. By any pilot-program measure, the system worked.

Yet much of what made it work, or could have made it fail, was outside the AI model. Did the agent retrieve the current waiver policy, or one revised away last month? Under whose authority did it reverse the charge, and what kept that authority from extending to a $3,500 transaction? If reversals suddenly doubled, could the bank suspend the agent’s refund authority while letting it keep answering questions? Upgrading to a smarter, more accurate model would not resolve any of these questions. To govern those actions, the model must be embedded in a larger production system that supplies trusted data, permissions, and runtime policy enforcement. This highlights the shift in system architecture: AI systems can interpret changing context and dynamically invoke tools that move money or trigger workflows. When individual teams build these systems on their own, the enterprise fractures into silos that are harder to secure, change, and run.

None of this requires replacing the enterprise technology stack. As argued in Point of View - Digital Transformation, transformation is a continuous, business-led process: set priorities, make deliberate changes, measure results, learn, and adapt. Accordingly, the Board’s question should reach beyond “What is our AI strategy?” It should ask whether the enterprise has the architecture and operating discipline to govern what AI can access and what it can do, and to show that it creates measurable value.

We return to the fee-reversal agent example throughout this piece, beginning with one of the system’s fastest-changing components: the model and its provider.

1. Design for Choice, Not Assumed Portability

Large, multi-year technology commitments limit an organization’s ability to adapt as AI capabilities and economics evolve. Models are the fastest-moving layer of the stack; the provider behind the fee-reversal agent today may not be the best choice, or available on the same terms, a year from now. Concentrating critical workloads with a single provider creates a dependency that is often difficult to unwind. A common, but flawed, IT response to this risk is simply centralizing model access. While a control layer or middleware can manage routing, credentials, and metering, centralizing network traffic alone does not make an application portable. If the underlying code still relies on a provider’s specific prompts, proprietary tool interfaces, or unique data formats, the enterprise remains locked in. A central routing hub cannot automatically enforce permissions across disparate data sources and business connectors.

Rather than assuming universal portability across all systems, the ability to switch models should be treated as a targeted financial investment, applied only where the flexibility outweighs the integration cost. The most resilient architectures isolate core business logic and data from the AI models themselves, using stable interfaces to prevent hardcoded dependencies. This allows organizations to evaluate candidate models against real-world tasks (quality, speed, and cost) and to validate fallback paths before deployment. For routine workloads, standardizing on a single provider is often the most efficient choice; for highly consequential systems, integrating a secondary provider may provide worthwhile resilience.

Multi-model routing is not an end in itself. The objective is to understand the true cost of vendor dependency and preserve a credible path to change providers without rebuilding entire workflows. By favoring modular design, the enterprise retains the leverage to adjust or halt projects that fail to deliver business value. The harder dependency is often underneath the model: the information it consumes and the decision history needed for audit, compliance, and learning.

2. Treat Data and Context as Durable Infrastructure

Data and context are the foundation of any enterprise AI system, yet they are rarely managed with the same rigor as traditional IT infrastructure. While relational databases remain essential for transactional records, AI dramatically expands the scope of what must be governed. To operate reliably, organizations now need to treat unstructured content (documents, messages, logs, and policies) as durable infrastructure, securing and managing it with the same discipline applied to structured systems. For example, if the bank agent retrieves a fee-waiver policy superseded last month, the failure lies in the information pipeline (versioning, publication, or retrieval), not in the model itself.

Managing this expanded scope requires more than a simple search index. Organizations have to establish the provenance, meaning, and currency of their information, as well as the business rules governing its use. Access permissions must persist through the retrieval process and extend into subsequent model and tool executions. If an AI system produces a flawed or harmful output, lineage and quality signals help teams trace it to its source data, retrieval context, or shifting business definitions.

For highly consequential systems, operators must be able to reconstruct the exact conditions that generated an output, including the source-data versions, retrieval context, model configuration, and any action taken. As established in Data Governance for AI Must Be Executable, policies become effective only when translated into enforceable controls that generate auditable evidence.

A resilient architecture treats data and context as managed assets capable of serving multiple applications under unified security controls. While applications still encode business rules and user experiences, the goal is to make information reusable without sacrificing its meaning, origin, or access restrictions, preventing the duplication of data into unmanaged silos for every new AI experiment.

3. Put Identity and Policy at the Point of Access and Action

When an AI agent connects to enterprise tools, the primary risk shifts from what the model might say to what the system is authorized to do: which actions it can take, on whose authority, and under what conditions.

To manage this risk, organizations require a policy enforcement layer that inspects data requests and tool calls prior to execution. Rather than relying solely on network perimeters, this layer applies identity and permissions directly at the point of action, enforcing approved scopes, validating requests, and recording decisions. For high-impact operations, the architecture needs a way to pause the automated workflow to require explicit human approval before an action proceeds. Even when the bank agent reverses a routine $35 charge, the policy enforcement layer checks whose authority it is acting under and whether it has permission to execute that specific financial transaction.

While standardized integration protocols provide necessary authorization mechanisms, merely adopting them does not automatically grant visibility or control over every data path. Governance also requires an inventory of approved workflows, least-privilege access, and permissions enforced both at the connection point and within the underlying application service.

The same discipline applies to AI features embedded in vendor platforms, such as Microsoft 365 Copilot, Salesforce, or an electronic health record system. The enterprise may not control the underlying model or service, but it can govern the feature’s configuration and integration. Before enabling it, scope user and service identities, data access, and available actions; verify which administrative controls the vendor supports; and confirm that execution logs are sufficiently detailed and can be retained and reviewed. If the available controls or evidence do not meet the needs of a use case, limit the feature’s permissions or exclude it from that workflow.

Agentic workloads demand more than basic model routing. A resilient production foundation orchestrates multi-step tasks, maintains a curated catalog of approved APIs, and provides observability into every automated decision. The strategic approach is not to build every capability immediately, but to deploy the minimum set of reusable controls necessary to secure the first governed workflows.

Every deployed agent requires a clear charter detailing its purpose, allowed tools, and level of autonomy. As established in Point of View: Agentic AI — Enterprise Adoption, organizations should begin with advisory roles or low-risk actions, expanding autonomy only when production evidence justifies the trust. For highly regulated or irreversible decisions, the system may analyze and recommend while final execution authority remains with a human.

Architectural controls should scale with the consequence of the action. An internal summarization tool does not require the same level of oversight as an agent capable of altering customer records or initiating payments. Securing data and enforcing policy at the point of action lays the groundwork for safe automation. The next test comes after deployment: running the system as a reliable business service.

4. Operate AI as a Production Service

A model that passes a pre-launch test can still fail when underlying data changes, user behavior shifts, or a provider updates an integration. Production readiness requires continuous evaluation and strict operational ownership, not just initial model selection.

For every consequential AI system, the enterprise needs a business owner accountable for real-world outcomes and a technical owner responsible for service health. Define success, identify unacceptable behaviors, and set triggers for reassessment. Monitoring should focus on actionable signals: data freshness, output quality, failure rates, and human overrides. Every critical metric requires a defined threshold, an escalation path, and a designated owner; a metric without an owner is merely a dashboard item.

Measure both leading indicators and business outcomes. Early indicators, such as exception rates and time-to-resolution, allow teams to adjust systems proactively. Outcome measures must reflect the core business purpose: customer experience, cycle time, operational resilience, productivity, or cost. The mandate is to scale what works, halt what does not, and stop treating deployment activity as evidence of value.

Finally, plan for degraded operation. If a model becomes unreliable, a designated team must be ready to take over the function through a tested manual path or a simpler fallback, and operators need the ability to revoke the agent’s authority to issue refunds while preserving read-only assistance. In the bank example, monitoring should flag reversal rates above expected thresholds and let the team disable refund actions without ending customer-support interactions. As argued in AI Risks Don’t Wait for Committees, governance sets policy; monitoring, clear escalation paths, and rehearsed recovery make it operational.

5. Build the Skills and Decision Rights to Run It

Six months after launch, the fee-reversal agent is performing well. Complaints are down, override rates are stable, and the retail banking team wants to raise the agent’s refund cap from $50 to $500. Who makes this decision?

Production AI creates recurring decisions that someone must own: who may expand an agent’s autonomy, accept risk within delegated limits, pause its actions, or approve scaling. As we explored in Who Should Own Enterprise AI Governance? and AI Governance Is Broken, those rights need clear owners: the business owner is accountable for outcomes, the technical owner monitors service health, and exceptions escalate to a cross-functional oversight body. If the answer is “whoever happens to be in the meeting,” the operating model has a critical gap.

For the fee-reversal agent, the answers should be settled before such questions are asked. If $500 falls within the business owner’s delegated limit, the business owner approves it, engineering implements the new cap, and the technical owner watches the first weeks of data. If it does not, the request goes to the oversight body as a routine matter. Either way, the path is known in advance. Wherever the system can enforce these boundaries, the decision rights behind them should appear as permissions, limits, approval gates, and escalation rules. As organizations are learning the hard way, a RACI chart can say who approves a higher refund cap, but only the system can keep the agent from exceeding it.

Clear decision rights solve only part of the problem. The people exercising these decision rights must also know how to run AI as a production system. That requires capabilities many IT organizations are still building, including product ownership for AI services, stronger data integration skills, and teams that can manage agent workflows, evaluate shifting behavior, and respond to incidents. Some of this comes from hiring and training; most comes from operating live systems. Business priorities, rather than technology adoption, should set the agenda, and teams aligned around customer and operational outcomes need room to adjust course within explicit boundaries.

Finally, these roles belong inside processes the enterprise already runs: security reviews, privacy assessments, procurement, risk management, and incident response. Simply implementing a parallel AI governance apparatus adds overhead without closing gaps. Where AI introduces new risks, add specific checks; where existing controls suffice, extend them. The bottom line is that a working operating model is built around live systems, not around another layer of documentation.

Read the full piece on Substack, found here.