That is why AI Agent governance scoping is crucial. If an autonomous AI Agent goes off track, the central question is which controls failed and who owns the outcome. In this article, we will first identify the stakeholders responsible when an AI agent fails, then outline practical controls that AI product managers and other stakeholders can use to bridge this accountability gap and reduce agentic AI risk.

Who is Accountable When an AI Agent Fails?

When an AI Agent makes a wrong decision, accountability cannot be assigned to one party by default. The failure must be traced across the full operating chain. As you explore the section that follows, you will see how accountability is distributed and why understanding each one is key to managing risk effectively.

AI Agent Developer or Model Provider

Model providers such as OpenAI, Anthropic, Google, and others build the underlying models and platforms that power AI Agents. Their exposure usually depends on whether the model was unsafe, defective, poorly documented, or marketed with unrealistic claims about reliability or autonomy. But AI Agent developers or model providers typically do not control how a business configures an agent for deployment. They also do not define the company’s approval rules, escalation paths, or spend limits.

That is why AI Agent liability cannot be judged only at the model layer. The provider-side safeguards, such as model cards, usage policies, safety testing, API controls, and terms of service, may show due diligence. But they do not remove the need for deployment-layer AI Agent governance. If a company connects an agent to sensitive workflows, gives it broad permissions, and fails to monitor its actions, responsibility will likely move closer to the deploying business.

Organization Deploying the AI Agent

The deploying company is the closest to the harm because it decides agent’s workflow, data access permissions, what actions it can execute, and which decisions require human approval or escalation. Let’s understand this with the Air Canada chatbot case. In Moffatt v. Air Canada, a customer relied on incorrect bereavement-fare information provided by Air Canada’s website chatbot. The tribunal rejected airline’s argument that the chatbot was a separate entity responsible for its own actions and held that the airline was responsible for information presented on its website.

The same logic applies when the failure is operational. In another incident, an AI coding agent deleted an entire company database in seconds, with backups also wiped after a Cursor tool powered by Anthropic’s Claude went rogue.

Both incidents show that AI Agent risk management must extend beyond the model layer and focus more deeply on the deployment layer. This is where the agent’s decision-making autonomy, workflow access, approval gates, monitoring, and rollback controls determine how much business risk it can actually create.

End User Who Set the Task or Permissions

An end user can contribute to AI Agent failure by giving vague instructions, uploading sensitive data, bypassing approved workflows, granting broad permissions, or using unapproved tools. These actions carry greater risk because a single prompt can trigger activity across multiple business systems, from data retrieval and API calls to workflow execution and external communication. The risk becomes harder to control in shadow AI environments, where employees may begin using AI Agents before the organization has defined access rules, data-handling standards, approval controls, or escalation paths. In such cases, the issue is not limited to individual misuse but also reflects a broader AI Agent governance scoping gap.

That is why AI Agent risk management needs an operating model built around controlled autonomy. Every agent should have defined access limits, approved actions, escalation triggers, and monitoring requirements. This need is already visible at the leadership level, with 78% of C-suite executives agreeing that gaining maximum benefit from Agentic AI requires a new operating model. Internally, an employee’s conduct may still inform root-cause analysis, access reviews, training, or disciplinary action. A mature AI Agent governance model should therefore enforce boundaries through role-based access control, approved tool lists, DLP policies, prompt monitoring, workflow approvals, spend limits, and escalation rules.

AI Agent Itself

It may be natural to say that the AI Agent made the mistake. But that only describes the mechanism of failure, not why it happened and who exactly is responsible for this failure. An AI agent has no legal personhood, no assets, no duty of care, and no ability to be sued, fined, insured, or disciplined. Therefore, its responsibility must trace back to the human and organizational decisions, including who selected the model, configured it, connected it to existing systems, approved its permissions, and monitored the actions.

The Air Canada case reinforces this point, where the chatbot was treated as part of the company’s customer-facing system, not independent. That same principle applies to enterprise AI Agents, where they may execute the wrong action, but accountability sits with the parties that designed, deployed, governed, or failed to control them. So, the real question is whether the organization can prove that the agent’s authority was properly scoped, its access was controlled, and its actions were logged.

AI Agent Accountability Framework for CXOs if it Fails

This is a practical decision matrix for AI Agent risk management based on the stakeholder analysis above. The decision makers can save this table to know where exactly the exposure sits before any incident.

How Can Enterprises Close the AI Agent Accountability Gap?

Organizations do not need to wait for regulation before tightening AI Agent accountability. The immediate priority is to reduce the probability of agentic failure and limit exposure when an error occurs. The following controls create a practical governance baseline for responsible AI Agent deployment.

Source

Add Human Checkpoints for High-Stakes Actions

Not every agent action needs human approval, but high-impact decisions should not execute autonomously. Any action involving financial thresholds, customer-facing communication, sensitive data, or irreversible production changes should require explicit human sign-off. This adds targeted control without slowing low-risk automation.

Maintain Comprehensive Decision Logs

If an organization cannot reconstruct why an AI Agent acted, it cannot assign accountability with confidence. Therefore, every consequential action should log the input, retrieved context, tools used, action parameters, decision logic, approver, and final outcome. These records support internal review, regulatory defense, and continuous governance improvement.

Scope Permissions and Action Limits

AI Agents should operate within defined access, spend, and execution limits. They should not access systems, approve transactions, or trigger actions beyond their assigned workflow. Hence, enforcing hard limits lets organizations reduce the risk of unauthorized actions, sensitive data exposure, and uncontrolled downstream impact.

Define Vendor Liability Terms

Third-party AI tools and agent frameworks should be onboarded with clear contractual accountability. Before deployment, organizations should define liability terms, indemnity coverage, service-level commitments, security obligations, and incident-response responsibilities. If the contract is silent, the deploying organization will often absorb most of the risk.

Review Insurance Coverage for Agentic AI Risk

Technology E&O and cyber insurance products are beginning to address Agentic AI exposure. Enterprises should, therefore, review whether existing policies cover autonomous decision errors, data leakage, financial loss, third-party tool failure, or regulatory claims. This conversation should happen before deployment, not after an incident.

Closing the Gap Starts with AI Agent Governance Scoping

Scoping an AI Agent’s decision-making authority is not a checkbox that has to be checked once during deployment. It is, however, more of a continuous discipline that has to keep pace with how much autonomy you hand the agent over time. Most organizations scope an agent's permissions carefully at launch, then quietly expand its access over the following months as it proves useful. That gradual creep is where the accountability gap usually reopens, because the original scoping decision was never revisited against the agent's new capabilities. 

Ultimately, closing this agent accountability gap is what separates scalable Agentic AI adoption from an ungoverned and risky adoption. By treating AI governance as an evolving discipline, you safeguard your data and protect your bottom line. This ensures AI Agents remain trusted operational assets, not unmanaged liabilities.

After this deadline, applications built on Assistants, Threads, and Runs endpoints will stop working, making Azure OpenAI Assistants API migration a business continuity priority. Adding to this migration pressure is a genuine wave of confusion in developer communities about what's actually happening. Early Microsoft guidance told Azure OpenAI users they were unaffected by OpenAI's own Assistants API shutdown. That guidance has since changed, creating real uncertainty about which retirement date applies to which product.

In this article, we will explore what's retiring, what isn't, a step-by-step Azure OpenAI Assistants API migration plan, and the gotchas that catch teams off guard.

What's Actually Retiring and What Isn't?

This is the most-searched point of confusion around this retirement, and it's worth resolving clearly before anything else: three separate things have three different fates, and people constantly conflate them.

1. OpenAI's own Assistants API (platform.openai.com)

This is the original Assistants API that developers use directly through OpenAI's platform, independent of Azure. OpenAI announced its deprecation on August 26, 2025, giving developers exactly one year's notice, with a shutdown date of August 26, 2026. As an alternative, OpenAI recommends the Responses API paired with the new Conversations API. [Source]

2. Azure OpenAI's Assistants API

This is Microsoft's own implementation of the Assistants pattern inside Azure OpenAI in Microsoft Foundry. For a long stretch, Microsoft's public guidance said Azure OpenAI was unaffected by OpenAI's deprecation, since Azure's Assistants API doesn't technically depend on OpenAI's platform-hosted endpoints. That guidance has changed now. The Assistants API is now deprecated and will be retired on August 26, 2026, with Microsoft directing developers to the generally available Microsoft Foundry Agents service as the replacement. [Source]

3. Foundry Agent Service (classic)

This is where the confusion peaks. Foundry Agent Service (classic) is a different, broader agent runtime than the Assistants API, and it has its own, later retirement date: March 31, 2027. If your workload calls the Assistants API directly, the August 26, 2026 deadline applies to you regardless of whether you also happen to touch the Foundry Agent Service (classic) elsewhere. [Source]

A Step-by-Step Guide: How to Migrate From Azure OpenAI Assistants API

A successful Azure OpenAI Assistants API migration requires more than replacing API calls. The following steps provide a practical migration path for AI developers.

1. Audit your codebase and subscription for Assistants usage

The first step is identifying every place where your organization uses the Assistants API. Currently, no single automated scanner provides a complete inventory of Assistants API dependencies across applications, subscriptions, and integrations. AI engineers need to combine multiple discovery methods to avoid missing hidden dependencies. Start by searching application repositories, notebooks, and scripts for references to assistants, threads, runs, AssistantClient, and AgentsClient.

Also review the Foundry classic portal and analyze Azure Monitor diagnostic logs or Application Insights data to identify requests targeting legacy endpoints such as /openai/assistants, /openai/threads, and /openai/runs.

2. Update SDKs and Packages for your Development Stack

The Azure OpenAI Assistants API migration requires updating language-specific SDKs and dependencies before replacing application logic. AI developers, hence, should pin dependency versions during migration rather than allowing automatic updates that could introduce unexpected changes during testing. The required changes vary by ecosystem:

• Python: Move to the current openai package and remove deprecated dependencies such as azure-ai-inference if you still use them.

• .NET: Adopt current packages including Azure.AI.Projects, Azure.AI.Projects.Agents, and Azure.Identity.

• Node.js/JavaScript: Use @azure/ai-projects and @azure/identity with supported Node.js versions.

• Java: Update to the current azure-ai-agents and azure-identity artifacts.

3. Migrate Agent Definitions

In the Assistants API model, teams defined an Assistant object containing the model configuration, instructions, and tool declarations. In the new architecture, these capabilities move into an Agent definition within the Microsoft Foundry Agents service. The migration requires reviewing and rebuilding:

• Model configuration

• System instructions

• Tool definitions

• Function declarations

• Retrieval or file-based capabilities.

4. Replace Threads and Runs with Conversations and Responses

This is the structural heart of the migration, replacing the Threads and Runs execution model. The newer architecture introduces a different model based on agents, conversations, and responses. Although conversations provide more streamlined state management, this change requires application-level updates. AI experts cannot simply rename Threads as Conversations or Runs as Responses because the execution flow, state handling, and tool interaction patterns have changed. The old object model and its replacement map roughly like this:

5. Re-Test State, Tool Calls, Outputs, and Error Handling

The mitigation plan for Azure OpenAI Assistants API retirement should not be treated as an API replacement followed by immediate deployment. Every critical workflow needs validation under the new execution model. AI development experts should also review any implementation that depends on synchronous Run polling because response-based execution follows a different interaction pattern. The entire testing should cover the following:

• Conversation state management

• Function calling behavior

• File search workflows

• Code interpreter execution

• Response accuracy and formatting

• Timeout handling

• Partial failures

• Tool execution errors.

6. Plan for Historical Thread Data Migration

This step trips up most AI development teams because no automated migration path exists from Threads to Conversations. OpenAI has stated explicitly that it will not build a Thread-to-Conversation migration tool, and Microsoft's guidance follows the same pattern on Azure. Without this preparation, historical context may remain accessible only through the legacy API until the retirement deadline, after which applications will no longer be able to retrieve it.

Organizations that need historical context for customer interactions, enterprise knowledge workflows, or compliance requirements must create their own migration strategy. This may involve:

• Exporting existing thread data before retirement

• Transforming historical messages into the new conversation format

• Writing required context into new agent workflows

• Defining retention policies for archived conversations.

Common Azure OpenAI Assistants API Migration Pitfalls

Beyond updating SDKs and replacing API calls, migrating the Azure OpenAI Assistants API involves several hidden challenges that can delay execution if overlooked. Understanding them early helps organizations avoid data loss, unexpected integration failures, and last-minute production risks.

No Direct History Migration Path

Existing conversation history cannot be moved through a one-click migration process. Therefore, organizations that need to preserve historical context must build their own extraction, transformation, and backfill workflows before the retirement deadline.

Not Exporting Critical Data Before the Cutoff

Existing Thread and Run data remains accessible only through the legacy API until the Assistants API retirement date. After August 26, 2026, the associated endpoints will no longer be available, so teams must export and preserve any required historical information before the cutoff.

Compromising Audit on External Integrations

Azure OpenAI Assistants API migration risks are not limited to internally developed applications. Every automation platform, workflow tool, and low-code integrations that rely on Assistants API capabilities may also require updates. For example, platforms such as Zapier have deprecated Assistants-based actions, requiring users to rebuild workflows using newer integration methods.

Not Treating August 26, 2026 as a Hard Deadline

Unlike some cloud service transitions that provide overlapping support periods, the Assistants API endpoints will stop functioning after August 26, 2026. Organizations that postpone Azure OpenAI Assistants API retirement planning may face production disruptions and unexpected dependency issues as the deadline approaches.

Confusing Microsoft Foundry and Assistants API Retirement Timelines

Some teams assume the March 31, 2027 retirement of Microsoft Foundry Agent Service (classic) gives them more migration time. However, that timeline applies to classic agent workloads and does not change the August 26, 2026 deadline for applications using Assistants, Threads, and Runs endpoints.

Move Before the Migration Window Closes

The Azure OpenAI Assistants API retirement creates a narrow window for organizations to assess their AI workloads, address hidden dependencies, and complete migration without disrupting business-critical applications. Organizations that treat this as a last-minute API update may face limited testing time, unexpected integration issues, and operational risks when legacy endpoints stop working.

For organizations that might lack the internal expertise or bandwidth to manage this large-scale AI migration, it is better to hire dedicated AI developers. This can help the internal engineering team accelerate code audits, architecture changes, testing, and production readiness. The priority should be to establish a clear migration roadmap now, so existing AI investments can continue delivering value on a supported agent foundation beyond the retirement deadline.

Frequently Asked Questions (FAQs)

How long does Azure OpenAI Assistants API migration take?

The Azure OpenAI Assistants API migration timelines depend on application complexity, integrations, custom tools, and historical data requirements. Simple implementations may require limited changes, while enterprise AI applications with multiple workflows need additional time for redesign, testing, and production validation.

Is there an Azure OpenAI Assistants API alternative?

Microsoft recommends migrating Assistants API workloads to the Microsoft Foundry Agents service, which provides the next-generation architecture for building and managing AI Agents with updated capabilities and workflows.

How should organizations prepare for Azure OpenAI Assistants API migration?

Organizations should first audit their applications, integrations, and automation workflows for Assistants API dependencies. Then update SDKs, migrate agent definitions, replace Threads and Runs with the new architecture, and thoroughly test tools, outputs, and application behavior. Another approach is to hire AI developers who have relevant experience in Azure OpenAI services, Assistants API migration, Microsoft Foundry Agents service, and agent workflow redesign.

Today, both approaches are widely used. Cloud AI has a growing market, projected to grow up to 133 billion in 2026. The on-device AI market is also catching up at USD 13.6 billion in the same year. Broadly, on-device AI wins on speed, privacy, and offline reliability, while cloud AI wins on raw model power and scalability. But which one is right for a given feature depends on more than infrastructure alone. The cost at scale, compliance exposure, and what your users actually expect from the experience—all factor into this decision. [Source: GrandViewResearch, Fortune Business Insight]

It sounds like a back-end decision, but it really is an architectural one. And this decision locks in your cost curve, your compliance posture, and your user experience. This is exactly why the on-device AI vs. cloud AI question is landing on CXO desks now instead of staying buried in an engineering ticket.

This piece breaks down on-device AI vs. cloud AI in plain terms, backs it with current data, and ends with a decision framework you can actually apply.

On-Device AI Vs. Cloud AI: The Quick Answer

If you want the short version before the details: on-device AI runs the model directly on the user's phone, using the chip's built-in AI accelerator, with no network call involved. Cloud AI sends a request to a hosted model on a remote server and waits for a response.

1. Choose on-device when the feature needs to be instant, needs to work offline, or touches sensitive data you'd rather not transmit at all.

2. Choose cloud AI when the task needs frontier-level reasoning, a large context window, or model capability that current mobile hardware simply can't run locally.

3. Choose both (which is where most mobile apps actually land) when your app has a mix of fast, frequent, sensitive tasks alongside occasional, complex ones. The rest of this post explains why.

On-Device AI: What is it, How Does it Work, and Where it Wins

On-device AI (also called edge AI in this context) means the model, usually a compact, quantized version, is bundled with or downloaded into the app. It runs on the phone's own silicon, such as Apple's Neural Engine, Qualcomm's Hexagon NPU, or Google's Tensor chips. In terms of deployment, frameworks like Core ML and newer options like Apple’s on-device AI Foundation Models are often used. There is no need for a server because the phone does everything.

This kind of an app architecture wins on four fronts:

Speed

A cloud AI API call typically takes 100–500ms just in network round-trip time, before the server even starts inferring. On-device AI inference skips that entirely by using quantized models on modern NPUs, which commonly return results in 5–20ms.

Privacy and Compliance Exposure

When inference happens locally on the device, sensitive data (health records, financial details, biometric scans) never leaves the device. This simplifies GDPR, HIPAA, or CCPA compliance. And that compliance can save millions. IBM's 2025 Cost of a Data Breach Report puts the global average cost of a breach at $4.44 million. Healthcare breaches are worse, averaging $7.42 million. Every data flow you can keep off the wire is a data flow that can't show up in that number.

Offline Reliability

On-device AI simply keeps working without an internet connection. Several field tools, travel apps, and rural or low-connectivity markets all depend on this functionality. ITU estimates that roughly 2.2 billion people, or 26% of the global population, remained offline in 2025. Any feature that depends on a live network call simply does not exist for that slice of your user base, or for anyone mid-connection-drop. [Source: ITU]

Cost at Scale

Once you have built and shipped it, on-device inference has close to zero marginal cost per request. You pay for integration and optimization once, not per query. Cloud AI runs the opposite way: ongoing API usage, monitoring, and model updates typically add another 15–25% of the initial build cost every year.

Cloud AI: What is it, How Does it Work, and Where it Wins

Cloud AI keeps the model on remote infrastructure. The app sends a request over the network (text, an image, or a prompt) to a hosted model from a provider like OpenAI, Anthropic, or Google, and gets a response back. Nothing runs locally beyond the API call itself. This is where the advantage comes into play.

That makes cloud AI the right call for:

Model Capability

Cloud models carry far more parameters and meaningfully stronger reasoning ability than anything that fits on a phone today. Plus, the provider improves the AI model centrally without you shipping a line of app code. This is highly advantageous for complex reasoning, multi-step workflows, and long-form content generation.

Context Window and Output Quality

Cloud-hosted AI models support far larger context windows than on-device options. For any feature where the user is willing to trade a few hundred milliseconds of latency for a noticeably better answer (model quality matters more than response time), cloud is the right default.

Cost at Low-to-Moderate Volume

Model efficiency gains have pushed inference costs down roughly 10–20x over the past two years, and at low-to-moderate request volumes, per-request API pricing stays genuinely manageable. This is often cheaper than the engineering cost of building and maintaining an on-device AI model for the same feature.

Iteration Speed

Because the model lives on the provider's infrastructure, you can swap or upgrade it as and when needed. This could be a newer model version, a different provider, or a fine-tuned variant, and this can be done without shipping a corresponding app store release. For teams iterating fast on prompt design or model selection, that is a development-velocity advantage that on-device AI architectures can't match.

Actual Decision Framework: How to Choose

Most comparison content stops at it depends. We will not do that in this one. Here is a QandA framework to run per feature (not once for the whole app), since most apps will get a mixed answer across their own feature set. Assess the following questions:

1. Does this feature need to work with no or unreliable connectivity? If yes, on-device AI isn't a nice-to-have; it is mandatory. Cloud-only simply is not an option here, as it relies on internet connectivity.

2. Is the data sensitive enough that it is itself the risk, and not just how it is handled after? Health, financial, and biometric data fall in this high-risk category. If the answer is yes, weigh the cost of on-device AI processing against the cost of a breach in your industry (again, healthcare tops IBM's list at $7.42 million per incident).

3. Does the UX need to feel instant, under roughly 100ms? If users notice a delay (camera-based features, live suggestions, real-time translation), the 100–500ms cloud round-trip is a real UX tax. On-device AI is the better fit.

4. Does the task genuinely require frontier-model reasoning, or is it closer to classification, extraction, or rewriting? Be honest here. Most AI features requested in most apps fall into the second bucket, which open-weight, on-device AI models handle well.

5. What does usage volume look like in 12–18 months, and does the cloud cost curve stay sane at that volume? Model a realistic user growth scenario, not just your current MAU (monthly active users). API costs that look fine in a pilot can become the biggest line item in your infrastructure budget within a year of real growth.

6. Does your team have the ongoing bandwidth to maintain an on-device AI model? Can it consistently support versioning, device-tier testing, and storage footprint? Or is that a "later" problem you're not ready to solve right now? On-device AI is not one-and-done. It carries real maintenance overhead that needs a home on your roadmap. If you cannot afford that, cloud AI might be the right choice for you.

Why Most Mobile Apps End up Hybrid: Cloud AI + On-Device AI

In practice, leading platforms have already found an answer to this question, and that is with a pattern, not a single choice. Apple's on-device Foundation Models framework and Google's Gemini Nano are specifically designed to handle the fast, frequent, private slice of AI work locally. Despite that, both companies still lean on cloud infrastructure for anything that needs bigger models. Qualcomm's own leadership has made the same case publicly, emphasizing that the future is not cloud or edge, but a compute architecture spanning both.

The pattern that has emerged: route fast, frequent, or sensitive tasks to on-device inference, and reserve cloud calls for complex reasoning that justifies the added latency and cost. A single app might run text prediction and camera-based detection on-device while sending an open-ended support query to the cloud — all in the same session, without the user noticing the switch.

Closing Thoughts

Cloud API or on-device AI isn't a technology preference. It is a build decision made at architecture time, with cost, privacy, and UX consequences that compound over the life of the app. The mobile app development projects getting this right are routing each feature to wherever it performs best, and building that routing logic in from the start rather than retrofitting it after a cloud bill or a compliance review forces the issue.

Run the 6-question framework above against your next AI feature before you write a line of inference code. In most cases, you won't get a single answer for your whole app, and that's exactly the point.