AI Agents in Production
From assistants to actors
An AI agent is a language model placed in a loop with tools. Instead of answering a single prompt, it receives a goal, plans a step, calls a tool, inspects the result, and repeats until the work is done or a boundary stops it. That loop is what separates an agent from a chat application or a retrieval pipeline, and it changes the engineering problem completely. A chat app that gets something wrong produces a bad paragraph. An agent that gets something wrong sends the email, merges the pull request, or approves the refund.
The reason agents dominate enterprise AI conversations now is that the underlying pieces matured at roughly the same time. Models became reliable enough to sustain long, multi-step tasks. The Model Context Protocol gave the industry a shared way to connect models to tools and data, so integrations stopped being bespoke. And the major platforms turned agent hosting into a managed service, with Azure AI Foundry and its competitors offering runtimes and identity integration out of the box. For engineering leaders in Sweden and the wider EU, the question has shifted from whether agents can do useful work to how to run them with the same discipline as any other production system: access control, cost budgets, observability and an audit trail.
The decisions that shape an agent programme
Build or buy the loop
The core agent loop is small enough that many teams are tempted to write their own, and mature enough that vendors now sell it as a product. Open frameworks, vendor SDKs and fully managed agent services each trade control for convenience differently. A hand-rolled loop is easy to debug and hard to keep current as protocols evolve. A managed runtime handles hosting and upgrades but couples you to a platform's assumptions about state, memory and tool invocation. The right answer usually follows from where your workloads already live and how much of the stack your team wants to own long term.
Tools are the real interface
Most agent failures in production trace back to tools, not models. A tool with a vague schema invites the model to guess. A tool that is not idempotent turns a harmless retry into a duplicate payment. Designing tools with strict schemas and safe re-execution semantics is unglamorous work, but it is where reliability is actually won. Standardising on a protocol such as MCP helps here too, because it forces tool contracts to be explicit and makes them portable across models and runtimes.
Security, sandboxing and identity
Because agents act rather than answer, security cannot rely on the model's own judgement. A model that promises to classify its actions as safe is not a sandbox; isolation has to be enforced outside the model, through restricted execution environments, network and domain allowlists, and scoped credentials. Identity is the newer half of this problem. When an agent calls internal APIs, it needs its own identity in your directory, with least-privilege permissions and a revocation path, rather than borrowing a service account or a developer's token. Directory-native agent identity, such as Microsoft's Entra Agent ID, exists precisely because auditors will eventually ask who did what.
A useful mental model: treat every production agent like a new employee with system access. It gets its own identity, the minimum permissions for its job, supervision proportional to the damage it could do, and a manager who reviews its work and its expenses.
The cost math is different
An agent can consume one hundred times the tokens of the chat interaction it replaces, because every loop iteration re-reads the accumulated context and every retry multiplies the total. Budgeting agents like chat workloads leads to unpleasant invoices. The levers are well understood: right-size the model per step rather than using the flagship everywhere, cache the stable parts of prompts, cap iterations, and measure cost per completed task instead of cost per token. Model choice is a recurring decision, not a one-off, since vendor pricing and capability tiers shift several times a year.
Scope: one agent, many agents, or a computer user
Multi-agent architectures are attractive on whiteboards and expensive in production. The patterns that ship tend to be modest: a planner delegating to a small number of specialists, or a pipeline with explicit handoffs, rather than free-form agent swarms. Computer-use agents, which drive real user interfaces through screenshots and clicks, occupy a separate niche. They compete with RPA for legacy systems that lack APIs, and the comparison is economic as much as technical: slower and costlier per run than a script, but far cheaper to adapt when the UI changes.
How to approach it
Teams that get agents into production reliably tend to follow a similar path.
- Start with a narrow workflow that has a measurable outcome and a human consuming the result, then widen autonomy as evidence accumulates.
- Build evaluation before building features. Agents need behavioural test suites that check what the agent did, not just what it said.
- Instrument from day one. Tracing every decision and tool call is what makes incidents debuggable and costs attributable.
- Put governance in the design, not the retrofit. The EU AI Act's transparency duties and GDPR's data protection requirements are cheaper to satisfy in the architecture than in an audit response.
None of this is exotic. It is the ordinary discipline of running software that acts on your behalf, applied to a component that is probabilistic. The articles below go deeper on each of these decisions, from framework selection and tool design to identity, sandboxing, evaluation and the cost models that keep an agent programme fundable.
All articles on this topic
This guide grows as we publish — newest articles appear here automatically.
A published attack chain achieved remote code execution against Claude Code in auto mode, with the safety classifier approving the steps that led to compromise and then blocking the cleanup command. What the break teaches teams running AI coding agents, and how to build real isolation on Azure with Hyper-V sandboxes, default-deny egress and short-lived credentials.
Anthropic's Model Hardware Standard, released as a research preview on 27 August, gives AI agents a standard way to operate microscopes, liquid handlers and robotic arms through read and write primitives exposed over MCP. Launch partners report instrument integrations in hours instead of weeks, and the timing puts agent-operated hardware on a collision course with the EU Machinery Regulation that applies from January 2027.
The Azure OpenAI Assistants API reached its retirement date on 26 August 2026, and the classic Foundry Agent Service it underpins retires 31 March 2027. A step-by-step migration guide to the new Foundry Agent Service on the Responses API: threads become conversations, runs become responses, assistants become versioned agents, and Microsoft's migration tool rewrites code but not stored state.
The MCP core maintainers published a new roadmap on 22 August 2026, four weeks after the 2026-07-28 spec made the protocol core stateless: five priorities covering agentic messaging, HTTP-native transport unification, agent identity with DPoP and Workload Identity Federation, improved primitives, and SDK developer experience. Here is how Azure-first EU teams should sort that list into adopt now, prepare, and watch.
Anthropic's 19 August GA wave took computer use out of beta, launched a browser use toolset, and moved the Files API, Agent Skills and enterprise user management to production on the Claude API. The same release added allowed_domains and blocked_domains restrictions for agent web tools: how the matching rules work, when to allowlist versus blocklist, and what the controls mean for EU governance.
Google shipped gemini-3.7-flash as generally available on 13 August 2026 at an introductory $0.75/$3.75 per million tokens, two weeks after OpenAI cut GPT-5.6 Luna by 80 percent and days after Anthropic locked Claude Sonnet 5 at $2/$10 permanently. We compare the three workhorse models on list price, context, Azure availability and EU residency, and show why cost per completed task beats cost per token.
Microsoft's Agent Framework Harness and Foundry Hosted Agents reached general availability in early August 2026, wrapping models with function invocation, context compaction, planning, memory, tool approvals and OpenTelemetry in a runtime that runs locally, in containers, or consumption-billed on Foundry. A decision framework for Azure teams weighing the GA runtime against a hand-rolled agent loop, with cost levers and the Swedish compliance picture.
The Model Context Protocol specification version 2026-07-28 shipped on 28 July with a stateless core, Multi Round-Trip Requests, required routing headers, cacheable list results and a hardened authorization model that deprecates Dynamic Client Registration in favour of Client ID Metadata Documents. All four Tier 1 SDKs shipped support on release day, so Azure teams running MCP servers on Container Apps, AKS or behind API Management can start migrating immediately.
Claude Opus 5 launched on 24 July 2026 at $5/$25 per million tokens with a 1M context window and day-one availability in Microsoft Foundry. For long-running agents the per-token price is the wrong unit: we work through cost per completed task against Sonnet 5 and GPT-5.6 Sol, and flag the EU data-residency caveat Swedish Azure teams need to check first.
GPT-5.6 (Sol, Terra, Luna) went GA in Microsoft Foundry on 9 July 2026, day-and-date with OpenAI, alongside a new Asia-Pacific Data Zone and a hosted agents runtime with VNet integration. A practical guide for Swedish and EU Azure teams: choosing between the three models, picking Global Standard versus EU Data Zone versus PTUs, worked cost math on the launch prices, and a two-week adoption checklist.
Anthropic launched Claude Sonnet 5 on 30 June 2026 at an introductory 2/10 dollars per million tokens, posting 63.2% on SWE-bench Pro and near-Opus agentic performance at 40-60% of the cost per task. We run the cost math against Opus 4.8 and GPT-5.5, set out a routing framework for when the cheap model wins, and draw the continuity lesson from the eighteen-day Fable and Mythos export-control pause.
Google made computer use a native tool in Gemini 3.5 Flash on 24 June 2026, putting vision-based UI agents in its low-cost tier for browser, mobile and desktop automation. A comparison with Azure AI Foundry's Computer Use and Browser Automation previews, with a decision framework for legacy-UI automation and the GDPR implications of shipping screenshots to a model endpoint.
Microsoft made Copilot Cowork generally available worldwide on 16 June 2026, giving every Microsoft 365 Copilot tenant an agentic system that plans and delivers multi-step work, billed through Copilot Credits at 0.01 dollars each with no usage bundled into the license. Swedish IT teams now need spending limits, a DPIA update for the Anthropic-by-default model lineup, and a clear answer on when Cowork beats building a Foundry agent.
GitHub’s new Copilot desktop app runs multiple coding agents in parallel, each session isolated in its own git worktree, and is already included in Copilot Pro, Pro+, Business and Enterprise subscriptions. Getting value from it depends less on prompting than on task selection, worktree hygiene, supervision cadence and engineered review capacity, because five agents can open pull requests faster than one reviewer can read them.
Microsoft Build 2026 turned Foundry into a full production-agent stack: Foundry IQ for unified retrieval, Toolboxes for managed tool access, agent memory, Voice Live and the experimental Scout Autopilot. Foundry IQ knowledge bases and Voice Live are generally available now, Toolboxes and memory sit in public preview, and Scout remains experimental, which sets the build, pilot and watch lanes for an Azure-first EU team.
Claude Opus 4.8 arrives at unchanged pricing with an effort control on all plans, a fast mode at a third of the previous fast-inference cost, and a Messages API change that lets system entries sit inside the messages array so mid-task instruction updates no longer invalidate the prompt cache. Worked token math shows cache hit rate remains the biggest cost lever, and a four-question framework matches effort, speed and fan-out to each workload.
Microsoft's 26 May Copilot Studio update takes computer-using agents to general availability, ships a redesigned workflows designer with agent nodes, and moves A2A communication and remote MCP support to GA. CUA runs bill at 5 credits per agent action, which makes adaptive UI agents cheaper than classic RPA at modest volumes and costlier at high, steady ones.
MCP's 2026-07-28 release candidate removes the initialize handshake and the Mcp-Session-Id header, so every request is self-contained and Azure-hosted MCP servers can run behind plain load balancers without sticky sessions or Redis session stores. Tasks and MCP Apps land as formal extensions, six SEPs harden authorization around OAuth 2.0 and OpenID Connect, and Roots, Sampling and Logging enter a 12-month deprecation window.
Google's I/O 2026 enterprise announcements, led by the Managed Agents API, Gemini Spark and Gemini 3.5 Flash, take direct aim at Azure AI Foundry Agent Service and the Copilot ecosystem, down to launch connectors for SharePoint and OneDrive. For Azure-first Swedish and EU teams the decision rests on four questions: where the data lives, whether the cost claim survives real traces, whether audit obligations can be met, and what a second platform costs.
Google shipped Gemini 3.5 Flash on day one of I/O 2026, the third frontier agentic coding model in five weeks after Claude Opus 4.7 and GPT-5.5, alongside a $100/month developer tier and the Antigravity 2.0 agent platform. For Azure-first teams the practical response is a reusable evaluation harness and a documented cost comparison for renewal leverage, not a migration.
Unit tests cover deterministic functions. Agent loops are not deterministic. The evaluation gap is where most production agent failures live, and where the regressions are easiest to catch with a small amount of disciplined infrastructure. Three eval dimensions, how to build a labelled set, and where LLM-as-judge actually works.
The AI agent SDK landscape in 2026 has narrowed to a handful of serious contenders, each with distinct strengths. A working comparison of Claude Agent SDK, LangGraph, AutoGen, Microsoft Agent Framework, and Mastra, with a decision matrix that lets you pick on the criteria that actually matter for your project.
Standard APM was built for HTTP requests and database queries. Agent runs produce a different signal shape: nested decisions, tool calls, hallucination signals, accumulating context. The observability stack has caught up. Here is the production setup that makes agents debuggable.
Agent loops cost 10x to 50x what a chatbot interaction costs; multi-agent systems add another order of magnitude. The cost compounding is structural, not a bug. The cost reduction is structural too. Decomposing where the tokens go and how to bring agent economics back from runaway to acceptable.
Most multi-agent demos collapse in production. The patterns that survive contact with real workloads are smaller in number and more disciplined than the demos suggest. Planner-worker, supervisor-team, critic-reviser, and hierarchical compositions, with the cost and failure tradeoffs that decide which one to pick when.
Browser-based AI agents moved from labs to early production in 2025 and 2026. Anthropic Computer Use, Microsoft Magentic, and OpenAI Operator each take a distinct architectural bet. This is the working comparison for engineers deciding which to deploy and which workloads they actually fit.
A production-shaped guide to designing AI agent tools that the model can actually use without breaking things. Schema choices, idempotency keys, error responses the model can act on, granularity tradeoffs, versioning, and the patterns that separate demo-quality tools from ones that hold up in real workloads.
A deep dive into Microsoft Entra Agent ID, the control plane for AI agent identity in 2026. Covers identity blueprints, attended and unattended authentication, tool-level RBAC, conditional access, OBO flows across multi-agent systems, and the audit logging that satisfies DORA, NIS2, and AI Act obligations.
The capability spectrum from stateless LLM to multi-agent orchestration is one of the most conflated concepts in the 2026 AI market. The distinctions matter. They change architecture, they change cost by an order of magnitude, and under the EU AI Act they change compliance posture.
Classic RAG hits a ceiling when questions require multi-hop reasoning or query refinement. Agentic RAG — treating retrieval as a tool, decomposing queries, adding self-correction loops — routinely wins where classic RAG plateaus.
One year after MCP shipped, this is what adoption actually looks like — server ecosystems, integration patterns, security concerns like tool poisoning and prompt injection, and the open questions heading into 2026.
A year-end, hype-free review of where agentic AI actually shipped in 2025, which patterns graduated from pilot to production, and which still need work before they are safe to trust with consequential decisions.
Microsoft Ignite BRKSP486: Citrix and Microsoft reveal the 7-stage AI maturity framework. Computer use agents operate existing apps without APIs. Progress from shadow AI to autonomous orchestration with secure infrastructure.
As AI evolves from passive tools to autonomous agents, accessibility and responsibility become deeply interconnected imperatives. Discover how Microsoft is shaping safe, inclusive, and trustworthy agentic AI through the PIRATE framework, diverse datasets, and accessibility-by-default tools.
Microsoft 365 Copilot unifies Chat, Search, Agents, Notebooks, and Create into one secure workflow. Discover how Work IQ, AI-enhanced search, autonomous agents, and integrated content creation eliminate context switching and boost productivity.
Discover how Microsoft 365 Copilot Chat with GPT-5, enhanced file uploads, voice interaction, Work IQ integration, and chat-first workflows transforms daily work. Create files from conversations, manage emails intelligently, and access AI assistance everywhere.
See the magic happen in real time. Learn how to train and deploy custom reasoning models with Azure ML and Microsoft Foundry—from fine-tuning to reinforcement learning, performance optimization with speculative decoding, distillation, and production deployment delivering measurable ROI.
Microsoft Ignite BRK203: Reasoning models as the brains behind autonomous agents. Multi-step problem solving, explainable decisions, agentic workflows (lead scoring, content generation, support). Foundry 11,000+ model catalog, customer stories from healthcare and legal sectors.
Agents need context. Discover Foundry IQ—the knowledge layer connecting AI agents to enterprise data with multi-source RAG orchestration, retrieval steering, dynamic security controls, and agentic RAG delivering 36% higher accuracy than traditional search.
From scientific research to enterprise AI transformation, discover how Microsoft Foundry unifies models from OpenAI, Anthropic, Cohere, Meta, and more into one secure platform. Learn intelligent model routing, cost optimization, and the game-changing Claude integration.
Microsoft Ignite BRK194: Model Context Protocol (MCP) standardizes agent-tool communication across platforms. Azure API Center integration, federated registries, cross-cloud orchestration, and enterprise governance for scalable agentic ecosystems.
Ride the agent revolution with Microsoft Foundry, the enterprise-ready Agent Factory. Build, test, and launch intelligent agents with 1,400+ tools, 11,000+ models, multi-agent orchestration, and seamless Microsoft 365 integration—all with bulletproof security and governance.
Microsoft Ignite BRK116: Deploy AI agents at scale with Azure App Service, AI Foundry, and MCP tools. Built-in observability, governance, security. Hitachi case study shows 73% downtime reduction and 41% cost savings.
Teams with strong DevOps practices are best positioned to harness AI power. Discover how GitHub Copilot and AI agents bring unprecedented speed, scale, and security across the software development lifecycle—from planning to production.
Frequently asked questions
What is the difference between an LLM application and an AI agent?
An LLM application makes a single model call per request, so the model only produces text. An agent runs the model in a loop where it chooses tools, observes results and decides the next step, which means it can take actions with real consequences. That autonomy is why agents need identity, sandboxing and evaluation practices that a chat app does not.
Why do AI agents cost so much more to run than chatbots?
Each loop iteration re-sends the accumulated conversation and tool results as input tokens, so a task involving dozens of steps can consume far more tokens than a single chat exchange. Costs are managed by using smaller models for routine steps, caching stable prompt prefixes, capping iterations and tracking cost per completed task rather than per request.
How do we secure an agent that has access to internal systems?
Enforce isolation outside the model: run tool execution in sandboxed environments, restrict network and domain access with allowlists, and give the agent its own directory identity with least-privilege scopes instead of shared service accounts. Log every tool call so actions are attributable in an audit. Never rely on the model classifying its own actions as safe.
Should we build our own agent loop or use a framework or managed service?
If your workloads already run on a platform like Azure, its managed agent services reduce operational work and keep pace with protocol changes such as MCP revisions. A custom loop gives full control and easier debugging but becomes maintenance you own. Many teams prototype with a framework and only replace pieces they have concrete reasons to control.
How do we test an agent before putting it in production?
Build an evaluation suite that scores agent behaviour on realistic tasks: whether the right tools were called with the right arguments and whether the end state is correct, not just whether the final answer reads well. Run it on every model or prompt change, and pair it with production tracing so real failures feed back into the test set.
Do EU rules like the AI Act apply to AI agents?
Yes. Agents are covered by the same framework as other AI systems, including transparency obligations when users interact with AI and stricter duties if the use case falls into a high-risk category. GDPR applies whenever agents process personal data, so data residency and access scoping belong in the architecture from the start.