Azure OpenAI On Your Data is retired: your Foundry IQ path

On 14 October 2026 Microsoft retired Azure OpenAI On Your Data. The Learn page now opens with a single sentence: the feature "retired on October 14, 2026, and is no longer supported by Microsoft." Every chat completions request that carries a data_sources block, or that hits the legacy /extensions/chat/completions path, is now calling a feature Microsoft no longer stands behind. The replacement is a Foundry IQ knowledge base on Azure AI Search, with Foundry Agent Service as an optional layer on top. Microsoft is explicit that "migration isn't automatic" and that the replacement paths "aren't feature-equivalent replacements." If you run a grounded chat application on Azure in Sweden or elsewhere in the EU, this is the week to find those calls and decide which of two replacement paths you take.

What retired, and what did not

The retired feature is the On Your Data extension to chat completions: the service-side retrieval that took your Azure AI Search index, Blob container, Cosmos DB for MongoDB vCore collection, Elasticsearch cluster or uploaded files and injected retrieved chunks into the prompt before the model answered. Microsoft's own guidance for identifying affected applications is a search through requests, code and configuration for three signals: data_sources, the older camel-case dataSources, and the /extensions/chat/completions URL path.

# Find On Your Data usage across a repo and its config
grep -rnE "data_sources|dataSources|/extensions/chat/completions" \
  --include="*.py" --include="*.cs" --include="*.ts" --include="*.js" \
  --include="*.json" --include="*.yaml" --include="*.yml" --include="*.bicep" .

Two things did not retire. Ordinary Azure OpenAI chat completions without a data source configuration are untouched. And the On Your Data retirement is separate from model retirement, even though the two collide: before retirement the feature supported only GPT-4o (versions 2024-05-13, 2024-08-06 and 2024-11-20) and GPT-4o-mini (2024-07-18), all of which sit on Microsoft's model retirement schedule in their own right. If your application was pinned to one of those versions purely because On Your Data required it, the migration is also your chance to move to a current model.

Also unchanged: the Azure AI Search index that On Your Data built for you. Indexes produced by file upload, Blob Storage or web ingestion remain compatible, and the migration sample is built around reusing them. What does not carry over is the service glue around that index. In Microsoft's words, "connector behavior, access control lists, and embeddings don't convert automatically," and scheduled indexers need to be evaluated separately.

The replacement: a knowledge base, with or without an agent

Foundry IQ is Microsoft's managed knowledge layer. A knowledge base groups one or more knowledge sources (your existing search index, Blob Storage, OneLake, SharePoint, or the web) behind a single retrieve endpoint, and the agentic retrieval engine in Azure AI Search does the work: it can plan subqueries with an LLM, run them in parallel, rerank each with the semantic ranker and merge the results with citations. Knowledge bases reached general availability on 2 June 2026, so the replacement is not a preview product, though several of the features that make it feel like On Your Data still are.

The migration guidance offers two paths, and the choice is simpler than the marketing suggests:

• Call the knowledge base directly. Microsoft recommends this for corpus question-and-answer, which is what most On Your Data deployments were. The notebook is blunt: "An agent is not required just to answer corpus questions, carry conversation history, or add another knowledge source."
• Wrap it in a Foundry Agent Service agent. Choose this only when the application needs tool calls or multi-step workflows beyond retrieval. The knowledge base and knowledge source stay the same; the agent becomes the outermost layer.

Foundry Agent Service is not required for Foundry IQ at all. You can call a knowledge base from Microsoft Agent Framework, from the Foundry IQ MCP server, or from any application that speaks the Azure AI Search knowledge base REST API or SDKs. For a team that had a thin web backend calling chat completions with a data source attached, the direct path is a like-for-like swap of one HTTP call for two: retrieve, then generate.

The decision that matters: GA retrieval or preview answers

Here is the catch Microsoft puts in the retirement notice itself. The migration sample uses auto retrieval and answer synthesis, and both require the 2026-08-01-preview API. Preview features "aren't recommended for production." The generally available alternative is 2026-04-01, which gives you minimal, extractive retrieval only: no LLM query planning, no answer synthesis, no multi-turn messages. Your application generates the answer and owns the conversation. The retirement notice says plainly that the two paths "aren't feature-equivalent replacements" for what you had.

The GA retrieve call is small. Your backend sends an intent, receives extractive chunks with references, and then calls whatever model you deploy to write the answer. That is the same shape as classic RAG, which Microsoft also lists as an acceptable GA destination.

POST {search-endpoint}/knowledgebases/{kb-name}/retrieve?api-version=2026-04-01
Content-Type: application/json
Authorization: Bearer {caller-or-app-token}

{
  "intents": [
    { "type": "semantic", "search": "What is our travel reimbursement limit for 2026?" }
  ]
}

Our recommendation for a regulated Swedish or EU workload: migrate to the GA path first, with your own answer generation, and treat query planning and answer synthesis as a later optimisation once they leave preview. A grounded chat application that was in production yesterday should not land on preview terms today.

How the On Your Data parameters map

Microsoft's migration notebook publishes a mapping from the old data_sources parameters to knowledge base settings. The important column is where each setting now lives, because several moved from per-request to per-resource.

Three behaviours deserve a test case each. First, strictness has no equivalent; the reranker threshold is a different dial and needs its own evaluation on your corpus. Second, index scoring profiles "are not honored by KB retrieve," so any boosting you tuned into the index for freshness or document type silently stops applying. Third, the new filter "replaces, rather than appends to, the static default," so a per-tenant filter you used to layer on top of a base filter must now be assembled in full on every request.

Citations and conversation history move into your code

On Your Data returned citations in its own shape inside the chat completion. A knowledge base returns references with [ref_id:<id>] markers, and the notebook warns that your application must "adapt to KB references rather than OYD's citation shape; this is not a wire-compatible drop-in replacement." If your frontend parses citation objects, that parser changes.

Conversation history is now yours. "A KB retrieve call does not create a persistent chat session." On the GA path you send a fresh intent for each follow-up and keep the transcript in your application; on the preview path you send prior turns in messages. Either way, the trusted filter and the user's token must be supplied again on every follow-up, not just the first turn.

The security model is stricter, and that is the point

The sharpest line in the migration guidance is about authorization: "Never replace per-user checks with a system prompt or an admin key." On Your Data could enforce document-level access when the source was Azure AI Search, but many deployments ran with a service key and relied on the system prompt to keep users in their lane. Foundry IQ wants the verified signed-in user's Search token on every retrieve call, passed as query source authorization, with ACLs synchronised into the index. Permissions are enforced only when the knowledge source supports document-level security and that synchronisation has been configured; a plain index with no ACL fields gives every caller everything.

The identity split Microsoft recommends is worth adopting even if you did not have it before:

• Setup principal: Search Service Contributor, used once to create the knowledge source and knowledge base.
• Query identity: Search Index Data Reader, used by the backend for every retrieve call.
• Outbound model calls: omit the API key from the knowledge base's model configuration and grant the search service's managed identity Cognitive Services User on the Azure OpenAI resource.

For NIS2-scoped organisations this is the migration's hidden benefit. Keyless, identity-based data access with per-user tokens is easier to defend in an audit than a shared key and a prompt instruction, and the retirement is a forcing function to get there.

Cost: what moves from one bill to two

On Your Data billed you for Azure AI Search (tier plus semantic ranker if enabled), embeddings and completion tokens. Agentic retrieval splits differently. Azure AI Search bills retrieval tokens consumed during subquery execution and semantic reranking, with the first 50 million tokens a month free on the default plan and pay-as-you-go on the standard plan after that. Azure OpenAI bills the input and output tokens for query planning and answer synthesis on whatever model you attach to the knowledge base. On the GA minimal path there is no planning model, so the Azure OpenAI line is only your own answer generation, exactly as before.

Microsoft's own worked example in the agentic retrieval overview is a useful order-of-magnitude check. It assumes 2,000 retrievals with three subqueries each, 50 chunks of 500 tokens reranked per subquery, and gpt-4o-mini for planning at 2,000 input and 350 output tokens per plan. The result is 150 million reranking tokens costing about $3.30 at a hypothetical rate, plus $0.60 for planning input and $0.42 for planning output, for $4.32 in total. The retrieval tokens dominate, and they scale with chunk size times reranked chunks times subqueries. That is the lever: an index re-chunked at 512 tokens halves the reranking bill compared to the 1,024-token default On Your Data ingestion used, at the cost of the retrieval granularity trade-offs the old documentation already described.

Reasoning effort sets the ceiling. Minimal allows up to 10 knowledge sources with no LLM at all. Low allows three sources and three subqueries with a 5,000-token synthesis budget. Medium allows five sources, five subqueries, iterative search and a 10,000-token budget. The default is low, which means a knowledge base created in the portal will call your planning model on every request unless you set it otherwise. The notebook's cost advice is unusually candid: "Measure quality, latency, and usage on your own queries rather than assuming lower cost or better answers than OYD."

Two billing mechanics are new. From 2026-04-01, agentic retrieval billing consent is a dedicated knowledgeRetrieval property on the search service, set through the management API and separate from the semanticSearch consent. And Foundry IQ Serverless, in preview since June with billing live since 13 September 2026, prices by compute unit per hour and gigabyte per month; Switzerland North is listed at $0.343 per compute unit hour and $0.286 per gigabyte month. For a low-traffic internal assistant that scales to zero overnight, serverless may undercut a Basic tier running around the clock, but only once it leaves preview.

The Swedish and EU angle

Sweden Central has every feature, with one asterisk. Microsoft's region table, updated 9 October 2026, shows Sweden Central supporting agentic retrieval, semantic ranker, serverless, availability zones and query rewrite, and offering agentic retrieval on the free tier. It also carries the "high demand" footnote, meaning new search service creation is currently blocked there. The same footnote sits on West Europe, North Europe and Germany West Central. If you need a new search service in the EU this month, the regions with a full feature set and no capacity block are Switzerland North, France Central, Poland Central and Italy North. Norway East supports agentic retrieval but has no semantic ranker in the table, which rules it out for a like-for-like migration. If your existing search service is already in Sweden Central, you keep it; the block applies to new services, not existing ones.

Your data flow diagram changes. On the preview path, the user's question and the conversation history leave Azure AI Search and go to the planning model on your Azure OpenAI deployment. Keep that deployment in an EU data zone or Sweden Central so the GDPR Article 30 record stays simple. On the GA path the knowledge base never calls a model, which is one fewer hop to document.

SharePoint remote sources carry a licence condition. Remote SharePoint knowledge sources go through the Copilot Retrieval API and require each end user to hold a Microsoft 365 Copilot licence. For a Swedish organisation that bought On Your Data precisely to avoid per-seat Copilot pricing, the indexed SharePoint source (ingested into the index with ACL sync) is the path that preserves that economics.

The AI Act does not care which retrieval engine you use. The transparency obligations that have applied since 2 August 2026 attach to the chatbot you expose, not to the plumbing behind it. A migration that changes the citation format is also a good moment to re-verify that users can still see which document an answer came from, since traceability is the practical evidence most deployers point to.

Migration checklist

• 1. Find every call. Grep for data_sources, dataSources and /extensions/chat/completions across code, infrastructure templates and API Management policies. Include Copilot Studio and Power Platform connectors that may have been configured by someone outside engineering.
• 2. Classify each source. Azure AI Search, Blob and file-upload indexes can be reused as a searchIndex knowledge source. Cosmos DB for MongoDB vCore and Elasticsearch are not migrated by the sample: either retrieve from them in your own code and generate the answer, or load a search index separately first.
• 3. Choose GA or preview per application. Production and regulated: 2026-04-01, intents, your own answer generation. Internal and low stakes: 2026-08-01-preview with synthesis, pinned, and scheduled for a revisit when it goes GA.
• 4. Map the parameters. Use the table above. Write a test for strictness replacement, for scoring-profile loss and for the filter-replaces-not-appends change.
• 5. Fix authorization before cutover. Per-user Search token on every retrieve, ACLs synchronised, managed identity for the model connection, no admin key in the query path.
• 6. Rewrite the citation parser and own the transcript. Handle [ref_id:<id>] references and validate that every cited ID exists in the returned references.
• 7. Evaluate on your corpus. Supported, unsupported and permission-denied questions, with latency and token usage recorded per query. Compare against a captured baseline from the old endpoint if you still have logs.
• 8. Set billing consent and a budget alert. Enable knowledgeRetrieval on the search service, set reasoning effort explicitly rather than accepting the default, and alert on Azure AI Search token consumption once the 50 million free tokens are consumed.
• 9. Update the model. Replace the GPT-4o deployment On Your Data forced on you with a current model, and check the model retirement schedule while you are there.

Conclusion

On Your Data was the easiest way to put a company's documents behind a chat model on Azure, and its retirement removes that convenience without a drop-in successor. The index survives. The glue does not. For most Swedish and EU teams the right move is the GA path: a knowledge base queried with intents, answers generated in your own code, per-user tokens on every call, and a chunk size chosen with the reranking bill in mind. Preview features can come later. The authorization tightening is the part worth doing even if nothing had been retired.