Building Knowledge-Powered Agents with Azure AI Search: RAG, Hybrid Search, and Agentic Retrieval - Microsoft Ignite 2025

RAG Fundamentals: Grounding Agents in Organizational Knowledge

Retrieval-Augmented Generation (RAG) solves the fundamental limitation of language models: they only know what they learned during training. RAG combines retrieval (finding relevant documents from knowledge bases) with generation (synthesizing answers using retrieved context), enabling agents to answer questions about proprietary data, current information, and domain-specific content while providing citations for verification.

System: You are an assistant that answers questions using
only the provided documents. Always cite sources.

Context Documents:
[1] Board Meeting Minutes - 2024-03-15: "...approved Azure
migration with 18-month timeline..."
[2] Cloud Strategy Memo: "...AWS initially considered but
Azure selected for Microsoft 365 integration..."

User Question: What did we decide about cloud migration?

Answer with citations:

RAG vs. Fine-Tuning: Complementary Approaches

Azure AI Search: The Foundation for Agentic Retrieval

Azure AI Search (formerly Azure Cognitive Search) provides enterprise-grade search infrastructure optimized for RAG applications. With support for hybrid search, semantic ranking, vector embeddings, and integrated skillsets for document processing, it serves as the knowledge retrieval engine for Azure OpenAI, Microsoft Copilot, and custom agentic applications.

Hybrid Search Architecture: Keyword + Vector + Semantic

Azure AI Search's hybrid approach combines three complementary search methods, each capturing different aspects of relevance. The fusion of these methods delivers superior results compared to any single technique.

Reciprocal Rank Fusion (RRF): Combining Keyword and Vector Scores

Keyword and vector search produce different relevance scores (BM25 score vs. cosine similarity) that can't be directly compared. Reciprocal Rank Fusion elegantly combines rankings without needing score normalization.

RRF_Score(doc) = Σ [ 1 / (k + rank_i(doc)) ]

where:
- k = constant (typically 60)
- rank_i(doc) = rank of document in search method i
- Σ sums across all search methods (keyword, vector)

Knowledge Sources: Connecting Agents to Data

BRK193 demonstrated connecting agents to three critical enterprise knowledge sources: SharePoint Online, web content, and Azure Blob Storage. Each source requires different indexing strategies, permission handling, and update cadences.

{
  "name": "sharepoint-datasource",
  "type": "sharepoint",
  "credentials": {
    "connectionString": "SharePointOnlineEndpoint=https://contoso.sharepoint.com;..."
  },
  "container": {
    "name": "defaultSiteLibrary",
    "query": "/sites/Engineering"
  },
  "dataChangeDetectionPolicy": {
    "@odata.type": "#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy",
    "highWaterMarkColumnName": "_metadata_storage_last_modified"
  }
}

{
  "name": "blob-indexer",
  "dataSourceName": "blob-datasource",
  "targetIndexName": "documents-index",
  "skillsetName": "document-processing-skillset",
  "parameters": {
    "batchSize": 50,
    "configuration": {
      "dataToExtract": "contentAndMetadata",
      "imageAction": "generateNormalizedImages",
      "parsingMode": "default"
    }
  },
  "fieldMappings": [
    { "sourceFieldName": "metadata_storage_path", "targetFieldName": "id" },
    { "sourceFieldName": "metadata_storage_name", "targetFieldName": "fileName" }
  ],
  "outputFieldMappings": [
    { "sourceFieldName": "/document/merged_content", "targetFieldName": "content" },
    { "sourceFieldName": "/document/organizations", "targetFieldName": "organizations" }
  ]
}

Agentic Retrieval: Query Planning and Knowledge Source Selection

Unlike traditional search where users manually select sources and refine queries, agentic retrieval systems autonomously plan multi-step search strategies. Agents analyze user intent, determine which knowledge sources to query, generate optimized queries for each source, and merge results intelligently.

Indexed vs. Remote Knowledge Sources

Agentic retrieval systems combine two types of knowledge sources, each with distinct characteristics and use cases.

Reasoning Effort Modes: Balancing Cost, Latency, and Accuracy

BRK193 revealed three reasoning effort modes that control how aggressively agents search for information and refine results. These modes provide developers with a dial to tune the cost-accuracy tradeoff based on application requirements.

Foundry IQ & MCP Protocol: Unified Knowledge Integration

Foundry IQ introduces a unified abstraction layer for connecting agents to diverse knowledge sources using the Model Context Protocol (MCP). Rather than building custom integrations for each data source, developers define MCP-compatible connectors that agents can discover and use dynamically.

{
  "name": "sharepoint-engineering",
  "type": "mcp-connector",
  "capabilities": {
    "search": {
      "methods": ["keyword", "vector", "hybrid"],
      "filters": ["path", "fileType", "modifiedDate"],
      "maxResults": 50
    },
    "retrieve": {
      "supportedFormats": ["text", "markdown", "raw"]
    }
  },
  "authentication": {
    "type": "oauth2",
    "scopes": ["Sites.Read.All"]
  },
  "metadata": {
    "description": "Engineering documentation from SharePoint",
    "dataClassification": "internal",
    "updateFrequency": "15min"
  }
}

Foundry IQ Workflow: Agent-Driven Knowledge Selection

Implementation Guide: Building Your First Knowledge Agent

This step-by-step guide walks through building a production-ready knowledge agent using Azure AI Search, Azure OpenAI, and the patterns demonstrated in BRK193.

# Create search service (Standard tier for hybrid search)
az search service create \
  --name my-knowledge-search \
  --resource-group knowledge-rg \
  --sku Standard \
  --location westeurope

# Create index with hybrid search fields
# (Use Azure Portal or REST API for detailed schema)
# Key fields: id, title, content, contentVector (1536 or 3072 dims),
# metadata (author, date, department), url

# Create SharePoint data source
POST https://[servicename].search.windows.net/datasources?api-version=2024-07-01
{
  "name": "sharepoint-docs",
  "type": "sharepoint",
  "credentials": {
    "connectionString": "SharePointOnlineEndpoint=https://contoso.sharepoint.com;ApplicationId=...;TenantId=..."
  },
  "container": { "name": "defaultSiteLibrary", "query": "/sites/KnowledgeBase" }
}

# Create skillset with text extraction + embedding generation
# Skillset includes: DocumentExtraction skill, SplitText skill (chunking),
# AzureOpenAIEmbedding skill (generate contentVector)

# Create indexer to run skillset and populate index
# Schedule: every 15 minutes for near-real-time updates

# Create Azure OpenAI resource
az cognitiveservices account create \
  --name my-openai \
  --resource-group knowledge-rg \
  --kind OpenAI \
  --sku S0 \
  --location westeurope

# Deploy models via Azure OpenAI Studio
# 1. text-embedding-3-large (for query and document embeddings)
# 2. gpt-4o (for answer generation)

from azure.search.documents import SearchClient
from openai import AzureOpenAI

# Initialize clients
search_client = SearchClient(endpoint, index_name, credential)
openai_client = AzureOpenAI(azure_endpoint, api_key)

def answer_question(query: str) -> dict:
    # 1. Generate query embedding
    query_vector = openai_client.embeddings.create(
        model="text-embedding-3-large",
        input=query
    ).data[0].embedding

    # 2. Hybrid search with RRF
    results = search_client.search(
        search_text=query,  # keyword search
        vector_queries=[{
            "vector": query_vector,
            "k_nearest_neighbors": 50,
            "fields": "contentVector"
        }],
        query_type="semantic",  # enable semantic ranking
        top=5
    )

    # 3. Build context from top results
    context = "\n\n".join([f"[{i+1}] {r['title']}: {r['content']}"
                             for i, r in enumerate(results)])

    # 4. Generate answer with citations
    completion = openai_client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "Answer using provided documents. Always cite sources."},
            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {query}"}
        ]
    )

    return {
        "answer": completion.choices[0].message.content,
        "sources": [{"title": r['title'], "url": r['url']} for r in results]
    }

Conclusion: The Knowledge-Powered Agent Future

Microsoft Ignite 2025 BRK193 demonstrated that the next generation of AI agents will be defined not by model size or parameter counts, but by their ability to access, reason over, and synthesize knowledge from diverse sources. Azure AI Search's hybrid search, semantic ranking, and agentic retrieval capabilities provide the foundation for building agents that deliver accurate, cited, and trustworthy answers grounded in organizational knowledge.