# Fine-Tuning in Microsoft Foundry: Building Production-Ready AI Agents - Microsoft Ignite 2025

Microsoft Ignite BRK188: Fine-tuning in Microsoft Foundry transforms generic models into production-ready agents. Synthetic data generation, supervised + reinforcement fine-tuning, 40-90% cost reduction, 95%+ accuracy. Real-world results: 2M docs/day, $27M savings.

- Published: 2025-11-28 · Category: AI & Cloud Infrastructure · Tags: Microsoft Ignite 2025, Microsoft Foundry, Fine-Tuning, Supervised Fine-Tuning, Reinforcement Fine-Tuning, Agentic RFT, Synthetic Data Generation, Azure OpenAI, Tool Calling, Data Extraction, Workflow Execution, Model Optimization, Production AI, Agent Accuracy, Cost Optimization, GPT-4o
- Author: Technspire AB, Stockholm (https://technspire.com)
- Canonical: https://technspire.com/en/blog/ignite-2025-brk188-fine-tuning-microsoft-foundry-production-ready-agents

### Baseline Performance Assessment (1-2 weeks)

- • Identify use case requiring fine-tuning (tool calling, data extraction, workflow execution)
- • Measure baseline with best-effort prompt engineering (accuracy, latency, cost)
- • Define success criteria (target accuracy, latency, cost reduction)
- • Estimate ROI (cost of fine-tuning vs. expected savings/value)
- • Validate data availability (need 1,000+ high-quality examples)

 2

### Training Data Preparation (3-4 weeks)

- • Collect real examples (historical data with known-good outputs)
- • Annotate data with expert labels (correct tool calls, extracted fields, classifications)
- • Use synthetic data generation to expand dataset (10× multiplier)
- • Split data: 80% training, 10% validation, 10% test
- • Format as JSONL (input-output pairs)
- • Quality assurance: review samples, ensure consistency

 3

### Model Selection and Training (2-3 weeks)

- • Choose base model (GPT-4o for accuracy, GPT-4o-mini for cost, Llama-3 for control)
- • Run fine-tuning in Foundry (developer tier for experimentation)
- • Hyperparameter tuning (learning rate, epochs, batch size)
- • Monitor training metrics (loss curves, validation accuracy)
- • Test multiple model versions (compare accuracy vs. cost trade-offs)
- • Select best performer for production

 4

### Validation and Testing (2-3 weeks)

- • Test on held-out test set (measure accuracy, latency, cost)
- • Compare to baseline (is fine-tuned model significantly better?)
- • Edge case testing (adversarial inputs, unusual formats, error conditions)
- • User acceptance testing (domain experts validate quality)
- • Performance benchmarking (throughput, concurrency, scaling behavior)
- • Document evaluation results and model limitations

 5

### Production Deployment (2-3 weeks)

- • Deploy fine-tuned model to Foundry inference endpoint
- • Canary rollout (5% → 25% → 100% of traffic)
- • Monitor production metrics (accuracy, latency, error rates)
- • Set up alerting for degradation (accuracy drops, latency spikes)
- • Implement fallback to baseline model if issues detected
- • Track business metrics (cost savings, throughput, user satisfaction)

 6

### Continuous Improvement (Ongoing)

- • Collect production data (new examples with errors to learn from)
- • Periodic retraining (monthly or quarterly with updated data)
- • A/B testing (compare new model versions vs. current production)
- • Explore reinforcement fine-tuning (if complex reasoning needed)
- • Model distillation (once large model proven, distill to smaller for cost)
- • Measure ROI continuously (track savings vs. training investment)

## Why This Matters for Swedish Organizations

Sweden's organizations face unique drivers for fine-tuning adoption:

- **Language requirements**: Swedish language AI needs fine-tuning on Swedish text. Generic models trained primarily on English underperform on Swedish documents, terminology, cultural context.
- **Regulatory compliance**: GDPR, NIS2, AI Act—fine-tuned models can be deployed on-premises or in EU data centers with full data control. Generic API models send data to US clouds.
- **Industry specialization**: Swedish strengths (manufacturing, healthcare, fintech, cleantech) require domain-specific agents. Fine-tuning teaches models Swedish industry terminology and workflows.
- **Cost efficiency**: Smaller Swedish organizations can't afford $33M/year AI bills. Fine-tuning enables 80-90% cost reduction through smaller, faster models.
- **Competitive advantage**: Agents that understand your specific business processes execute faster and more accurately than competitors using generic models.
- **Data sovereignty**: Training data stays in Sweden. Fine-tuned models deployed in Swedish Azure regions. No data leaves EU.

### Key Takeaways from BRK188

- ✓ **Fine-tuning transforms generic models into production-ready agents** with 95%+ accuracy and 80-90% cost reduction
- ✓ **Microsoft Foundry provides end-to-end platform**: synthetic data generation, supervised + reinforcement fine-tuning, automated deployment
- ✓ **Agentic Reinforcement Fine-Tuning (RFT)** teaches models optimal tool usage and reasoning strategies (+9-19 point accuracy gains)
- ✓ **Use case validation critical**: Fine-tune when accuracy 5%, or high cost/latency
- ✓ **Real-world results**: Customer document management scaled to 2M docs/day, $27M annual savings, 98.7% accuracy
- ✓ **Developer training tier** enables low-cost experimentation before production deployment
- ✓ **Open-source models supported**: Llama, Mistral, Phi alongside Azure OpenAI for cost/control flexibility
- ✓ **Organizations report 40-90% cost reduction and 3-7× throughput improvement** with fine-tuned agents

Fine-tuning isn't optional for production agents—it's the difference between a demo that impresses and a system that delivers value. Microsoft Foundry makes fine-tuning accessible: synthetic data generation solves the training data challenge, reinforcement fine-tuning enables optimal reasoning, and automated deployment gets models to production fast. For Swedish organizations building agents that must handle Swedish language, comply with EU regulations, and operate cost-effectively at scale, fine-tuning in Foundry is the path from prototype to production.

---

Technspire AB builds AI agents, Azure OpenAI solutions, and production web platforms for Swedish and EU enterprises. Book a call: https://calendly.com/technspire · hello@technspire.com · More articles: https://technspire.com/en/blog · Site overview for agents: https://technspire.com/llms.txt