Microsoft: Azure AI, ONNX and Phi

Models and offerings #

Who they are

This hub is for teams whose infrastructure is already on Azure, for anyone who needs to run a model on Windows or inside a desktop app, and for people after a small, well-licensed model. Microsoft Research has worked on machine learning since the 1990s, co-created the ONNX format with Facebook in 2017, and has been OpenAI’s largest investor and exclusive cloud partner since 2019. That partnership gives Azure the GPT models under Azure terms, and since 2025 Microsoft has added Anthropic’s Claude and most open families to the same catalogue.

Its place in the field is the enterprise distribution channel and the Windows on-ramp rather than frontier research, although the Phi series from Microsoft Research has shown repeatedly that careful data makes small models punch above their size. Microsoft also maintains DeepSpeed, which made large-scale PyTorch training practical before FSDP existed.

Tooling and APIs

The cloud product is Microsoft Foundry at ai.azure.com, called Azure AI Studio until late 2024 and Azure AI Foundry until the rename took effect in January 2026. The portal URL and role definitions did not change. It fronts Azure OpenAI (GPT models with the OpenAI API shape and Azure authentication), Claude at Anthropic’s list prices billed through the Marketplace, and a catalogue of open models deployed as serverless pay-per-token endpoints or on managed GPU compute. Pricing is per model and per region and changes often, so see Azure’s pricing pages rather than any number here. SDKs: openai with an Azure endpoint, anthropic with the Foundry client class, and azure-ai-projects for agents.

The on-device product is Foundry Local, a runtime your application ships with. It runs on Windows, macOS on Apple silicon and Linux, uses ONNX Runtime underneath, picks the GPU or NPU execution provider automatically, adds about 20 MB to a package, needs no Azure subscription and exposes an OpenAI-compatible API including the Responses format. Its catalogue is curated and includes gpt-oss, Qwen, DeepSeek, Mistral, Phi and Whisper conversions. Microsoft’s own documentation says it is for single-user client apps and points server workloads to vLLM or Triton.

# needs-api
import os
from openai import OpenAI

client = OpenAI(api_key="not-needed", base_url=os.environ["FOUNDRY_LOCAL_ENDPOINT"])
chat = client.chat.completions.create(
    model=os.environ.get("FOUNDRY_LOCAL_MODEL", "phi-4-mini"),
    messages=[{"role": "user", "content": "In one sentence, what does a learning rate control?"}],
)
print(chat.choices[0].message.content)

ONNX Runtime (MIT) is the engine under Foundry Local, Windows ML and a great deal of production inference on CPUs. Version 1.32.0 shipped on 13 November 2026 and the execution providers for CUDA, WebGPU and CoreML now ship as separate plugin packages on their own release cadence. QNN covers Qualcomm NPUs in Copilot+ PCs and DirectML covers any Windows GPU. onnxruntime-genai adds the generation loop for LLMs.

Phi models are on Hugging Face under microsoft and all carry the MIT licence, which makes them the least encumbered small models from a large vendor. Phi-4 (14 B parameters, 16 k context, MIT, December 2024) is the dense text base. Phi-4-mini (3.8 B, 128 k context, MIT, February 2025) and Phi-4-multimodal are the edge sizes. Phi-4-reasoning-vision-15B (15 B, 16 k context, MIT, 4 March 2026) is the newest at the time of writing and takes image input. No Phi-5 has been released. The model cards report scores on MMMU, MathVista and similar benchmarks, and those are the vendor’s numbers.

Pain points: Azure OpenAI deployments are per region with per-deployment quotas and model versions that you must upgrade yourself, Foundry has been renamed three times in three years and documentation URLs rot, and ONNX export from PyTorch fails on dynamic control flow.

Migration notes

From OpenAI direct to Azure OpenAI: the same openai SDK with AzureOpenAI, an endpoint URL, an API version string and a deployment name in place of the model name. Features arrive on Azure weeks after openai.com and some never do. Going the other way is the reverse substitution.

To ONNX: torch.onnx.export or the Optimum library for Transformers models produces a graph that ONNX Runtime runs on any of its providers. Quantise with ONNX Runtime’s tools or olive. Models with custom operators or data-dependent shapes are the ones that fail, and the fix is usually to export a submodule. The frameworks reference compares ONNX with TensorRT, Core ML and GGUF.

Phi to and from Llama: Phi-4-mini uses a Llama-like architecture and the usual stacks (vLLM, llama.cpp, MLX, Ollama) support it. Chat templates differ, so use apply_chat_template.

Watch for

Lock-in on Azure is organisational rather than technical: the model APIs are OpenAI-compatible and the open models are portable, but quotas, private networking, content filters and Entra authentication are Azure-specific, and provisioned throughput is a commitment. Azure OpenAI applies content filters by default that can block legitimate requests, and the deprecation calendar for model versions is shorter than openai.com’s.

Foundry Local’s catalogue is curated, so a model you want may not be in it, and the runtime downloads models and execution providers on first use, which matters offline. Diagnostics are optional and user-initiated according to Microsoft. ONNX Runtime plugin providers have minimum core versions, so pin them together.

The Phi licences are clean, but the training data is synthetic and not published, so the models are open weights rather than open source, and their reasoning variants are tuned for benchmark-style problems. Microsoft’s enterprise terms for Copilot products differ from the Foundry API terms on data use. Read the one that applies.