Phi (Microsoft)

Phi is a family of small language, reasoning, and multimodal artificial-intelligence models developed by Microsoft Research. The family began with the 1.3-billion-parameter Phi-1 coding model in June 2023 and subsequently expanded to include general-purpose text models, long-context models, sparse Mixture-of-Experts systems, text-image-audio models, mathematical reasoning models, hybrid state-space architectures, and models for visual and graphical-user-interface grounding.

The Phi project investigates whether careful data curation, synthetic-data generation, training curricula, and post-training can enable comparatively compact models to perform competitively with much larger systems on selected tasks. This does not eliminate the importance of model scale, data volume, or inference-time computation; rather, Phi research provides evidence that data quality and training methodology can materially alter the relationship between parameter count and benchmark performance.

Microsoft describes the current Phi releases as open-source models distributed under the MIT License. The official Phi-4 checkpoints discussed in this article use that license and permit academic and commercial use under its terms. However, Microsoft does not publish the complete document-level training corpora or a fully reproducible end-to-end pipeline for creating the frontier checkpoints. Consequently, open-weight is the more precise general description under definitions that require training data and training code in addition to model parameters.

As of July 11, 2026, the newest broadly released general Phi model is Phi-4-reasoning-vision-15B, published on March 4, 2026. It accepts text and images, produces text, and combines the Phi-4-reasoning language backbone with a SigLIP-2 visual encoder. Microsoft also released the specialized Phi-Ground-Any-4B graphical-interface grounding model in May 2026. Microsoft had not announced a Phi-5 generation by the stated date.

Philosophy: “Textbooks Are All You Need”

The original Phi research was based on the hypothesis that models can learn more effectively from data that is coherent, explanatory, diverse, and appropriate to their level of capability than from an equally sized sample of weakly filtered web text. The phrase “Textbooks Are All You Need” refers to the title of the first Phi research paper and should not be interpreted literally as meaning that the models were trained only on conventional textbooks.

The principal elements of the approach include:

• “Textbook-quality” data: Microsoft filtered programming code, web pages, question-and-answer material, academic content, and other sources for educational value, clarity, reasoning depth, and correctness. The resulting material was used either directly for training or as source material for synthetic-data generation.
• Synthetic data: Phi-1 was trained on approximately 6 billion tokens of filtered code and related web material and 1 billion tokens of synthetic Python textbooks and exercises generated with GPT-3.5. It was trained for four days on eight Nvidia A100 GPUs. The Phi-4 report describes 50 broad synthetic-dataset types containing approximately 400 billion unique, unweighted source-pool tokens. This is not the model's total training-token count: repetitions and the complete mixture of synthetic, rewritten, web, code, book, academic, and question-and-answer data resulted in approximately 9.8 trillion training tokens.
• Curriculum and iterative generation: Synthetic examples are created through methods including seed curation, multi-stage prompting, self-revision, instruction reversal, answer checking, code execution, rejection sampling, and the transformation of source passages into exercises or conversations.
• Post-training: Later instruction checkpoints use combinations of supervised fine-tuning, preference optimization, reinforcement learning, safety data, and red-team feedback. The exact process differs by model and should not be assumed to be identical across the entire family.

The Phi-4 ablation experiments found that synthetic-only training improved several reasoning and coding evaluations but weakened knowledge-heavy performance and increased hallucinations. Microsoft therefore retained curated organic web data, code, acquired books, academic sources, and question-and-answer material. The project supports a data-quality hypothesis, not the claim that synthetic data or small parameter counts eliminate factual limitations or cause a model to reason in the same manner as a human.

Evolution of the Phi Models

• Phi-1 (1.3 billion parameters): Introduced in June 2023 as a model specialized for basic Python programming. It was trained on approximately 7 billion tokens and achieved developer-reported pass@1 scores of 50.6% on HumanEval and 55.5% on MBPP. These results demonstrated that carefully filtered and generated training data could yield strong coding performance at a relatively small parameter count.

• Phi-1.5 (1.3 billion parameters): Released in September 2023. It retained the Phi-1 data sources and added synthetic material covering common-sense reasoning, everyday knowledge, natural-language tasks, and general text generation. Phi-1.5 was a base language model rather than a safety-aligned conversational assistant; the technical report warned that it could generate inaccurate, biased, toxic, or otherwise undesirable content.

• Phi-2 (2.7 billion parameters): Released in December 2023. Phi-2 expanded the synthetic and filtered training mixture for language understanding, mathematics, coding, and common-sense reasoning. The original release did not undergo reinforcement learning from human feedback or instruction-specific safety alignment. Its current official repository uses the MIT License.

• Phi-3 (3.8–14 billion parameters): Introduced in April 2024. The principal text-model sizes were Phi-3-mini (3.8B), Phi-3-small (7B), and Phi-3-medium (14B). Mini was trained on approximately 3.3 trillion tokens; Small and Medium were trained on approximately 4.8 trillion. Microsoft released context variants supporting 4K or 128K tokens for Mini, 8K or 128K for Small, and 4K or 128K for Medium. The family also included the 4.2B Phi-3-Vision text-and-image model. Microsoft reported that a four-bit Phi-3-mini package occupied approximately 1.8 GB and generated more than 12 tokens per second in an offline demonstration on an iPhone 14. This was a specific optimized demonstration rather than a guarantee for every device, runtime, prompt length, or Phi model.

• Phi-3.5: Announced in August 2024 and composed of three principal checkpoints:
  • Phi-3.5-mini-instruct — a 3.8B dense multilingual text model with a 128K context window.
  • Phi-3.5-MoE-instruct — a sparse decoder-only MoE model with approximately 42 billion total parameters, 6.6 billion active parameters, 16 experts, and two selected experts per token. It was trained on approximately 4.9 trillion tokens and supports a 128K context.
  • Phi-3.5-Vision-instruct — a 4.2B text-and-image model supporting single and multiple images and a 128K context. It produces text rather than images.

• Phi-4 (14 billion parameters): Announced in December 2024, with MIT-licensed weights made publicly available through Microsoft and Hugging Face. It is a dense decoder-only text model with a 16K context window, not 128K. Microsoft reports that it was trained on 9.8 trillion tokens using 1,920 H100-80GB GPUs for 21 days. Its training mixture combined synthetic data, filtered public-domain web material, web rewrites, code, academic sources, licensed books, and question-and-answer datasets.

• Phi-4-mini-instruct (3.8B) and Phi-4-multimodal-instruct (5.6B): Announced in February 2025 and published in early March under the MIT License.
  • Phi-4-mini-instruct is a dense text model with Grouped-Query Attention, shared input-output embeddings, a vocabulary of approximately 200,000 tokens, a 128K context, and documented tool-enabled function-calling syntax. Microsoft reports approximately 5 trillion training tokens and text support for 23 languages.
  • Phi-4-multimodal-instruct uses Phi-4-mini as its language backbone and adds vision and speech encoders and modality adapters. It accepts text, images, and audio and generates text, with a 128K context. Microsoft reports training on approximately 5 trillion text tokens, 1.1 trillion image-text tokens, and 2.3 million hours of speech. The model uses modality-specific Low-Rank Adaptation components and routing described as a Mixture-of-LoRAs; it does not natively generate images or audio.

• Phi-4-reasoning and Phi-4-reasoning-plus (14B): Released on April 30, 2025 under the MIT License. Both retain Phi-4's dense architecture but extend the context to 32K and generate a reasoning section followed by a summarized answer. Phi-4-reasoning was supervised-fine-tuned on curated “teachable” prompts and reasoning demonstrations generated primarily with OpenAI's o3-mini. Phi-4-reasoning-plus adds a short outcome-based reinforcement-learning stage. Microsoft reports that the Plus model provides higher mathematical accuracy but generates approximately 50% more output tokens on average, increasing latency.

• Phi-4-mini-reasoning (3.8B): Released in April 2025 as a mathematics-focused reasoning model with a 128K context. Its training recipe consists of large-scale mid-training on distilled long reasoning traces, supervised fine-tuning, rollout DPO, and reinforcement learning with verifiable rewards. The model card states that the mathematical synthetic corpus was generated using DeepSeek-R1. Microsoft explicitly describes the checkpoint as designed and tested for mathematical reasoning rather than as a universal general-purpose assistant.

• Phi-4-mini-flash-reasoning (3.8B): Announced in May and released in June 2025 under the MIT License. It uses the SambaY decoder-hybrid-decoder architecture rather than a conventional all-attention transformer. SambaY combines state-space layers, Differential Attention, Grouped-Query Attention, a Gated Memory Unit, shared key-value memory, and one global-attention layer. The model supports a 64K context. Microsoft reported up to ten-times-higher decoding throughput than Phi-4-mini-reasoning for a specific vLLM test with a 2K prompt and 32K generated continuation; the ratio is not universal across hardware or workloads.

• Phi-4-reasoning-vision-15B: Released on March 4, 2026 under the MIT License. It combines the Phi-4-reasoning backbone with a SigLIP-2 vision encoder through a mid-fusion architecture. The model has approximately 15 billion parameters, accepts text and images, produces text, and supports 16,384 input tokens. Its dynamic-resolution encoder can represent an image using up to 3,600 visual tokens. It can be prompted to use extended reasoning for visual mathematics, science, document, and chart tasks or direct processing for captioning, detection, and grounding.

• Phi-Ground-Any-4B: Published in May 2026 as a specialized computer-use and GUI-grounding model. It was fine-tuned from Phi-3.5-Vision and uses a fixed input resolution of 1,680×1,008 pixels. It maps natural-language instructions and screenshots to locations or actions within graphical interfaces. Phi-Ground-Any is a task-specific derivative rather than a successor generation to Phi-4.

Architecture and Technical Features

• Architecture: Most Phi text models use dense autoregressive decoder-only transformer architectures. Important exceptions include Phi-3.5-MoE, which uses sparse expert routing; Phi-4-multimodal, which adds vision and speech encoders and adapters; Phi-4-mini-flash-reasoning, which combines state-space and attention layers; and Phi-4-reasoning-vision, which connects a visual encoder to the language backbone through mid-fusion.
• Attention mechanisms: Attention differs by checkpoint. Early compact Phi models primarily used conventional multi-head attention. Phi-3-small uses Grouped-Query Attention and alternates dense and block-sparse attention. Phi-4-mini and its derivatives use Grouped-Query Attention. GQA reduces the number of key-value heads relative to query heads, lowering KV-cache memory and memory bandwidth during generation.
• FlashAttention: Several supported runtimes use FlashAttention kernels to reduce memory movement during exact attention computation. FlashAttention is an implementation technique rather than a learned feature unique to the Phi architecture, and it should not be described as mandatory for every Phi checkpoint.
• Mixture-of-Experts: Phi-3.5-MoE contains 16 feed-forward experts and routes each token to two of them. Its 6.6-billion active-parameter count describes arithmetic for an individual token, not storage requirements: the complete approximately 42-billion-parameter checkpoint must generally remain in memory or be distributed across devices.
• Multimodal fusion: Phi-4-multimodal uses a shared language backbone with modality-specific vision and speech components. Phi-4-reasoning-vision instead uses mid-fusion, projecting visual tokens into the language-model embedding space and injecting them into selected layers.
• Hybrid state-space processing: Phi-4-mini-flash-reasoning uses state-space components for linear-complexity sequence processing and an attention component for global information exchange. Its Gated Memory Unit shares representations between layers, reducing redundant memory operations during long generation.
• Reasoning output: Phi-4-reasoning models can emit text enclosed in <think>...</think> or comparable reasoning sections before a final response. Such generated reasoning is useful as an intermediate working trace but is not guaranteed to be a complete or faithful explanation of the model's internal computation.
• Context length: Context limits are checkpoint-specific:
  • Phi-3 Mini: 4K or 128K;
  • Phi-3 Small: 8K or 128K;
  • Phi-3 Medium: 4K or 128K;
  • Phi-3.5 Mini, MoE, and Vision: 128K;
  • base Phi-4: 16K;
  • Phi-4-mini and Phi-4-multimodal: 128K;
  • Phi-4-reasoning and Phi-4-reasoning-plus: 32K;
  • Phi-4-mini-reasoning: 128K;
  • Phi-4-mini-flash-reasoning: 64K;
  • Phi-4-reasoning-vision: 16,384 tokens.

A maximum context length states how many tokens a compatible implementation can accept. It does not guarantee perfect retrieval, reasoning, or attention to every supplied fact throughout a sequence of that length.

• On-device deployment: Phi's small and quantized checkpoints are intended for memory- and latency-constrained systems. Suitable models can run on phones, personal computers, embedded systems, or edge servers, but not every Phi checkpoint is practical on ordinary mobile hardware. The 14B and 15B models, high-resolution visual processing, and long-context workloads can still require high-memory GPUs, multiple devices, or aggressive quantization.
• Framework support: Depending on the checkpoint, Phi is available through Microsoft Foundry, Hugging Face, GitHub Models, Ollama, llama.cpp, vLLM, SGLang, ONNX Runtime, Foundry Local, and Nvidia NIM. Availability, quantizations, supported modalities, and optimized kernels differ between models and platforms.

Performance and Benchmarks

The following table reproduces selected developer-reported results from the Phi-3 and Phi-4 technical reports. The checkpoints, evaluation harnesses, prompts, sampling settings, and release dates differ, so the values should not be treated as a single independently audited leaderboard.

Selected Phi benchmark results reported by Microsoft

Model	Parameters	MMLU	MT-Bench	HumanEval
Phi-3-mini	3.8B	68.8%	8.38	58.5%
Phi-3-small	7B	75.3%	8.70	59.1%
Phi-3-medium	14B	78.2%	8.91	55.5%
Phi-4	14B	84.8%	Not reported in the same evaluation	82.6%

In the Phi-4 report's OpenAI SimpleEval-based comparison, Phi-4 scored 56.1% on GPQA and 80.4% on MATH, compared with 50.6% and 74.6% for the tested GPT-4o endpoint. GPT-4o remained ahead on MMLU, HumanEval, factual SimpleQA, and several other evaluations. The result supports a benchmark-specific statement that Phi-4 exceeded that GPT-4o endpoint on GPQA and MATH; it does not establish that Phi-4 was generally more capable than GPT-4o.

Microsoft reported the following reasoning results under the protocols in the Phi-4-reasoning technical report:

• Phi-4-reasoning: 75.3% on AIME 2024 and 62.9% on AIME 2025;
• Phi-4-reasoning-plus: 81.3% on AIME 2024 and 78.0% on AIME 2025;
• DeepSeek-R1-Distill-Llama-70B: 69.3% and 51.5%;
• DeepSeek-R1: 78.7% and 70.4%.

The same report shows that larger proprietary systems remained stronger on several evaluations: for example, the tested o3-mini and Gemini 2.5 Pro configurations exceeded Phi-4-reasoning-plus on AIME 2024. Results also vary with the number of generated samples, averaging method, reasoning length, and token budget.

Microsoft's claim of up to ten-times-higher throughput for Phi-4-mini-flash-reasoning applies to a particular long-generation setup using vLLM and specified accelerator hardware. Real-world throughput and latency depend on prompt length, generation length, batch size, numerical precision, quantization, compiler, memory bandwidth, and hardware.

Specialized Applications

Phi checkpoints and third-party derivatives have been applied to several specialized research areas:

• On-device and offline assistants: Quantized Phi-3 and Phi-4-mini checkpoints can support local summarization, classification, drafting, structured extraction, mathematical assistance, and limited question answering without sending every prompt to an external API. Local execution can improve data control and availability but does not automatically guarantee privacy, security, or factual correctness.
• Medical and clinical language processing: Microsoft's MediPhi collection contains seven 3.8B models derived from Phi-3.5-mini-instruct and specialized for biomedical literature, medical guidelines, clinical documents, medical coding, and related tasks. They are research and development models rather than clinically approved diagnostic systems. Medical output requires professional review, application-specific testing, and regulatory assessment.
• Visual mathematics and document analysis: Phi-4-reasoning-vision is intended for reasoning over diagrams, handwritten equations, charts, tables, scientific material, and documents. It can still misread small text, complex page layouts, visual relationships, or ambiguous diagrams.
• Computer-use agents: Phi-4-reasoning-vision and Phi-Ground models support screenshot understanding, graphical-interface grounding, and mapping instructions to interface locations. A grounding prediction is not equivalent to safe autonomous control; production agents require permission boundaries, confirmation mechanisms, monitoring, and resistance to prompt injection.
• Hate-speech research: HateTinyLLM, a research model adapted from Phi-2 using LoRA, was evaluated for hate-speech classification. Its results apply to the authors' datasets and evaluation procedure and do not establish equivalent performance across languages, communities, platforms, or evolving forms of coded abuse.
• Game-strategy research: SC-Phi2 adapted Phi-2 to predict strategic actions in StarCraft II. It is an example of domain fine-tuning rather than a capability included in the unmodified base model.

Responsible AI and Safety

Microsoft documents checkpoint-specific safety processes rather than one uniform procedure for the complete Phi family.

For Phi-3, the company described a “break-fix” cycle consisting of repeated rounds of safety-data curation, post-training, benchmarking, red teaming, vulnerability identification, and corrective training. The evaluations covered single-turn and multi-turn interactions and several harm categories. Phi-3.5 Mini and MoE received additional multilingual testing.

Phi-4 used supervised fine-tuning and Direct Preference Optimization for instruction following and safety alignment. Phi-4-mini used SFT and DPO, while Phi-4-multimodal additionally documents RLHF. Phi-4-reasoning-vision was safety-post-trained with helpfulness and harmlessness examples and evaluated for harmful content, jailbreaks, groundedness, fabricated information, and reproduction of protected content.

These processes reduce measured risks but do not prove that a model is safe under every prompt, language, fine-tune, quantization, or deployment. Downloadable weights can be modified, alignment can be weakened through fine-tuning, and external tools can introduce risks that were not present in standalone model testing. Applications should use layered safeguards such as access control, input and output filtering, retrieval-source validation, tool permissions, logging, human review, and application-specific red teaming.

The earliest Phi base models were not designed as aligned production assistants. Phi-1.5 and Phi-2 can produce toxic, biased, fabricated, or unsafe output and should not inherit safety claims made for later instruction-tuned checkpoints.

Limitations

• Factual knowledge and hallucinations: Small parameter counts can limit the amount and diversity of knowledge represented by a model. Phi checkpoints can fabricate citations, entities, events, code behaviour, calculations, or explanations. Retrieval-augmented generation may improve access to current information but does not guarantee that retrieved material will be selected or interpreted correctly.
• Static training data: Phi models are trained on offline datasets with checkpoint-specific cut-off dates. They do not know later events unless current information is supplied through retrieval, tools, fine-tuning, or the prompt.
• Language differences: Language support varies substantially. Base Phi-4 is primarily optimized for English; Phi-4-mini documents 23 text languages; Phi-4-multimodal has different language sets for text and speech and primarily English vision training; Phi-4-mini-reasoning and Phi-4-mini-flash-reasoning are documented as English-focused. A family-level statement that all Phi models are multilingual is therefore inaccurate.
• Reasoning specialization: Phi-4-reasoning, reasoning-plus, and mini-reasoning emphasize mathematics, science, coding, and formal multi-step tasks. Their longer answers increase inference cost and do not necessarily improve ordinary conversation, factual recall, creative writing, or every professional task.
• Generated reasoning traces: Text inside reasoning blocks may be incomplete, post-hoc, inconsistent with the final answer, or factually wrong. It should not be treated as a transparent record of internal neural computation.
• Function calling: Structured tool use is documented for checkpoints such as Phi-4-mini, but it is not an automatic capability of every Phi model. Applications must validate generated arguments, enforce schemas, authorize tools, and restrict side effects.
• Multimodal limitations: Phi-4-multimodal and Phi-4-reasoning-vision produce text only. They can misrecognize speech, objects, diagrams, tables, handwriting, interface controls, spatial relationships, and visually embedded instructions.
• Long-context limitations: A 64K or 128K context does not guarantee perfect memory over the complete input. Accuracy can depend on the position, density, formatting, and relevance of information and may decline near the maximum supported length.
• Device requirements: “Small language model” is a relative term. A quantized 3.8B model may run on selected mobile devices, but a 14B or 15B checkpoint, a 128K KV cache, or high-resolution visual input can require substantial memory and accelerator capacity.
• Benchmark dependence: Most headline results were produced by Microsoft using specified internal or public evaluation pipelines. Prompt formatting, decontamination, sampling, inference-time compute, scoring, and model versions can materially change results. Independent production testing is required.
• Training-data transparency: Microsoft provides aggregate data descriptions and some data cards but not the complete source corpus or fully reproducible training pipeline. This limits independent auditing for copyright status, personal information, demographic representation, duplication, and benchmark contamination.
• High-risk use: Phi models are not, by default, validated autonomous systems for medicine, law, finance, employment, education assessment, public benefits, critical infrastructure, or physical control. Such applications require domain experts, formal evaluation, human oversight, security controls, monitoring, and compliance with applicable law.

External links

• Official Microsoft Phi page
• Microsoft models on Hugging Face
• Phi Cookbook on GitHub
• Microsoft Foundry model catalogue
• Phi (language model) — Wikipedia

Literature

• Gunasekar, S.; et al. (2023). Textbooks Are All You Need. arXiv:2306.11644.
• Li, Y.; et al. (2023). Textbooks Are All You Need II: phi-1.5 Technical Report. arXiv:2309.05463.
• Abdin, M.; et al. (2024). Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arXiv:2404.14219.
• Haider, E.; et al. (2024). Phi-3 Safety Post-Training: Aligning Language Models with a “Break-Fix” Cycle. arXiv:2407.13833.
• Abdin, M.; et al. (2024). Phi-4 Technical Report. arXiv:2412.08905.
• Abdin, M.; et al. (2025). Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs. arXiv:2503.01743.
• Microsoft Research (2025). Phi-4-reasoning Technical Report. PDF.
• Xu, H.; et al. (2025). Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Math. arXiv:2504.21233.
• Ren, L.; et al. (2025). Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation. arXiv:2507.06607.
• Aneja, J.; et al. (2026). Phi-4-Reasoning-Vision: A Multimodal Reasoning Model. arXiv:2603.03975.
• Microsoft Research (2026). Covering Human Action Space for Computer Use: Data Synthesis and Benchmark. arXiv:2605.12501.
• Ainslie, J.; et al. (2023). GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. arXiv:2305.13245.
• Dao, T.; et al. (2022). FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. arXiv:2205.14135.