Gemma (Google)

Gemma is a family of open-weight artificial-intelligence foundation models developed by Google DeepMind and other teams at Google. The family includes general-purpose text and multimodal models as well as models specialized for programming, translation, embeddings, function calling, safety classification, medicine, therapeutic research, privacy-preserving training, document grounding, and other tasks.

Google describes Gemma as drawing on the research and technology used to develop the Gemini family. This means that the families share research lineage and some technical methods; Gemma models are not simply reduced copies of particular Gemini checkpoints. The name comes from the Latin word gemma, meaning "precious stone".

Licensing differs between generations. Gemma 1, Gemma 2, Gemma 3, and many related models are governed by Google's custom Gemma Terms of Use, which allow research and commercial use subject to redistribution requirements and a prohibited-use policy. Gemma 4 and DiffusionGemma were released under the Apache License 2.0. Google publishes model weights and implementation resources but does not publish the complete document-level training corpora or a fully reproducible end-to-end training pipeline for its frontier models. Consequently, open-weight is the more precise general description; the exact license attached to each checkpoint must be checked separately.

As of July 11, 2026, the latest mainline generation is Gemma 4. Google released the E2B, E4B, 26B-A4B, and 31B checkpoints on March 31, 2026, followed by Gemma 4 12B Unified on June 3. The newest specialized family member is DiffusionGemma, an experimental discrete text-diffusion model released on June 10, 2026.

Development and Releases

The Gemma family has evolved from compact English-language text models into a collection containing multilingual, multimodal, recurrent, encoder-decoder, sparse Mixture-of-Experts, and diffusion architectures.

First Generation: Gemma 1

The first Gemma models were released on February 21, 2024. The initial family contained two text-only decoder models:

• Gemma 2B
• Gemma 7B

Both sizes were released as pretrained base checkpoints and instruction-tuned checkpoints. They supported a context window of 8,192 tokens. Gemma 2B was trained on approximately 2 trillion tokens, while Gemma 7B was trained on approximately 6 trillion tokens. The corpus consisted primarily of English-language web documents, code, and mathematical material.

Google reported that Gemma surpassed comparably sized open-weight models on 11 of the 18 text benchmarks evaluated in the technical report. These were developer-conducted evaluations and depended on the prompts, checkpoints, and evaluation procedures used in the report.

Gemma 1 used a decoder-only transformer with RMSNorm, RoPE, and a gated feed-forward architecture. Gemma 2B used Multi-Query Attention, whereas Gemma 7B used conventional Multi-Head Attention.

Second Generation: Gemma 2

Google released Gemma 2 9B and Gemma 2 27B on June 27, 2024. A 2B version followed on July 31.

All three Gemma 2 models remained text-only and primarily English-language. Their context window was 8,192 tokens, not 80,000 tokens. The reported pre-training volumes were:

• Gemma 2 2B: approximately 2 trillion tokens;
• Gemma 2 9B: approximately 8 trillion tokens;
• Gemma 2 27B: approximately 13 trillion tokens.

Gemma 2 introduced several architectural changes:

• Grouped-Query Attention (GQA), reducing KV-cache requirements relative to full Multi-Head Attention;
• alternating local sliding-window and global-attention layers;
• logit soft-capping to improve training stability;
• additional normalization around transformer sublayers;
• knowledge distillation for the 2B and 9B models using the larger 27B model as a teacher.

The alternating attention architecture improved computational efficiency, but it did not increase the supported context to 80K. The official model card and technical report both specify an 8,192-token context.

Third Generation: Gemma 3

Google released Gemma 3 in 1B, 4B, 12B, and 27B sizes on March 10, 2025. A compact 270M checkpoint followed on August 14, 2025.

Capabilities and context limits differed by size:

• Gemma 3 270M and 1B are text-only models with a 32K context window.
• Gemma 3 4B, 12B, and 27B accept text and images and generate text, with a context window of up to 128K tokens.
• The multimodal models can process video by accepting sequences of sampled image frames; they do not natively generate video.
• The training corpus covered more than 140 languages, although performance and officially evaluated coverage vary by language.

The 4B, 12B, and 27B models use a SigLIP-based vision encoder. Images are normalized and represented as visual tokens that are supplied to the language model. Gemma 3 increased the ratio of local-attention layers to global-attention layers to reduce KV-cache memory at long context lengths.

The model-card training volumes are approximately 2 trillion tokens for 1B, 4 trillion for 4B, 12 trillion for 12B, and 14 trillion for 27B. Google lists 6 trillion tokens for the later 270M model. The knowledge cutoff for the documented Gemma 3 training corpus is August 2024.

Gemma 3n

Gemma 3n was released in E2B and E4B variants on June 26, 2025. It was designed for local execution on phones, tablets, and laptops and accepts text, images, audio, and video while producing text output. Its context window is 32K tokens.

The letter "E" denotes an effective parameter count rather than the complete number of weights stored in the checkpoint. Gemma 3n uses a Matryoshka Transformer architecture in which smaller subnetworks are nested inside a larger model. The E4B model contains the E2B submodel, allowing selected portions of the network to be activated according to available compute. Per-Layer Embeddings and conditional loading of audio or visual parameters further reduce operating-memory requirements for particular workloads.

Fourth Generation: Gemma 4

The first Gemma 4 checkpoints were released on March 31, 2026, with public launch material following in early April. The family contains five principal sizes:

• Gemma 4 E2B — approximately 2.3 billion effective parameters and about 5.1 billion parameters when embeddings are counted;
• Gemma 4 E4B — approximately 4.5 billion effective parameters and about 8 billion including embeddings;
• Gemma 4 12B Unified — an approximately 12-billion-parameter dense model;
• Gemma 4 26B-A4B — a sparse Mixture-of-Experts model with approximately 25.2 billion total parameters and 3.8 billion active parameters;
• Gemma 4 31B — an approximately 30.7-billion-parameter dense model.

All five models accept text and images and generate text. E2B, E4B, and 12B Unified also accept audio for speech recognition and speech translation. Video understanding is implemented by processing sampled video frames. The official model card gives a maximum of 30 seconds for audio and 60 seconds of video when sampled at one frame per second.

The E2B and E4B models support contexts of up to 128K tokens. The 12B, 26B-A4B, and 31B models support up to 256K tokens. Gemma 4 was pre-trained on material covering more than 140 languages, while Google describes out-of-the-box capability in more than 35 languages. The documented training-data cutoff is January 2025.

Gemma 4 introduced:

• configurable thinking mode for additional inference-time reasoning;
• native function calling and structured tool use;
• native system-role support;
• variable-resolution image processing;
• dense and sparse MoE architectures within one generation;
• dedicated Multi-Token Prediction draft models for speculative decoding;
• quantization-aware checkpoints for lower-memory deployment.

Gemma 4 12B is described as "Unified" because it does not use separate large vision and audio encoders. Raw image patches and short audio segments are projected directly into the language model's embedding space through lightweight projection layers. The other multimodal Gemma 4 models use dedicated encoders.

The Gemma 4 26B-A4B model contains 128 routed experts. Eight routed experts and one shared expert participate in processing each token. Active-parameter count describes only part of the arithmetic performed for a token; the full model weights must still be stored or made accessible during inference.

Google released Gemma 4 under the Apache License 2.0, replacing the custom Gemma license for this generation.

Architecture and Technical Features

The Gemma family does not use one unchanged architecture. Gemma 1, Gemma 2, and the main text components of Gemma 3 are autoregressive decoder-only transformers. Later derivatives include recurrent models, encoder-decoder models, sparse MoE models, multimodal encoders, and discrete-diffusion generation.

• Decoder-Only Transformer Core: Most general-purpose Gemma models generate output autoregressively by predicting the next token from preceding context. They use causal attention and gated feed-forward layers.
• RMSNorm: Gemma uses Root Mean Square Layer Normalization rather than conventional LayerNorm. Later generations use additional pre- and post-normalization arrangements to improve stability.
• Gated Feed-Forward Layers: Gemma uses GeGLU or related gated feed-forward mechanisms rather than a simple ReLU layer. The exact implementation differs between generations.
• Rotary Position Embeddings (RoPE): Gemma 1–3 use RoPE to encode token position. Gemma 4 applies ordinary RoPE to local-attention layers and Proportional RoPE (p-RoPE) to global layers.
• Multi-Query and Grouped-Query Attention: Gemma 1 2B used Multi-Query Attention, sharing key and value representations across query heads. Gemma 2 and later core models use Grouped-Query Attention or related key-value sharing arrangements to reduce KV-cache memory.
• Local and Global Attention: Gemma 2 introduced alternating local sliding-window and global-attention layers. Gemma 3 increased the proportion of local layers for more efficient 128K processing. Gemma 4 uses four local blocks per global block in E2B and five local blocks per global block in the other models, with the final layer always using global attention.
• Multimodal Encoders: Gemma 3 uses a SigLIP-derived visual encoder for its 4B, 12B, and 27B models. Gemma 3n adds separate vision and audio processing components. Gemma 4 uses improved vision encoders and, in E2B and E4B, a Conformer-based audio encoder; Gemma 4 12B instead uses an encoder-free projection architecture.
• Mixture-of-Experts: Gemma 4 26B-A4B and DiffusionGemma use sparse MoE layers. Only a subset of experts is activated for each token, lowering arithmetic relative to a dense model with the same total parameter count. Sparse activation does not reduce model storage to the active-parameter count.
• Matryoshka Transformer: Gemma 3n nests smaller subnetworks inside a larger model so that a device can activate only the capacity required for a task.
• Multi-Token Prediction: Gemma 4 includes separate draft models that predict several future tokens for speculative decoding. The main model verifies these candidates in parallel; the technique is intended to improve serving speed without changing accepted output tokens.
• Quantization: Google distributes quantized checkpoints for several Gemma generations. Quantization reduces memory consumption and can improve inference speed, but quality, supported context, and hardware acceleration depend on the precision and runtime used.

Published context limits are model-specific:

• Gemma 1 and Gemma 2: 8,192 tokens;
• Gemma 3 270M and 1B: 32K;
• Gemma 3 4B, 12B, and 27B: 128K;
• Gemma 3n: 32K;
• Gemma 4 E2B and E4B: 128K;
• Gemma 4 12B, 26B-A4B, and 31B: 256K.

A nominal context limit states the maximum accepted sequence length. It does not guarantee perfect retrieval, reasoning, or factual consistency across every prompt of that length.

Model Family and Variants

In addition to the principal generations, Google has released multiple specialized Gemma variants.

• CodeGemma: A family of code-specialized models released in April 2024. It includes a 2B checkpoint intended for code completion and infilling, together with 7B pretrained and instruction-tuned models for code and natural-language tasks.
• DataGemma: An experimental Gemma 2-based project connecting language models to Google Data Commons. Its Retrieval-Interleaved Generation (RIG) system identifies statistical claims and requests corresponding Data Commons values. Its RAG configuration uses a Gemma 2 model to generate retrieval queries and supplies the retrieved material to a larger model, such as Gemini 1.5 Pro, for final generation. DataGemma therefore refers to a research workflow and specialized checkpoints rather than a single standalone database-answering model.
• PaliGemma and PaliGemma 2: Vision-language models combining a SigLIP vision encoder with Gemma or Gemma 2 language components. PaliGemma 2 uses 2B, 9B, or 27B language backbones, giving total model sizes of approximately 3B, 10B, and 28B. The models are intended primarily as foundations for fine-tuning on image captioning, visual question answering, OCR, detection, segmentation, and short-video tasks.
• RecurrentGemma: A family based on the Griffin architecture, which combines local attention with gated linear recurrence. The 2B and 9B variants use a bounded recurrent state during generation, avoiding the KV-cache growth of global transformer attention. RecurrentGemma was released as an experimental efficiency-oriented alternative rather than a universal replacement for transformers.
• Gemma 3n: An on-device multimodal family supporting text, images, audio, and video input. Its MatFormer and conditional-loading mechanisms allow developers to trade capability for memory and computation.
• MedGemma: Medical-domain models based on Gemma 3. The initial collection included a 4B multimodal model and 27B text and multimodal variants. MedGemma 1.5 4B expanded support for CT, MRI, whole-slide pathology, longitudinal chest X-rays, medical documents, and electronic health records. Google describes MedGemma as a starting point for application development, not as a clinically validated autonomous diagnostic system. Developers remain responsible for adaptation, validation, regulatory compliance, and human oversight.
• TxGemma: Gemma 2 models in 2B, 9B, and 27B sizes fine-tuned on Therapeutics Data Commons material. They are intended for research involving molecules, proteins, nucleic acids, therapeutic targets, and property prediction. TxGemma outputs require task-specific scientific validation and are not substitutes for laboratory, clinical, or regulatory assessment.
• TranslateGemma: Translation-specialized 4B, 12B, and 27B models based on Gemma 3. They were trained through supervised fine-tuning and reinforcement-learning stages and cover 55 languages. Their release on January 15, 2026 did not replace the general-purpose Gemma family.
• T5Gemma and T5Gemma 2: Encoder-decoder models created by adapting pretrained decoder-only Gemma checkpoints. T5Gemma 2 extends the approach to Gemma 3 and provides 270M–270M, 1B–1B, and 4B–4B encoder-decoder configurations with multilingual, multimodal, and long-context capabilities.
• EmbeddingGemma: A 308-million-parameter multilingual text-embedding model based on Gemma 3 and T5Gemma initialization. It supports more than 100 languages and is intended for retrieval, semantic similarity, classification, and clustering. It produces vector representations rather than conversational text.
• FunctionGemma: A 270-million-parameter Gemma 3 derivative trained for structured function calling. Google presents it as a foundation for further task-specific fine-tuning rather than as a general dialogue model.
• VaultGemma: A 1-billion-parameter Gemma 2-derived model trained from scratch with sequence-level differential privacy. Google reports a privacy guarantee of ε ≤ 2.0 and δ ≤ 1.1×10⁻¹⁰. Differential privacy reduces the risk of reproducing individual training examples but introduces a capability trade-off and does not by itself guarantee application-level privacy.
• ShieldGemma: Safety-classification models designed to label text inputs or outputs according to specified policies. The original ShieldGemma family contains 2B, 9B, and 27B Gemma 2-based text classifiers. ShieldGemma 2 is a 4B Gemma 3-based image-safety classifier. These systems provide classification signals; they do not independently guarantee that an application will block every unsafe input or output.
• Gemma Scope: An interpretability suite providing sparse autoencoders and related tools for examining internal model activations. The original release targeted Gemma 2; Gemma Scope 2, released in December 2025, covers Gemma 3 models and is intended for research into features, refusals, hallucinations, jailbreaks, and other internal behaviours.
• DiffusionGemma: An experimental model based on the Gemma 4 26B-A4B MoE architecture, with approximately 25.2 billion total and 3.8 billion active parameters. Unlike conventional left-to-right generation, it refines 256-token canvases through discrete diffusion and uses block-autoregressive continuation for longer output. It accepts text, images, and video frames and generates text. Google reported up to four-times-higher generation throughput in selected GPU configurations; these are vendor measurements and depend on hardware, denoising steps, implementation, prompt length, and output length.
• DolphinGemma: A research project developed with the Wild Dolphin Project and the Georgia Institute of Technology to model the structure of dolphin vocalizations and generate dolphin-like sound sequences. Google announced the project in April 2025, but the current DeepMind project page states that the model remains in development and will be made openly available after release. It should not be described as an already published production model.

Availability and Application

Gemma weights are distributed through Kaggle and Hugging Face. Google also supplies examples, integrations, or deployment paths for Keras, JAX, PyTorch, Hugging Face Transformers, Ollama, llama.cpp, MLX, Google AI Edge, LiteRT-LM, Android, Google Colab, and Vertex AI Model Garden. Support varies by model generation, modality, precision, and runtime.

Small and quantized Gemma models can run locally on selected phones, laptops, and desktop computers. Larger models may require high-memory GPUs, multiple accelerators, or server infrastructure. The ability to download a model does not imply that every size can run efficiently on ordinary consumer hardware.

Typical applications include:

• conversational assistants and text generation;
• summarization and document analysis;
• programming assistance;
• image and video-frame understanding;
• speech recognition and speech translation with supported Gemma 3n and Gemma 4 models;
• semantic search and retrieval using EmbeddingGemma;
• function calling and local agents;
• multilingual translation;
• safety classification;
• medical and therapeutic research;
• private or offline processing;
• model-interpretability research.

Gemma 4 31B and 26B-A4B are also available through supported Google-hosted interfaces, while downloadable weights can be deployed independently. Hosted-service terms, quotas, supported modalities, prices, and data handling are separate from the license governing downloaded model weights.

Licensing must be evaluated per model:

• Gemma 1–3 and many related checkpoints use the custom Gemma Terms, which permit commercial use but impose redistribution obligations and prohibited-use restrictions.
• Gemma 4 and DiffusionGemma use Apache License 2.0.
• MedGemma and TxGemma use the separate Health AI Developer Foundations terms.
• Community fine-tunes may add their own conditions but cannot remove obligations inherited from the underlying checkpoint.

Neither the custom Gemma Terms nor Apache 2.0 grants rights to third-party material that might be supplied to or produced by an application. Developers remain responsible for copyright, privacy, data protection, sector regulation, and the rights associated with their own datasets and outputs.

Safety and Responsibility

Google documents several safety measures used during Gemma development:

• Training-Data Filtering: The model cards describe filtering for child sexual abuse material, selected personal and sensitive information, low-quality material, unsafe content, and benchmark contamination. Automated filtering reduces risk but cannot establish that every undesirable or copyrighted item has been removed.
• Post-Training Alignment: Instruction-tuned checkpoints use supervised fine-tuning and preference-optimization methods involving human or model-generated feedback. The exact post-training recipe differs between generations.
• Safety Evaluation and Red Teaming: Google reports automated and human testing for dangerous content, sexual content, hate, harassment, child-safety risks, and other misuse categories. Such testing covers selected threat models and does not prove safety under every fine-tune, prompt, language, or deployment configuration.
• Responsible Generative AI Toolkit: Google publishes guidance and evaluation tools for model comparison, safety assessment, model cards, and responsible deployment. The toolkit supports developers but is not an automatic safety layer.
• ShieldGemma: ShieldGemma models can classify text or images according to defined safety policies and can be placed before or after a generative model. Their decisions should be evaluated for false positives, false negatives, language coverage, and application-specific requirements.
• Gemma Scope: Sparse-autoencoder interpretability resources can help researchers inspect model features and investigate behaviours such as refusals, jailbreaks, or hallucinations. Interpretability tools do not provide a complete causal explanation of all model outputs.
• Privacy Research: VaultGemma demonstrates differential-privacy training at one-billion-parameter scale. Its privacy guarantee concerns the specified training procedure and does not replace secure storage, access control, consent, minimization, or other application-level protections.

Gemma models retain limitations common to generative AI:

• they can produce false, outdated, fabricated, or internally inconsistent information;
• output quality varies by language, domain, prompt, model size, and quantization;
• long context does not guarantee that all supplied information will be used correctly;
• multimodal models can misread images, handwriting, charts, medical scans, or small text;
• instruction tuning can be bypassed or changed through fine-tuning;
• benchmark results may not predict performance in production environments;
• models can reproduce social, cultural, or demographic biases present in training data.

Applications involving medicine, law, finance, employment, public benefits, critical infrastructure, or physical safety require domain-specific validation, suitable human oversight, access controls, monitoring, and compliance with applicable regulation. MedGemma and TxGemma are development foundations rather than autonomous clinical or scientific decision-makers.

External links

• Official Gemma page on Google DeepMind
• Official Gemma documentation
• Gemma release chronology
• Gemma models on Hugging Face
• Gemma models on Kaggle
• Gemma (language model) — Wikipedia

Literature

• Mesnard, Thomas; Hardin, Cassidy; Dadashi, Robert; et al. (2024). Gemma: Open Models Based on Gemini Research and Technology. arXiv:2403.08295.
• Rivière, Morgane; Pathak, Shreya; Sessa, Pier Giuseppe; et al. (2024). Gemma 2: Improving Open Language Models at a Practical Size. arXiv:2408.00118.
• Kamath, Aishwarya; Ferret, Johan; Pathak, Shreya; et al. (2025). Gemma 3 Technical Report. arXiv:2503.19786.
• Gemma Team (2026). Gemma 4 Technical Report. arXiv:2607.02770.
• Zhao, Heri; Hui, Jeffrey; Howland, Joshua; et al. (2024). CodeGemma: Open Code Models Based on Gemma. arXiv:2406.11409.
• Beyer, Lucas; Steiner, Andreas; Pinto, André Susano; et al. (2024). PaliGemma: A Versatile 3B VLM for Transfer. arXiv:2407.07726.
• Steiner, Andreas; Pinto, André Susano; Tschannen, Michael; et al. (2024). PaliGemma 2: A Family of Versatile VLMs for Transfer. arXiv:2412.03555.
• Botev, Alex; De, Soham; Smith, Samuel L.; et al. (2024). RecurrentGemma: Moving Past Transformers for Efficient Open Language Models. arXiv:2404.07839.
• Radhakrishnan, Pranav; et al. (2024). Knowing When to Ask — Bridging Large Language Models and Data. arXiv:2409.13741.
• Zhang, Biao; Suganthan, Paul; Liu, Gaël; et al. (2025). T5Gemma 2: Seeing, Reading, and Understanding Longer. arXiv:2512.14856.
• Finkelstein, Mara; et al. (2026). TranslateGemma Technical Report. arXiv:2601.09012.
• Ainslie, Joshua; Lee-Thorp, James; de Jong, Michiel; et al. (2023). GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. arXiv:2305.13245.
• Su, Jianlin; Lu, Yu; Pan, Shengfeng; et al. (2021). RoFormer: Enhanced Transformer with Rotary Position Embedding. arXiv:2104.09864.
• Zhang, Biao; Sennrich, Rico (2019). Root Mean Square Layer Normalization. arXiv:1910.07467.