---
title: "T5 (Text-to-Text Transfer Transformer)"
source: "https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)"
wiki: "systems-analysis.info/eng"
article: "T5_(Text-to-Text_Transfer_Transformer)"
language: "en"
categories:
  - "Category:English"
  - "Category:Google"
  - "Category:Large language models"
  - "Category:LLM families"
  - "Category:Machine learning"
  - "Category:Technology"
revision_id: 412
wiki_created_at: 2026-09-06T22:23:12Z
wiki_modified_at: 2026-09-06T22:23:12Z
downloaded_at: 2026-09-07T22:23:14Z
---

# T5 (Text-to-Text Transfer Transformer)

**T5** (**T**ext-to-**T**ext **T**ransfer **T**ransformer) is a family of [large language models](https://systems-analysis.info/eng/Large_language_model "Large language model") developed by researchers at Google AI and introduced in 2019<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)</sup>. The key innovation of T5 is its unified **"text-to-text"** framework, which treats every natural language processing (NLP) task as a problem of converting one text sequence into another. This allowed for the use of a single model, loss function, and training procedure for a wide range of tasks, such as translation, summarization, question answering, and classification<sup>[\[2\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-google-blog-t5-2)</sup>.

The model is based on the standard [encoder-decoder](https://systems-analysis.info/eng/Encoder%E2%80%93decoder_architecture "Encoder–decoder architecture") [transformer architecture](https://systems-analysis.info/eng/Transformer_architecture "Transformer architecture"), which distinguishes it from models like [BERT](https://systems-analysis.info/eng/BERT_(language_model) "BERT (language model)") ([encoder-only](https://systems-analysis.info/eng/Encoder-only_models "Encoder-only models")) and [GPT](https://systems-analysis.info/eng/GPT_(OpenAI) "GPT (OpenAI)") ([decoder-only](https://systems-analysis.info/eng/Decoder-only_models_(architecture) "Decoder-only models (architecture)")). The work on T5 was conceived as a large-scale empirical study to systematically explore and compare various transfer learning techniques in NLP, rather than creating a fundamentally new method<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)</sup>.

## The "Text-to-Text" Paradigm

The central idea of T5 is that all tasks are formulated in a unified format. The model receives text as input and also generates text as output. To enable the model to distinguish between the tasks it is given, a special text **instruction prefix** is added to the input sequence<sup>[\[2\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-google-blog-t5-2)</sup>.

- **Translation**: \`translate English to German: That is good.\` → \`Das ist gut.\`
- **Sentiment Classification**: \`sst2 sentence: a very exciting film.\` → \`positive\`
- **Summarization**: \`summarize: \[long article text\]\` → \`\[short summary\]\`

This approach radically simplifies the process of applying the model, eliminating the need to develop specific *task-specific heads* for each individual task, which was characteristic of architectures like BERT<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)</sup>.

## Architecture and Scaling

### Encoder-Decoder Architecture

T5 uses a standard transformer architecture consisting of two parts<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)</sup>:

- **[Encoder](https://systems-analysis.info/eng/Encoder_(Transformer) "Encoder (Transformer)")**: Processes the entire input sequence at once, creating a rich, contextualized representation. Like in BERT, the T5 encoder is **bidirectional**.
- **[Decoder](https://systems-analysis.info/eng/Decoder_(Transformer) "Decoder (Transformer)")**: Generates the output text [token](https://systems-analysis.info/eng/Token_(LLM) "Token (LLM)") by token (autoregressively), using the representation provided by the encoder.

This hybrid structure allows T5 to effectively solve both language understanding and text generation tasks<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)</sup>.

### Key Improvements

The T5 architecture includes several changes compared to the original transformer model:

- **Relative Positional [Embeddings](https://systems-analysis.info/eng/Embedding_(NLP) "Embedding (NLP)")**: Instead of absolute sinusoidal embeddings, T5 uses a simplified but effective form of relative position encoding, where a learnable scalar bias is added to the attention logits, depending only on the relative distance between tokens<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)</sup>.
- **Modified Layer Normalization (Layer Norm)**: Normalization is moved outside the *residual connection*, and the additive bias is removed to improve training stability.

### Model Scales

In the original paper, the model was presented in several configurations with varying numbers of parameters, which allowed for a systematic study of the effect of scale<sup>[\[3\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-huggingface-t5-doc-3)</sup>:

- **T5-Small**: ~60 million parameters
- **T5-Base**: ~220 million parameters
- **T5-Large**: ~770 million parameters
- **T5-3B**: ~3 billion parameters
- **T5-11B**: ~11 billion parameters

The study showed that increasing the model's scale is one of the most reliable ways to improve its performance<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)</sup>.

## Pre-training: C4 Dataset and the Span Corruption Task

### The Span Corruption Task

For [pre-training](https://systems-analysis.info/eng/Pre-training_of_large_language_models "Pre-training of large language models") T5, a *denoising* objective was chosen, specifically a variant called **span corruption**<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)</sup>. The method works as follows:

1.  In the input text, 15% of the tokens are randomly masked.
2.  Unlike the MLM method in BERT, where individual tokens are masked, T5 masks entire contiguous *spans* of tokens.
3.  Each corrupted span is replaced by a single unique sentinel token (e.g., \`\<X\>\`, \`\<Y\>\`).
4.  The model is trained to generate the sequence of dropped-out spans at the output, separated by the corresponding sentinel tokens.

This approach forces the model to predict entire sequences of text, which proved to be a more effective pre-training task than simple language modeling<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)</sup>.

### The C4 Dataset (Colossal Clean Crawled Corpus)

To realize the potential of transfer learning, the researchers created a massive and high-quality cleaned text dataset called **C4**, with a size of about 750 GB<sup>[\[2\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-google-blog-t5-2)</sup>. It was derived from a large-scale cleaning and filtering of the publicly available **Common Crawl** web corpus<sup>[\[4\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-c4-dataset-pwc-4)</sup>. The cleaning process included removing duplicates, boilerplate text ("Lorem ipsum"), incomplete sentences, and filtering out offensive language<sup>[\[5\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-dodge2021-doc-c4-5)</sup>.

### Criticism of the C4 Dataset

Despite the stated goal of creating a "clean" corpus, the C4 filtering process was criticized for systemic biases. Studies showed that the profanity filter disproportionately removed texts related to LGBTQ+ communities, as well as texts in African-American English (AAE)<sup>[\[5\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-dodge2021-doc-c4-5)</sup>. Additionally, a significant amount of offensive and copyrighted content was found in the dataset. These issues illustrate the difficulty of creating objectively "high-quality" datasets and how technical filtering decisions can lead to unintended social biases.

## Results and Performance

At the time of its publication, T5 set new *state-of-the-art* performance records on numerous benchmarks, including **[GLUE](https://systems-analysis.info/eng/GLUE_Benchmark "GLUE Benchmark")**, **[SuperGLUE](https://systems-analysis.info/eng/SuperGLUE_(benchmark) "SuperGLUE (benchmark)")**, **SQuAD**, and summarization tasks<sup>[\[2\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-google-blog-t5-2)</sup>. The largest configuration, **T5-11B** (trained on roughly 1 trillion tokens from C4), scored **88.9** on **SuperGLUE** — within about a point of the human baseline of 89.8 — and reached a GLUE average of about 90.3; it did not, however, surpass specialized systems on WMT translation tasks, a gap the authors attributed to T5's English-only pre-training and its lack of backtranslation<sup>[\[1\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-raffel2020-1)[\[2\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-google-blog-t5-2)</sup>. These results confirmed the study's central hypothesis: the combination of a unified framework, massive scale, and a high-quality dataset is an extremely powerful strategy for achieving cutting-edge results in NLP.

## Evolution and Variants of T5

The T5 approach served as the foundation for many subsequent models:

- **mT5**: A multilingual version of T5 trained on the **mC4** corpus, covering **101 languages**<sup>[\[6\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-xue2020-mt5-6)</sup>.
- **ByT5**: An experimental version that completely dispenses with [tokenization](https://systems-analysis.info/eng/Tokenization_(NLP) "Tokenization (NLP)") and operates directly on **raw UTF-8 bytes**. This makes it robust to typos and allows it to process any language "out of the box"<sup>[\[7\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-xue2022-byt5-7)</sup>.
- **Switch Transformer**: A scalable version of T5 that introduced the **[Mixture-of-Experts](https://systems-analysis.info/eng/Mixture-of-Experts_(MoE) "Mixture-of-Experts (MoE)") (MoE)** architecture, allowing the number of parameters to be increased to trillions while maintaining reasonable computational costs<sup>[\[8\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-fedus2021-switch-8)</sup>.
- **FLAN-T5**: This is not a new architecture but a standard T5 that has undergone an additional [fine-tuning](https://systems-analysis.info/eng/Fine-tuning_(deep_learning) "Fine-tuning (deep learning)") step on hundreds of tasks formulated as instructions (*instruction tuning*). This significantly improved its ability to generalize to new, unseen tasks in a **zero-shot** setting (without examples)<sup>[\[9\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-chung2022-flan-9)</sup>.
- **UL2**: A model that builds on the ideas of T5, using a new pre-training objective called **Mixture of Denoisers**, which combines various text masking schemes to improve versatility<sup>[\[10\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-tay2022-ul2-10)</sup>.
- **Other variants**: Additional models in the family include **LongT5**, adapted for the efficient processing of long input sequences<sup>[\[11\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-longt5-11)</sup>, and **Sentence-T5**, which turns T5 into a strong sentence-embedding model<sup>[\[12\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-sentence-t5-12)</sup>.
- **T5Gemma** (2025): A revival of the encoder-decoder approach by Google, created by **adapting** pretrained decoder-only **Gemma 2** models into the T5-style encoder-decoder architecture through continued pre-training with the UL2 objective. Released in July 2025 in both Gemma 2-based configurations (2B-2B, 9B-9B, and an "unbalanced" 9B-2B) and newly trained T5-sized models, it showed that such adapted models can match or exceed their decoder-only counterparts while offering a better quality–efficiency trade-off<sup>[\[13\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-t5gemma-13)</sup>. A successor, **T5Gemma 2** (December 2025), is based on Gemma 3 and adds multimodal (vision) input, a 128K-token context window, and support for more than 140 languages<sup>[\[14\]](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_note-t5gemma2-14)</sup>.

## External links

- <a href="https://github.com/google-research/text-to-text-transfer-transformer" class="external text" rel="nofollow">Official T5 repository on GitHub</a>
- <a href="https://research.google/blog/exploring-transfer-learning-with-t5-the-text-to-text-transfer-transformer/" class="external text" rel="nofollow">Google Research blog post about the T5 study</a>
- <a href="https://en.wikipedia.org/wiki/T5_(language_model)" class="external text" rel="nofollow">T5 (language model) — Wikipedia</a>

## Literature

- Vaswani, A. et al. (2017). *Attention Is All You Need*. <a href="https://arxiv.org/abs/1706.03762" class="external text" rel="nofollow">arXiv:1706.03762</a>.
- Raffel, C. et al. (2020). *Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer*. <a href="https://arxiv.org/abs/1910.10683" class="external text" rel="nofollow">arXiv:1910.10683</a>.
- Xue, L. et al. (2021). *mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer*. <a href="https://arxiv.org/abs/2010.11934" class="external text" rel="nofollow">arXiv:2010.11934</a>.
- Dodge, J. et al. (2021). *Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus*. <a href="https://arxiv.org/abs/2104.08758" class="external text" rel="nofollow">arXiv:2104.08758</a>.
- Fedus, W. et al. (2021). *Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity*. <a href="https://arxiv.org/abs/2101.03961" class="external text" rel="nofollow">arXiv:2101.03961</a>.
- Ni, J. et al. (2021). *Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models*. <a href="https://arxiv.org/abs/2108.08877" class="external text" rel="nofollow">arXiv:2108.08877</a>.
- Guo, M. et al. (2021). *LongT5: Efficient Text-To-Text Transformer for Long Sequences*. <a href="https://arxiv.org/abs/2112.07916" class="external text" rel="nofollow">arXiv:2112.07916</a>.
- Xue, L. et al. (2022). *ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models*. <a href="https://arxiv.org/abs/2105.13626" class="external text" rel="nofollow">arXiv:2105.13626</a>.
- Tay, Y. et al. (2022). *UL2: Unifying Language Learning Paradigms*. <a href="https://arxiv.org/abs/2205.05131" class="external text" rel="nofollow">arXiv:2205.05131</a>.
- Chung, H. W. et al. (2022). *Scaling Instruction-Finetuned Language Models*. <a href="https://arxiv.org/abs/2210.11416" class="external text" rel="nofollow">arXiv:2210.11416</a>.
- Longpre, S. et al. (2023). *The Flan Collection: Designing Data and Methods for Effective Instruction Tuning*. <a href="https://arxiv.org/abs/2301.13688" class="external text" rel="nofollow">arXiv:2301.13688</a>.
- Zhang, B. et al. (2025). *Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation (T5Gemma)*. <a href="https://developers.googleblog.com/en/t5gemma/" class="external text" rel="nofollow">Google Developers Blog</a>.
- Zhang, B. et al. (2025). *T5Gemma 2: Seeing, Reading, and Understanding Longer*. <a href="https://arxiv.org/abs/2512.14856" class="external text" rel="nofollow">arXiv:2512.14856</a>.

## References

1.  <span id="cite_note-raffel2020-1">↑ <sup>[1.00](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-0)</sup> <sup>[1.01](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-1)</sup> <sup>[1.02](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-2)</sup> <sup>[1.03](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-3)</sup> <sup>[1.04](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-4)</sup> <sup>[1.05](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-5)</sup> <sup>[1.06](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-6)</sup> <sup>[1.07](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-7)</sup> <sup>[1.08](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-8)</sup> <sup>[1.09](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-raffel2020_1-9)</sup> Raffel, Colin; Shazeer, Noam; Roberts, Adam; et al. "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer". *Journal of Machine Learning Research*. <a href="http://jmlr.org/papers/v21/20-074.html" class="external autonumber" rel="nofollow">[1]</a></span>
2.  <span id="cite_note-google-blog-t5-2">↑ <sup>[2.0](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-google-blog-t5_2-0)</sup> <sup>[2.1](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-google-blog-t5_2-1)</sup> <sup>[2.2](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-google-blog-t5_2-2)</sup> <sup>[2.3](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-google-blog-t5_2-3)</sup> <sup>[2.4](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-google-blog-t5_2-4)</sup> "Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer". *Google Research Blog*. <a href="https://research.google/blog/exploring-transfer-learning-with-t5-the-text-to-text-transfer-transformer/" class="external autonumber" rel="nofollow">[2]</a></span>
3.  <span id="cite_note-huggingface-t5-doc-3">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-huggingface-t5-doc_3-0) "T5". *Hugging Face Transformers Documentation*. <a href="https://huggingface.co/transformers/v4.12.5/model_doc/t5.html" class="external autonumber" rel="nofollow">[3]</a></span>
4.  <span id="cite_note-c4-dataset-pwc-4">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-c4-dataset-pwc_4-0) "C4 Dataset". *Papers With Code*. <a href="https://paperswithcode.com/dataset/c4" class="external autonumber" rel="nofollow">[4]</a></span>
5.  <span id="cite_note-dodge2021-doc-c4-5">↑ <sup>[5.0](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-dodge2021-doc-c4_5-0)</sup> <sup>[5.1](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-dodge2021-doc-c4_5-1)</sup> Dodge, J.; Sap, M.; Marasović, A.; et al. "Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus". *arXiv*. <a href="https://arxiv.org/abs/2104.08758" class="external autonumber" rel="nofollow">[5]</a></span>
6.  <span id="cite_note-xue2020-mt5-6">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-xue2020-mt5_6-0) Xue, Linting; Constant, Noah; Roberts, Adam; et al. "mT5: A massively multilingual pre-trained text-to-text transformer". *arXiv*. <a href="https://arxiv.org/abs/2010.11934" class="external autonumber" rel="nofollow">[6]</a></span>
7.  <span id="cite_note-xue2022-byt5-7">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-xue2022-byt5_7-0) Xue, Linting; Barua, Aditya; Constant, Noah; et al. "ByT5: Towards a token-free future with pre-trained byte-to-byte models". *arXiv*. <a href="https://arxiv.org/abs/2105.13626" class="external autonumber" rel="nofollow">[7]</a></span>
8.  <span id="cite_note-fedus2021-switch-8">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-fedus2021-switch_8-0) Fedus, William; Zoph, Barret; Shazeer, Noam. "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity". *arXiv*. <a href="https://arxiv.org/abs/2101.03961" class="external autonumber" rel="nofollow">[8]</a></span>
9.  <span id="cite_note-chung2022-flan-9">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-chung2022-flan_9-0) Chung, Hyung Won; et al. "Scaling Instruction-Finetuned Language Models". *arXiv*. <a href="https://arxiv.org/abs/2210.11416" class="external autonumber" rel="nofollow">[9]</a></span>
10. <span id="cite_note-tay2022-ul2-10">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-tay2022-ul2_10-0) Tay, Yi; Dehghani, Mostafa; Tran, Vinh; et al. "UL2: Unifying Language Learning Paradigms". *arXiv*. <a href="https://arxiv.org/abs/2205.05131" class="external autonumber" rel="nofollow">[10]</a></span>
11. <span id="cite_note-longt5-11">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-longt5_11-0) Guo, Mandy; et al. "LongT5: Efficient Text-To-Text Transformer for Long Sequences". *arXiv*. <a href="https://arxiv.org/abs/2112.07916" class="external autonumber" rel="nofollow">[11]</a></span>
12. <span id="cite_note-sentence-t5-12">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-sentence-t5_12-0) Ni, Jianmo; et al. "Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models". *arXiv*. <a href="https://arxiv.org/abs/2108.08877" class="external autonumber" rel="nofollow">[12]</a></span>
13. <span id="cite_note-t5gemma-13">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-t5gemma_13-0) Zhang, Biao; et al. (2025). "T5Gemma: A new collection of encoder-decoder Gemma models". *Google Developers Blog*. <a href="https://developers.googleblog.com/en/t5gemma/" class="external autonumber" rel="nofollow">[13]</a></span>
14. <span id="cite_note-t5gemma2-14">[↑](https://systems-analysis.info/eng/T5_(Text-to-Text_Transfer_Transformer)#cite_ref-t5gemma2_14-0) Zhang, Biao; et al. (2025). "T5Gemma 2: Seeing, Reading, and Understanding Longer". *arXiv*. <a href="https://arxiv.org/abs/2512.14856" class="external autonumber" rel="nofollow">[14]</a></span>