--- title: "Stochastic parrot" source: "https://systems-analysis.info/eng/Stochastic_parrot" wiki: "systems-analysis.info/eng" article: "Stochastic_parrot" language: "en" categories: - "Category:English" - "Category:Large language models" - "Category:Machine learning" - "Category:Technology" revision_id: 347 wiki_created_at: 2026-09-06T22:22:26Z wiki_modified_at: 2026-09-06T22:22:26Z downloaded_at: 2026-09-07T22:22:52Z --- # Stochastic parrot **Stochastic parrot** is a metaphor used in the field of artificial intelligence to describe [large language models](https://systems-analysis.info/eng/Large_language_model "Large language model") (LLMs) as systems that can combine linguistic forms in statistically plausible ways, but lack any genuine understanding of their meaning^{[\[1\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-mint_def-1)}. The term was introduced in March 2021 in the research paper *On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?*, published at the FAccT conference. The authors were Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell^{[\[2\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-bender_2021-2)}. ## Definition and Concept According to the paper's authors, a stochastic parrot is "a system for haphazardly stitching together sequences of linguistic forms observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning"^{[\[2\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-bender_2021-2)}. The term consists of two parts: - ***Stochastic*** — from the Ancient Greek *στοχαστικός* ("based on conjecture"), which in modern mathematics refers to a process determined by a random probability distribution^{[\[1\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-mint_def-1)}. - ***Parrot*** — an allusion to the ability of parrots to mimic human speech without understanding its meaning^{[\[1\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-mint_def-1)}. The concept posits that LLMs, trained to predict the next word in a sequence, are essentially sophisticated autocomplete systems that manipulate symbols without access to their meaning. ## Key Arguments of "On the Dangers of Stochastic Parrots" The paper highlights four main categories of risks associated with developing excessively large language models. ### 1. Unknowable Training Data LLMs are trained on vast, unannotated datasets collected from the internet (e.g., Common Crawl). Such datasets inevitably contain biases, toxic language, and hegemonic viewpoints that harm vulnerable groups. For example, internet content disproportionately represents white men from developed countries (67% of Reddit users in the US are male)^{[\[2\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-bender_2021-2)}. ### 2. Lack of Genuine Language Understanding The authors argue that LLMs do not possess a genuine understanding of language. They refer to the theory that language is a system of signs where the form (the word) is inextricably linked to the meaning (the concept). The training data for LLMs contains only form, denying the model access to meaning. Therefore, LLMs only mimic meaningful speech. ### 3. Synthetic Text and Potential Harm Since LLMs generate grammatically correct and convincing text, people are inclined to attribute meaning to it and trust it. This creates a risk of spreading disinformation, hate speech, and fraud. The more perfect the imitation, the higher the risk that people will overestimate the AI's capabilities and entrust it with critical decisions. ## Academic Impact and Publication Controversy The paper became the center of a major scandal at Google, where co-authors Timnit Gebru and Margaret Mitchell were employed at the time. In late 2020, during an internal review, Google management demanded that the authors either retract the paper or remove the names of Google employees from it. Timnit Gebru, a leading AI ethics researcher, refused to comply with this demand, which led to her dismissal from Google in December 2020. In February 2021, Margaret Mitchell, who had spoken out in support of Gebru, was also fired^{[\[3\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-techreview_paper-3)}. These events caused a widespread public outcry. More than 2,200 Google employees and thousands of members of the academic community signed a protest letter, accusing the company of academic censorship and suppressing research that could affect its commercial interests^{[\[4\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-verge_paper-4)}. Ultimately, the paper was published in March 2021 at the FAccT conference. ## Reception, Debates, and Evolving Views The "stochastic parrot" metaphor quickly spread and became a central point in the debate about the nature of artificial intelligence. The American Dialect Society (ADS) chose **"stochastic parrot"** as its AI-related Word of the Year for 2023, where it surpassed even "ChatGPT" and "LLM"^{[\[1\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-mint_def-1)}. ### Criticism of the Concept and Evidence of Understanding The concept has been challenged by many leading researchers. - **Geoffrey Hinton**, one of the "godfathers" of deep learning, argued that "to accurately predict the next word, you need to understand the sentence"^{[\[1\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-mint_def-1)}. In 2023, after leaving Google, he stated that large models already "understand" what they are taught and can draw their own conclusions^{[\[5\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-cbs_hinton-5)}. - **Emergent abilities**: Studies have shown that upon reaching a certain scale, LLMs exhibit a sudden emergence of new abilities, such as solving arithmetic problems, that were not explicitly programmed into them^{[\[6\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-wiki_parrot-6)}. - **Internal world models**: A 2022 study showed that a model trained to play Othello based on textual records of moves spontaneously formed an internal representation of the game board, suggesting the development of an abstract model of the world it describes. - **Performance on benchmarks**: Modern models, such as [GPT](https://systems-analysis.info/eng/GPT_(OpenAI) "GPT (OpenAI)")-4, achieve human-level (or higher) performance on complex professional exams, which some argue is impossible without understanding^{[\[7\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-pmc_debate-7)}. ### Ironic Usage and Public Discourse The term became so popular that it was even used ironically by OpenAI CEO **Sam Altman**, who tweeted: "i am a stochastic parrot, and so r u". With this, he implied that human speech is also largely a probabilistic prediction of the next word, playing on the criticism leveled at AI^{[\[1\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-mint_def-1)}. ### Impact on Scientific Discourse The "stochastic parrot" metaphor remains central to debates about the capabilities and limitations of LLMs. It has helped articulate the problem of language models' lack of genuine understanding and has drawn attention to the risks associated with their development. At the same time, rapid progress in the LLM field compels a constant reassessment of this metaphor, as the latest models exhibit increasingly complex behavior that does not fit the image of a "mindless parrot". The term continues to influence scientific discourse, emphasizing the importance of critically analyzing the capabilities of AI systems and their social consequences^{[\[7\]](https://systems-analysis.info/eng/Stochastic_parrot#cite_note-pmc_debate-7)}. ## External links - Stochastic parrot — Wikipedia ## See also - [LLM hallucinations](https://systems-analysis.info/eng/LLM_hallucinations "LLM hallucinations") - [Bias in large language models](https://systems-analysis.info/eng/Bias_in_large_language_models "Bias in large language models") - [Generation bias (LLM)](https://systems-analysis.info/eng/Generation_bias_(LLM) "Generation bias (LLM)") - [Explainable AI](https://systems-analysis.info/eng/Explainable_AI "Explainable AI") - [Pre-training of large language models](https://systems-analysis.info/eng/Pre-training_of_large_language_models "Pre-training of large language models") - [Training large language models](https://systems-analysis.info/eng/Training_large_language_models "Training large language models") - [Transformer architecture](https://systems-analysis.info/eng/Transformer_architecture "Transformer architecture") - [Multimodal large language models](https://systems-analysis.info/eng/Multimodal_large_language_models "Multimodal large language models") - [Large language model architectures](https://systems-analysis.info/eng/Large_language_model_architectures "Large language model architectures") ## Literature - Bender, E. M.; Gebru, T.; McMillan-Major, A.; Mitchell, M. (2021). *On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?*. FAccT 2021. - Floridi, L.; Chiriatti, M. (2020). *GPT-3: Its Nature, Scope, Limits, and Consequences*. Minds & Machines, 30(4), 681-694. - Bommasani, R. et al. (2021). *On the Opportunities and Risks of Foundation Models*. arXiv:2108.07258. - Weidinger, L. et al. (2021). *Ethical and Social Risks of Harm from Language Models*. arXiv:2112.04359. - Kaplan, J. et al. (2020). *Scaling Laws for Neural Language Models*. arXiv:2001.08361. - Wei, J. et al. (2022). *Emergent Abilities of Large Language Models*. arXiv:2206.07682. - Perez, E. et al. (2022). *Red Teaming Language Models with Language Models*. EMNLP 2022. - Bai, Y. et al. (2022). *Constitutional AI: Harmlessness from AI Feedback*. arXiv:2212.08073. - Du, Z. et al. (2024). *Understanding Emergent Abilities of Language Models from the Loss Perspective*. arXiv:2403.15796. - Gerstgrasser, M. et al. (2024). *Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data*. arXiv:2404.01413. ## References 1. ↑ ^{[1.0](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-mint_def_1-0)} ^{[1.1](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-mint_def_1-1)} ^{[1.2](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-mint_def_1-2)} ^{[1.3](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-mint_def_1-3)} ^{[1.4](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-mint_def_1-4)} ^{[1.5](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-mint_def_1-5)} 'Stochastic Parrot': A Name for AI That Sounds a Bit Less Intelligent. *Mint*. [1] 2. ↑ ^{[2.0](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-bender_2021_2-0)} ^{[2.1](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-bender_2021_2-1)} ^{[2.2](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-bender_2021_2-2)} Bender, Emily M., et al. "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?". *Conference on Fairness, Accountability, and Transparency (FAccT '21)*. [2] 3. [↑](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-techreview_paper_3-0) Hao, Karen. "We read the paper that forced Timnit Gebru out of Google. Here's what it says". *MIT Technology Review*. [3] 4. [↑](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-verge_paper_4-0) Vincent, James. "Timnit Gebru's actual paper may explain why Google ejected her". *The Verge*. [4] 5. [↑](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-cbs_hinton_5-0) "Geoffrey Hinton on the promise, risks of artificial intelligence". *60 Minutes - CBS News*. [5] 6. [↑](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-wiki_parrot_6-0) "Stochastic parrot". *Wikipedia*. [6] 7. ↑ ^{[7.0](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-pmc_debate_7-0)} ^{[7.1](https://systems-analysis.info/eng/Stochastic_parrot#cite_ref-pmc_debate_7-1)} "The debate over understanding in AI's large language models". *PMC*. [7]