Generation bias (LLM)

From Systems analysis Wiki
Jump to navigation Jump to search

Bias in large language models (LLMs) is the systematic distortion of generated text, where the model reflects or amplifies existing societal stereotypes and prejudices related to gender, race, culture, political views, and other social categories. This phenomenon occurs because LLMs are trained on vast amounts of human data, which inevitably contain biased information[1].

Bias is one of the key ethical and technical challenges in AI development, as it can lead to discrimination, the spread of misinformation, and the erosion of trust in technology.

Types of Bias in LLMs

Bias in LLMs can manifest in various forms.

Gender Bias

Models tend to reproduce traditional gender stereotypes, associating professions and characteristics with a specific gender.

  • A 2024 UNESCO study showed that LLMs are four times more likely to describe women in domestic roles ("home," "family," "children") than men, while men are associated with concepts like "business" and "career"[2].
  • A study in Nature Scientific Reports revealed significant gender and racial bias in the content generated by seven leading LLMs, including ChatGPT and LLaMA[3].
  • In the Russian-language context, models often default to the masculine gender for neutral roles (e.g., "врач" [doctor], "директор" [director]) and struggle to generate feminatives.

Racial and Ethnic Bias

LLMs can exhibit subtle discrimination against various ethnic groups.

  • A Bloomberg study showed that ChatGPT 3.5 preferred resumes from Asian candidates over Black candidates[4].
  • In the Russian-language context, the RuBia dataset revealed that models can reproduce antisemitic and anti-immigrant stereotypes (e.g., agreeing with the statement "immigrants are lazy") if they are present in the training corpus[5].

Political and Ideological Bias

Despite claims of neutrality, many LLMs show a tendency toward a specific political spectrum.

  • A study by the Centre for Policy Studies found a left-liberal bias in 23 out of 24 tested LLMs[6].
  • Testing by the University of Washington and Carnegie Mellon showed that ChatGPT and GPT-4 were the most left-libertarian, while Meta's LLaMA was the most right-authoritarian[7].

Mechanisms of Bias Emergence

  • Training Data: The primary source. LLMs are trained on vast text corpora from the internet, which mirror society with all its stereotypes.
  • Architecture and Training Algorithms: The transformer architecture itself can amplify existing correlations in the data.
  • Fine-tuning and RLHF: The reinforcement learning from human feedback (RLHF) stage can also introduce bias, as human evaluators are inevitably guided by their own views.

Detection and Mitigation Methods

Bias Detection

  • Stereotype Test Sets: Specialized datasets are used, such as:
    • CrowS-Pairs: Covers nine types of bias, including race, religion, and age[8].
    • StereoSet: Measures stereotypical bias in four domains: gender, profession, race, and religion[9].
    • RuBia: A specialized dataset for detecting bias in Russian-language models[10].
    • Multilingual Resources: Adaptations such as French CrowS-Pairs[11] and the Chinese Bias Benchmark (CBBQ)[12].
    • Domain-Specific Analysis: Studies on bias in recruiting[13], medicine[14], and other areas.

Bias Mitigation

  • Data Level (Pre-processing): Cleaning, filtering, and rebalancing training corpora. Methods are described in the Holistic AI documentation[15].
  • Training Level (In-processing): Modifying training algorithms to account for fairness.
  • Output Level (Post-processing): Filtering and moderating already generated responses.

Bias in AI has serious consequences, including discrimination in critical areas and the spread of misinformation.

  • Regulation: Governments around the world are beginning to introduce regulations to control AI.
  • In Europe, the AI Act has been adopted and is coming into force in stages from August 1, 2024. It imposes strict requirements on high-risk systems, including mandatory bias assessment, and provides for fines of up to 7% of a company's global turnover[16].
  • In Russia, leading technology companies signed a voluntary Code of Ethics in AI in 2021, committing to minimize discrimination. By the end of 2021, over 100 organizations had signed it[17].

The fight against bias is a constant compromise. Overly aggressive filtering can lead to "excessive political correctness," where the model refuses to discuss any sensitive topics. Therefore, developers are seeking a balance between the model's safety, objectivity, and informativeness.

See also

Literature

  • Guo, Y. et al. (2024). Bias in Large Language Models: Origin, Evaluation, and Mitigation. arXiv:2411.10915.
  • Gallegos, I. O. et al. (2023). Bias and Fairness in Large Language Models: A Survey. arXiv:2309.00770.
  • Bender, E. M. et al. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. doi:10.1145/3442188.3445922.
  • Nadeem, M. et al. (2020). StereoSet: Measuring Stereotypical Bias in Pretrained Language Models. arXiv:2004.09456.
  • Nangia, N. et al. (2020). CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models. ACL 2020.
  • Bai, X. et al. (2024). Measuring Implicit Bias in Explicitly Unbiased Large Language Models. arXiv:2402.04105.
  • Hofmann, V. et al. (2024). AI Generates Covertly Racist Decisions about People Based on Their Dialect. Nature, 633, 147-154. Full text.
  • Fang, X. et al. (2024). Bias of AI-Generated Content: An Examination of News Produced by Large Language Models. Scientific Reports, 14, 5224. Full text.
  • Grigoreva, V. et al. (2024). RuBia: A Russian Language Bias Detection Dataset. arXiv:2403.17553.
  • Du, L. et al. (2024). Causal-Guided Active Learning for Debiasing Large Language Models. arXiv:2408.12942.
  • Ayaz, A. et al. (2023). Taught by the Internet: Exploring Bias in OpenAI’s GPT-3. arXiv:2306.02428.

References

  1. ↑ "Bias in Large Language Models: Origin, Evaluation, and Mitigation." arXiv. [1]
  2. ↑ "Generative AI: UNESCO study reveals alarming evidence of regressive gender stereotypes." UNESCO. [2]
  3. ↑ "Gender and race stereotypes in Large Language Models." Nature Scientific Reports. [3]
  4. ↑ "ChatGPT’s Racial Bias in Hiring Decisions." Business Insider. [4]
  5. ↑ "RuBia: A Russian-language Bias Detection Dataset." The Moonlight. [5]
  6. ↑ "Left-leaning bias commonplace in AI-powered chatbots, shows new report." Centre for Policy Studies. [6]
  7. ↑ "AI language models are rife with political biases." MIT Technology Review. [7]
  8. ↑ "CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models." ACL Anthology. [8]
  9. ↑ "StereoSet: Measuring stereotypical bias in pretrained language models." arXiv. [9]
  10. ↑ "RuBia: A Russian Language Bias Detection Dataset." arXiv. [10]
  11. ↑ "French CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in French Language Models." ACL Anthology. [11]
  12. ↑ "CBBQ: A Chinese Bias Benchmark for Large Language Models." arXiv. [12]
  13. ↑ "Bias in Large Language Models and Who Should Be Held Accountable." Stanford Law School. [13]
  14. ↑ "Racial bias in psychiatric diagnosis and treatment with large language models." Nature Digital Medicine. [14]
  15. ↑ "Preprocessing Bias Mitigation." Holistic AI Documentation. [15]
  16. ↑ "EU AI Act: First Rules Take Effect on Prohibited AI Systems." Jones Day. [16]
  17. ↑ "Over 100 organizations signed up for Code of Ethics in AI by end of 2021." TASS. [17]
Retrieved from "https://systems-analysis.info/eng/index.php?title=Generation_bias_(LLM)&oldid=165"