---
title: "Neural Text Degeneration — นิวรัลเทกซ์ดีเจเนอเรชัน (การเสื่อมสภาพของข้อความในโมเดลภาษาประสาทเทียม)"
source: "https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)"
wiki: "systems-analysis.info/int"
article: "Neural_Text_Degeneration_—_นิวรัลเทกซ์ดีเจเนอเรชัน_(การเสื่อมสภาพของข้อความในโมเดลภาษาประสาทเทียม)"
language: "th"
categories:
  - "Category:Core LLM concepts"
  - "Category:Large language models"
  - "Category:Machine learning"
  - "Category:Thai"
revision_id: 4887
wiki_created_at: 2026-09-06T23:41:28Z
wiki_modified_at: 2026-09-06T23:41:28Z
downloaded_at: 2026-09-07T23:05:32Z
---

# Neural Text Degeneration — นิวรัลเทกซ์ดีเจเนอเรชัน (การเสื่อมสภาพของข้อความในโมเดลภาษาประสาทเทียม)

**นิวรัลเทกซ์ดีเจเนอเรชัน** หรือ **การเสื่อมสภาพของข้อความในการสร้างด้วยโครงข่ายประสาทเทียม** (อังกฤษ: *Neural Text Degeneration*) คือกลุ่มของความบกพร่องในการสร้างข้อความของโมเดลภาษาประสาทเทียม ซึ่งทำให้ผลลัพธ์ที่ได้ไม่เป็นธรรมชาติ ไม่ว่าจะเป็นการซ้ำซากมากเกินไป มีลักษณะเป็นแม่แบบ มีเนื้อหาน้อย หรือสูญเสียความเชื่อมโยง คำนี้แพร่หลายขึ้นหลังจากงานวิจัยของ Ari Holtzman และคณะ เรื่อง *The Curious Case of Neural Text Degeneration* (2019; ICLR 2020) ซึ่งแสดงให้เห็นว่ากลยุทธ์การ decoding ที่อิงการเพิ่มค่าความน่าจะเป็นโดยตรงมักให้ผลลัพธ์ที่มีคุณภาพต่ำในงานการสร้างข้อความแบบเปิด<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

## คำอธิบายเบื้องต้น

**นิวรัลเทกซ์ดีเจเนอเรชัน** คือสถานการณ์ที่โมเดล**สร้างข้อความต่อได้อย่างเป็นทางการ** แต่ทำได้ไม่ดีในแง่ของภาษาที่เป็นธรรมชาติ

โดยทั่วไปจะปรากฏในลักษณะดังนี้:

- โมเดลเริ่มซ้ำคำหรือวลีเดิมซ้ำๆ
- ข้อความกลายเป็นแม่แบบและ «ไร้ชีวิตชีวา» มากเกินไป
- คำตอบสูญเสียความเชื่อมโยงทางความหมาย
- คำที่สุ่มหรือไม่เหมาะสมเริ่มปรากฏในข้อความที่ต่อออกมา

**กล่าวอีกนัยหนึ่ง** โมเดลจะกลายเป็นสิ่งที่ *«ระมัดระวังมากเกินไป»* และเลือกสิ่งเดิมซ้ำๆ ตลอดเวลา หรือในทางกลับกัน ได้รับอิสระมากเกินไปและเริ่มดึงข้อความต่อที่อ่อนแอและมีสัญญาณรบกวน ในทั้งสองกรณี ข้อความจะเลิกมีลักษณะคล้ายภาษามนุษย์ที่เป็นธรรมชาติ<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

## ประวัติการศึกษา

ปัญหานี้ถูกอธิบายอย่างละเอียดโดย Holtzman และคณะในปี 2019 โดยอาศัยข้อมูลจากการสร้างข้อความแบบเปิด โดยเฉพาะเรื่องราว การต่อข้อความอิสระ และการตอบโต้ในบทสนทนา ผู้เขียนแสดงให้เห็นว่าวิธีการ decoding ที่มุ่งเพิ่มค่าความน่าจะเป็น เช่น **greedy search** และ **beam search** มักนำไปสู่ลำดับที่ไม่เป็นธรรมชาติ ซ้ำซาก และมีเนื้อหาน้อยในงานแบบเปิด<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ในงานชิ้นเดียวกันยังแสดงให้เห็นอีกว่าแนวทางตรงข้าม ซึ่งก็คือ**การสุ่มตัวอย่างแบบ stochastic ล้วนๆ** โดยไม่ตัดส่วนหาง ก็มีปัญหาเช่นกัน เพราะโมเดลอาจเลือก token จาก «หางที่ไม่น่าเชื่อถือ» ของการแจกแจงความน่าจะเป็น ซึ่งทำให้ความเชื่อมโยงลดลงและเพิ่มความเสี่ยงของข้อความต่อที่สุ่มและไม่สอดคล้องกัน<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

งานวิจัยในภายหลังได้ชี้แจงและขยายคำอธิบายนี้ โดยเฉพาะ Finlayson และคณะได้เสนอการตีความเชิงทฤษฎีว่าเหตุใดวิธีการตัดส่วนหาง (เช่น Top-p) จึงมักช่วยลดการเสื่อมสภาพของข้อความได้ เนื่องจากการตัดส่วนหางช่วยหลีกเลี่ยง token ที่โมเดลไม่น่าเชื่อถือเป็นพิเศษ<sup>[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

Meister และคณะยังเชื่อมโยงปัญหาการเสื่อมสภาพกับการละเมิด local typicality อีกด้วย โดยข้อความของมนุษย์มักพยายามรักษาเนื้อหาข้อมูลให้ใกล้เคียงกับ conditional entropy ที่คาดหวัง แทนที่จะเพิ่มค่าความน่าจะเป็นของ token ถัดไปแต่ละตัวให้สูงสุดเพียงอย่างเดียว<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

## รูปแบบหลักของการแสดงออก

นิวรัลเทกซ์ดีเจเนอเรชันอาจแสดงออกมาไม่เพียงแค่ในรูปแบบของการซ้ำที่ชัดเจน แต่ยังรวมถึงรูปแบบที่ซ่อนเร้นกว่า ตั้งแต่ความยากจนทางความหมายไปจนถึงการสลายตัวของความเชื่อมโยงอย่างค่อยเป็นค่อยไป ข้อความลักษณะนี้มักยังคงถูกต้องทางไวยากรณ์อยู่ แต่เริ่มแตกต่างอย่างเห็นได้ชัดจากภาษามนุษย์ที่เป็นธรรมชาติในด้านความหลากหลาย ความมีข้อมูล และความสม่ำเสมอในการพัฒนาความคิด<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

### การซ้ำและการวนซ้ำ

รูปแบบที่รู้จักกันดีที่สุดคือ**การซ้ำของ token วลี หรือโครงสร้างทางวากยสัมพันธ์ทั้งหมด** ข้อความเริ่ม «วนเวียน» โดยเพิ่มข้อความต่อที่แทบจะเหมือนกันซ้ำแล้วซ้ำอีก<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

การเสื่อมสภาพลักษณะนี้อาจปรากฏในระดับต่างๆ:

- การซ้ำคำเดี่ยว
- การผลิตซ้ำวลีเดิมซ้ำๆ
- การวนกลับมาที่ความคิดเดิมซ้ำๆ
- การคัดลอกโครงสร้างทางวากยสัมพันธ์เดิมหลายครั้งด้วยการเปลี่ยนแปลงน้อยที่สุด<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

สิ่งที่น่าสังเกตเป็นพิเศษคือการซ้ำมักกลายเป็น**การเสริมตัวเอง** เมื่อโครงสร้างที่เกิดขึ้นครั้งหนึ่งเริ่มเสริมความแข็งแกร่งให้ตัวเอง เนื่องจากโมเดลยังคงอ้างอิงส่วนที่สร้างไปแล้วและสร้างซ้ำในรูปแบบที่ปรับเปลี่ยนเล็กน้อย ผลลัพธ์คือข้อความอาจดูเหมือนยังคงเดินหน้าอยู่ แต่ในความเป็นจริงหยุดพัฒนาในเชิงเนื้อหา<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

### ความเป็นแม่แบบและความยากจนของเนื้อหา

แม้จะไม่มีวงจรซ้ำที่ชัดเจน โมเดลก็อาจสร้างข้อความที่คาดเดาได้และมีเนื้อหาน้อยเกินไป ผลลัพธ์ดังกล่าวยังคงถูกต้องทางไวยากรณ์ แต่กลายเป็นข้อความที่ธรรมดา น่าเบื่อ และยากจนในเชิงข้อมูล<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ในการเสื่อมสภาพรูปแบบนี้ ข้อความมักดู «ถูกต้อง» แต่ให้ความรู้สึกว่าปลอดภัยและเฉลี่ยมากเกินไป โมเดลเลือกคำและสำนวนที่คาดหวังมากที่สุด ทำให้:

- ความหลากหลายทางคำศัพท์ลดลง
- แม่แบบการพูดเดิมถูกทำซ้ำบ่อยขึ้น
- สำนวนที่ไม่ชัดเจนแต่มีความหมายมากกว่าหายไป
- ข้อความมีรูปแบบที่เรียบร้อยขึ้นแต่ยากจนในแง่ความหมาย<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

นั่นคือเหตุผลที่การเสื่อมสภาพไม่ได้ดูเหมือนข้อผิดพลาดที่ชัดเจนเสมอไป บางครั้งมันแสดงออกเป็น**ข้อความที่คาดเดาได้มากเกินไป «เรียบ» และว่างเปล่า** ซึ่งเขียนได้ถูกต้องในเชิงรูปแบบ แต่แทบไม่ให้ข้อมูลใหม่<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

### การสูญเสียความเชื่อมโยง

เมื่อมีความสุ่มมากเกินไปหรือควบคุมส่วนหางของการแจกแจงได้ไม่ดี การสร้างข้อความอาจยังคงความถูกต้องทางไวยากรณ์ในระดับท้องถิ่น แต่สูญเสียความเชื่อมโยงทางความหมายโดยรวม กล่าวคือ คำที่ไม่เหมาะสม การเปลี่ยนหัวข้ออย่างกะทันหัน และช่องว่างทางตรรกะเริ่มปรากฏในข้อความ<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ในสถานการณ์เช่นนี้ ประโยคที่อยู่ติดกันอาจดูยอมรับได้แยกกัน แต่ความเชื่อมโยงเชิงเหตุผล เชิงหัวข้อ และเชิงตรรกะระหว่างกันอ่อนแอลง ซึ่งแสดงออกในลักษณะที่:

- ประโยคถูกต้องทางไวยากรณ์แต่เชื่อมโยงกับประโยคก่อนหน้าได้ไม่ดี
- ความคิดเปลี่ยนทิศทางอย่างกะทันหันโดยไม่มีการเปลี่ยนผ่านที่ชัดเจน
- ข้อความต่อที่ยอมรับได้ในระดับท้องถิ่นแต่ไม่เหมาะสมในบริบทปรากฏขึ้น
- ทิศทางการให้เหตุผลโดยรวมกลายเป็นหลวมและไม่มั่นคง<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

กล่าวอีกนัยหนึ่ง โมเดลอาจรักษา**ความเรียบในระดับท้องถิ่น**ในระดับวลีแต่ละประโยค แต่สูญเสีย**ความเชื่อมโยงระดับโลก**ในระดับย่อหน้าทั้งหมดหรือคำตอบที่ยาว<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

### การเลื่อนดริฟต์ทางความหมาย

อีกรูปแบบหนึ่งของการเสื่อมสภาพคือ**การค่อยๆ เคลื่อนความหมาย** ซึ่งข้อความไม่ล่มสลายทันที แต่ค่อยๆ เบี่ยงเบนออกจากหัวข้อเดิม งาน หรือเส้นทางการบรรยาย<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ต่างจากความไม่เชื่อมโยงที่ชัดเจน การเสื่อมสภาพในรูปแบบนี้พัฒนาขึ้นอย่างค่อยเป็นค่อยไป:

- โมเดลเริ่มต้นด้วยคำตอบที่เกี่ยวข้อง
- จากนั้นเพิ่มรายละเอียดที่เป็นทั่วไปหรือเชื่อมโยงอ่อนแอมากขึ้นเรื่อยๆ
- หลังจากนั้นหัวข้อเดิมเลือนหาย
- ในที่สุดข้อความอาจยังคงราบรื่นในเชิงรูปแบบ แต่ตอบสนองต่อคำถามเดิมได้ไม่ดีแล้ว<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

การดริฟต์ดังกล่าวเห็นได้ชัดเจนเป็นพิเศษในการสร้างข้อความยาว เนื่องจากโมเดลไม่จำเป็นต้องตกอยู่ในวงวน แต่ทีละขั้นตอนสูญเสียวินัยทางหัวข้อ และข้อความต่อไปหยุดมีความเกี่ยวข้องอย่างแท้จริง<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

### การลดลงของความมีข้อมูล

การเสื่อมสภาพอาจแสดงออกมาไม่เพียงแค่ในการซ้ำ แต่ยังรวมถึงการที่ข้อความหยุดเพิ่มข้อมูลใหม่ โมเดลยังคงเขียนต่อ แต่ «ความหนาแน่น» ทางความหมายของผลลัพธ์ลดลง และข้อความต่อกลายเป็นสิ่งที่ว่างเปล่าในเชิงเนื้อหา<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

ในข้อความดังกล่าว คำและประโยคยังคงสะสมอยู่ แต่การก้าวหน้าทางความหมายช้าลง วลีใหม่:

- กล่าวซ้ำสิ่งที่พูดไปแล้ว
- เพิ่มข้อเท็จจริงใหม่ในปริมาณน้อยที่สุด
- แทนที่ความเฉพาะเจาะจงด้วยคำทั่วไป
- รักษาปริมาณของข้อความโดยไม่มีการเติบโตของเนื้อหาที่เทียบเคียงได้<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

สิ่งนี้มีความสำคัญเป็นพิเศษเพราะการเสื่อมสภาพลักษณะนี้อาจดูสังเกตได้น้อยกว่าการซ้ำที่ชัดเจน ข้อความดูเหมือน «ปกติ» แต่เมื่ออ่านอย่างตั้งใจจะพบว่าส่วนใหญ่กำลังทำซ้ำความคิดเดิมด้วยคำพูดที่ต่างกัน<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

### การเสื่อมสภาพที่ซ่อนเร้นโดยไม่มีความล้มเหลวที่ชัดเจน

นิวรัลเทกซ์ดีเจเนอเรชันไม่ได้ปรากฏเป็นความเสียหายอย่างรุนแรงเสมอไป ในบางกรณีข้อความไม่มีวงจรซ้ำที่ชัดเจน ช่องว่างทางตรรกะที่รุนแรง หรือ «สัญญาณรบกวน» ที่เห็นได้ชัด แต่ก็ยังเบี่ยงเบนออกจากการเขียนของมนุษย์ที่เป็นธรรมชาติอย่างเห็นได้ชัด<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

รูปแบบที่ซ่อนเร้นนี้แสดงออกมาเป็นการผสมผสานของ:

- ความสามารถในการคาดเดาได้มากเกินไป
- จังหวะของวลีที่ซ้ำซากน่าเบื่อ
- สำนวนที่ปลอดภัยแต่อ่อนแอ
- ขาดความน่าประหลาดใจ ความละเอียดอ่อน และความลึกทางความหมาย<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

นั่นคือเหตุผลที่นิวรัลเทกซ์ดีเจเนอเรชันไม่ได้จำกัดอยู่แค่การวนซ้ำ แต่ครอบคลุมกลุ่มอาการที่กว้างกว่า ตั้งแต่การซ้ำที่ชัดเจนไปจนถึงความยากจนทางความหมายอย่างละเอียด ซึ่งข้อความยังคงถูกต้องในเชิงรูปแบบแต่สูญเสียความหลากหลาย การแสดงออก และคุณค่าในเชิงเนื้อหา<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

## สาเหตุ

### ความไม่ตรงกันระหว่างการฝึกและการ decoding

โมเดลภาษาโดยทั่วไปได้รับการฝึกเพื่อทำนาย token ถัดไปตามการแจกแจงความน่าจะเป็น อย่างไรก็ตาม ในระหว่างการสร้างข้อความ ข้อความมักถูกดึงออกมาไม่ใช่ในรูปแบบตัวอย่างจากการแจกแจงดังกล่าว แต่ผ่านขั้นตอนที่**เพิ่มค่าความน่าจะเป็นอย่างเข้มงวด**ในแต่ละขั้นตอนหรือตลอดทั้งลำดับ ความแตกต่างระหว่างเป้าหมายการฝึกและกฎการ decoding นี้เป็นหนึ่งในสาเหตุหลักของการเสื่อมสภาพในงานแบบเปิด<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

### «หางที่ไม่น่าเชื่อถือ» ของการแจกแจง

ในทางกลับกัน อิสระในการสุ่มตัวอย่างโดยสมบูรณ์ก็อาจเป็นอันตรายได้เช่นกัน แม้แต่ token ที่มีความน่าจะเป็นต่ำมากโดยทั่วไปก็มีความน่าจะเป็นไม่เป็นศูนย์ และในนั้นมีตัวเลือกที่อ่อนแอ มีสัญญาณรบกวน หรือไม่สอดคล้องกันจำนวนมาก หากโมเดลเลือก token จากหางดังกล่าว คุณภาพของข้อความต่อก็อาจลดลงอย่างรวดเร็ว<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

### การละเมิด local typicality

Meister และคณะเชื่อมโยงการเสื่อมสภาพกับข้อเท็จจริงที่ว่าข้อความที่เป็นธรรมชาติมักรักษาเนื้อหาข้อมูลให้ใกล้เคียงกับ conditional entropy ที่คาดหวัง เมื่อการ decoding เบี่ยงเบนไปจาก «ความเป็นปกติ» นี้อย่างเป็นระบบ ข้อความก็อาจกลายเป็นสิ่งที่ธรรมดาเกินไปหรือมีสัญญาณรบกวนมากเกินไป<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

## ความเชื่อมโยงกับวิธีการ decoding

นิวรัลเทกซ์ดีเจเนอเรชันเกี่ยวข้องอย่างใกล้ชิดไม่เพียงแค่กับคุณภาพของโมเดลภาษาเอง แต่ยังรวมถึง**วิธีการ**ที่ token ถัดไปถูกเลือกจากการแจกแจงความน่าจะเป็นของโมเดล ชุดความน่าจะเป็นชุดเดียวกันสามารถให้ผลลัพธ์ที่แตกต่างกันอย่างเห็นได้ชัดขึ้นอยู่กับกลยุทธ์การ decoding บางวิธีเสริมการซ้ำและความเป็นแม่แบบ บางวิธีเพิ่มความหลากหลายแต่เสี่ยงต่อการสูญเสียความเชื่อมโยง และบางวิธีพยายามรักษาสมดุลระหว่างสองขั้วนี้<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

### Greedy search และ beam search

Greedy search และ beam search มีแนวโน้มที่จะเกิดการซ้ำและความเป็นแม่แบบในงานการสร้างข้อความแบบเปิดเป็นพิเศษ เนื่องจากเลือกข้อความต่อที่มีความน่าจะเป็นสูงที่สุดอย่างเป็นระบบ<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ในกรณีของ**greedy search** โมเดลเลือก token ที่มีความน่าจะเป็นสูงที่สุดหนึ่งตัวในแต่ละขั้นตอน แนวทางนี้ทำงานได้ดีในที่ที่งานมีชุดคำตอบที่ถูกต้องค่อนข้างแคบ อย่างไรก็ตาม ในการสร้างข้อความอิสระมักทำให้ข้อความคาดเดาได้มากเกินไป โมเดลเลือกข้อความต่อที่ «ปลอดภัย» ที่สุดซ้ำแล้วซ้ำอีก ทำให้:

- แนวโน้มการซ้ำโครงสร้างที่เลือกไปแล้วแข็งแกร่งขึ้น
- ความหลากหลายของสำนวนลดลง
- ข้อความกลายเป็นแม่แบบและยากจนในเชิงเนื้อหา<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

**Beam search** ขยายพื้นที่การค้นหาบางส่วนเนื่องจากติดตามสมมติฐานที่มีความน่าจะเป็นสูงสุดหลายรายการพร้อมกัน อย่างไรก็ตาม ในงานข้อความแบบเปิดนี้ไม่ได้ป้องกันการเสื่อมสภาพเสมอไป ในทางตรงกันข้าม เนื่องจากมีแนวทางทั่วไปในการเพิ่มค่าความน่าจะเป็น มันยังอาจเลือกข้อความต่อที่ «ถูกต้อง» มากเกินไปแต่ไม่เป็นธรรมชาติอย่างเป็นระบบ ซึ่งเพิ่มความเสี่ยงของ:

- ความน่าเบื่อ
- การซ้ำ
- ข้อความที่เรียบเกินไปแต่มีเนื้อหาน้อย<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

กล่าวอีกนัยหนึ่ง วิธีการที่เลือกข้อความต่อที่มีความน่าจะเป็นสูงสุดอย่างสม่ำเสมอเกินไป มักชนะในด้านความคาดเดาได้ในระดับท้องถิ่น แต่แพ้ในด้านความเป็นธรรมชาติของการสร้างข้อความยาว<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

### การสุ่มตัวอย่างล้วนๆ

การสุ่มตัวอย่างที่ไม่มีการจำกัดเข้มงวดเพิ่มความหลากหลาย แต่หากไม่มีตัวกรองเพิ่มเติม มักจะจับ token ที่อ่อนแอจากส่วนหางของการแจกแจงได้บ่อยขึ้น ซึ่งอาจทำให้ความเชื่อมโยงของข้อความลดลง<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ใน**การสุ่มตัวอย่างล้วนๆ** token ถัดไปถูกเลือกแบบสุ่มตามการแจกแจงความน่าจะเป็นเต็มรูปแบบของโมเดล ต่างจาก greedy search หรือ beam search แนวทางนี้ให้อิสระแก่โมเดลมากขึ้นและช่วยหลีกเลี่ยงการ «ติดค้าง» ในแม่แบบเดิมเร็วเกินไป<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

อย่างไรก็ตาม อิสระนี้มีด้านลบ แม้ว่าความน่าจะเป็นของ token หลายตัวจะต่ำมาก แต่โดยทั่วไปยังคงมีโอกาสไม่เป็นศูนย์ที่จะถูกเลือก ด้วยเหตุนี้โมเดลอาจจับข้อความต่อจาก «หาง» ของการแจกแจง ซึ่งมักพบ:

- คำที่สุ่มและอ่อนแอในเชิงบริบท
- ข้อความต่อที่ไม่สอดคล้องกัน
- token ที่ยอมรับได้ในระดับท้องถิ่นแต่ลดทอนความเชื่อมโยงโดยรวมของข้อความ<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ผลลัพธ์คือการสุ่มตัวอย่างล้วนๆ มักลดความเป็นแม่แบบ แต่พร้อมกันนั้นก็ทำให้ข้อความไม่มั่นคงมากขึ้น ข้อความอาจยังคงหลากหลายแต่กลายเป็นสิ่งที่หลวม ไม่สม่ำเสมอ หรือไม่มั่นคงทางความหมาย<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

### วิธีการตัดส่วนหางของการแจกแจง

วิธีการตัดส่วนหางของการแจกแจง โดยเฉพาะ Top-p และ Top-k ถูกเสนอขึ้นเป็นวิธีปฏิบัติในการลดการเสื่อมสภาพ วิธีการเหล่านี้จำกัดชุด token ที่อนุญาตและลดความน่าจะเป็นของข้อความต่อที่ไม่ประสบความสำเร็จ<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

ตรรกะทั่วไปของวิธีการเหล่านี้คือไม่ให้โมเดลเลือก token ถัดไปจากคำศัพท์ทั้งหมด แต่พิจารณาเฉพาะส่วนที่มีความน่าจะเป็นสูงที่สุดของการแจกแจงเท่านั้น:

- **Top-k** เหลือเพียง token ที่มีความน่าจะเป็นสูงสุดจำนวนคงที่
- **Top-p** เหลือ «นิวเคลียส» แบบไดนามิกของ token ที่ความน่าจะเป็นรวมกันถึงเกณฑ์ที่กำหนด $p$<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ข้อจำกัดดังกล่าวลดความเสี่ยงที่โมเดลจะเลือกข้อความต่อที่อ่อนแอเกินไปจากหางที่ไม่น่าเชื่อถือ แต่ไม่ได้บังคับให้เลือกตัวเลือก «ดีที่สุด» ตัวเดียวเสมอไป ด้วยเหตุนี้วิธีการตัดส่วนหางมักเป็นทางออกระดับกลางระหว่างสองขั้ว:

- การเพิ่มค่าที่เข้มงวดเกินไป
- การสุ่มตัวอย่างที่อิสระและมีสัญญาณรบกวนมากเกินไป<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ต่อมา Finlayson และคณะเสนอการตีความเชิงทฤษฎีที่เข้มงวดกว่าเกี่ยวกับประสิทธิภาพนี้ พวกเขาแสดงให้เห็นว่า truncation method ช่วยทิ้ง token ที่มีแนวโน้มที่จะไม่น่าเชื่อถือจากมุมมองของการแจกแจงข้อความที่แท้จริง ดังนั้นการตัดส่วนหางสามารถมองได้ไม่เพียงแค่เป็นเทคนิค heuristic เชิงวิศวกรรม แต่ยังเป็นวิธีลดโอกาสของข้อผิดพลาดที่เป็นระบบในการเลือก token ถัดไปอีกด้วย<sup>[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

### กลยุทธ์สมัยใหม่

วิธีการที่ใหม่กว่า เช่น **locally typical sampling** มุ่งลดการซ้ำที่เสื่อมสภาพพร้อมรักษาคุณภาพการสร้างข้อความ<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup> ในทำนองเดียวกัน **contrastive search** ถูกเสนอขึ้นเป็นวิธีเพิ่มความหลากหลายโดยไม่เสียสละความเชื่อมโยงของข้อความ<sup>[\[4\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-contrastive2022-4)</sup>

วิธีการเหล่านี้พัฒนาแนวคิดที่ว่าปัญหาไม่ได้อยู่เพียงแค่ว่าควรเหลือ token **กี่ตัว** แต่ token **ใด**ที่ควรถือเป็นตัวเลือกที่ดี

**Locally typical sampling** ไม่ได้มุ่งแค่ความน่าจะเป็นสูง แต่มุ่งความใกล้ชิดของเนื้อหาข้อมูลของ token กับ conditional entropy ที่คาดหวัง กล่าวอีกนัยหนึ่ง วิธีการนี้พยายามไม่เพียงแค่เลือกคำที่มีความน่าจะเป็นสูงสุด แต่เลือกคำที่สอดคล้องกับระดับข้อมูลที่ «ปกติ» สำหรับบริบทนั้นดีกว่า ซึ่งช่วยลด:

- การซ้ำที่เสื่อมสภาพ
- ความเป็นแม่แบบมากเกินไป
- ความยากจนทางความหมายของข้อความพร้อมรักษาความเชื่อมโยงโดยรวม<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

**Contrastive search** ใช้ตรรกะที่แตกต่างออกไป โดยพยายามเลือกข้อความต่อที่ทั้งยังคงน่าจะเป็น และไม่คล้ายกับบริบทที่สร้างไปแล้วมากเกินไป ด้วยเหตุนี้จึงลดโอกาสที่โมเดลจะเสริมโครงสร้างเดิมซ้ำๆ เพียงเพราะมันมีความน่าจะเป็นสูงมากในระดับท้องถิ่น<sup>[\[4\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-contrastive2022-4)</sup>

เมื่อเทียบกับกลยุทธ์ decoding แบบดั้งเดิม วิธีการเหล่านี้ไม่เพียงแค่จำกัดการเลือก แต่พยายามทำให้เกณฑ์การเลือกเองไวต่อคุณภาพของการสร้างข้อความยาวมากขึ้น กล่าวคือ ว่าข้อความจะยังคงมีเนื้อหา หลากหลาย และเชื่อมโยงกันตลอดหลายประโยคหรือย่อหน้าหรือไม่<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)[\[4\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-contrastive2022-4)</sup>

### การพึ่งพาโหมดการ decoding

ความเชื่อมโยงระหว่างนิวรัลเทกซ์ดีเจเนอเรชันและการ decoding ไม่ได้จำกัดอยู่แค่การเปรียบเทียบวิธีการ «ดี» กับ «ไม่ดี» อัลกอริทึมเดียวกันอาจทำงานต่างกันขึ้นอยู่กับว่ามันจำกัดการแจกแจงในแต่ละขั้นตอนอย่างเข้มงวดหรืออิสระเพียงใด<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

โดยทั่วไปสามารถแบ่งได้เป็นสามโหมด:

- **การเลือกที่เข้มงวดเกินไป** — โมเดลเกือบจะเลือก token ที่มีความน่าจะเป็นสูงสุดเสมอ ซึ่งเสริมความเป็นแม่แบบและการซ้ำ
- **การเลือกที่อิสระเกินไป** — โมเดลได้รับการเข้าถึงข้อความต่อที่อ่อนแอในวงกว้างเกินไป ซึ่งเพิ่มความเสี่ยงของความไม่เชื่อมโยง
- **การเลือกที่จำกัดพอดี** — โมเดลรักษาความหลากหลายแต่อยู่ในขอบเขตของส่วนที่น่าเชื่อถือพอสมควรของการแจกแจง<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

นั่นคือเหตุผลที่นิวรัลเทกซ์ดีเจเนอเรชันมักถูกมองว่าเป็นผลลัพธ์ไม่เพียงแค่ของคุณภาพของโมเดลเอง แต่ยังรวมถึง**โหมดการ decoding ที่ไม่เหมาะสม** ไม่ว่าจะเข้มงวดเกินไป อ่อนเกินไป หรือไม่สอดคล้องกับงานการสร้างข้อความเฉพาะ<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

**พูดง่ายๆ** การเสื่อมสภาพมักเกิดขึ้นที่ขั้วสองด้าน เมื่อโมเดลเลือกเฉพาะคำที่มีความน่าจะเป็นสูงสุดอย่างเข้มงวดเกินไป หรือในทางตรงกันข้าม ได้รับอิสระมากเกินไปและเริ่มดึงข้อความต่อที่สุ่มและอ่อนแอ กลยุทธ์ที่มั่นคงที่สุดมักอยู่ระหว่างโหมดขั้วสองด้านนี้<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

## วิธีการบรรเทา

แม้ว่าจะไม่มีวิธีสากลในการกำจัดนิวรัลเทกซ์ดีเจเนอเรชันอย่างสมบูรณ์ แต่ในทางปฏิบัติได้มีการพัฒนาวิธีการต่างๆ ที่ช่วยลดการแสดงออกที่พบบ่อยที่สุดได้อย่างเห็นได้ชัด โดยเฉพาะการซ้ำ ความเป็นแม่แบบ การสูญเสียความเชื่อมโยง และการจับ «หางที่มีสัญญาณรบกวน» ของการแจกแจง วิธีการเหล่านี้แตกต่างกันในด้าน**วิธีการ**จำกัดการเลือก token ถัดไป บางวิธีตัดตัวเลือกที่อ่อนแอออกอย่างง่าย บางวิธีคำนึงถึงรูปแบบของการแจกแจงอย่างละเอียดกว่า และบางวิธีพยายามสร้างสมดุลระหว่างความเชื่อมโยงและความหลากหลายโดยตรง<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)[\[5\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shi2024-5)</sup>

### การตัดส่วนหางของการแจกแจง

วิธีที่พบบ่อยที่สุดในการต่อสู้กับนิวรัลเทกซ์ดีเจเนอเรชันคือการจำกัดการเลือก token ถัดไปไม่ให้ใช้คำศัพท์ทั้งหมด แต่เฉพาะส่วนที่น่าจะเป็นมากที่สุดเท่านั้น นั่นคือพื้นฐานของ Top-p และ Top-k<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

แนวคิดหลักของวิธีการเหล่านี้คือโมเดลไม่ควรเลือกอย่างอิสระจาก token ทุกตัวที่เป็นไปได้ เนื่องจากส่วนสำคัญของการแจกแจงอยู่ใน «หางที่ไม่น่าเชื่อถือ» ซึ่งมีตัวเลือกที่อ่อนแอ มีสัญญาณรบกวน หรือไม่สอดคล้องกัน การตัดส่วนหางช่วย**จำกัดพื้นที่การเลือกให้แคบลงโดยเทียม**และลดโอกาสที่การสร้างข้อความจะเบี่ยงเบนไปสู่ความไม่เชื่อมโยงหรือเริ่มแตกสลาย<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

ในทางปฏิบัติมีสองรูปแบบที่เป็นที่รู้จักมากที่สุด:

- **Top-k** — เหลือเพียง token ที่มีความน่าจะเป็นสูงสุดจำนวนคงที่
- **Top-p** (*nucleus sampling*) — เหลือ «นิวเคลียส» แบบไดนามิกของ token ที่ความน่าจะเป็นรวมกันถึงเกณฑ์ที่กำหนด $p$<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

ความหมายโดยรวมของวิธีการทั้งสองคือเดียวกัน ไม่ให้โมเดลเลือกจากข้อความต่อที่อ่อนแอมากเกินไป ทั้งนี้ Top-p มักถือว่ามีความยืดหยุ่นมากกว่า เนื่องจากขนาดของชุดที่อนุญาตเปลี่ยนแปลงจากขั้นตอนหนึ่งไปอีกขั้นตอนขึ้นอยู่กับรูปแบบปัจจุบันของการแจกแจง<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup>

Finlayson และคณะเสนอการตีความที่เข้มงวดกว่าว่าเหตุใด truncation method จึงมักได้ผล โดยการตัดส่วนหางช่วยทิ้ง token ที่อาจไม่เป็นของพื้นที่รองรับของการแจกแจงข้อความ «ที่แท้จริง» กล่าวคือเป็น token ที่ไม่น่าเชื่อถือเป็นพิเศษจากมุมมองของโมเดล ดังนั้นวิธีการดังกล่าวไม่เพียงแค่ «ตัดหาง» ในเชิงกลไก แต่ยังลดโอกาสของข้อผิดพลาดที่เป็นระบบในขั้นตอนการเลือก token อีกด้วย<sup>[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

**พูดง่ายๆ** การตัดส่วนหางของการแจกแจงช่วยให้โมเดลไม่เดินไกลเกินไปในพื้นที่ของตัวเลือกที่อ่อนแอและสุ่ม ด้วยเหตุนี้ข้อความมักยังคงเชื่อมโยง เป็นธรรมชาติ และมีแนวโน้มเสื่อมสภาพน้อยลง

### วิธีการสุ่มตัวอย่างแบบปรับตัวมากขึ้น

Locally typical sampling, η-sampling และแนวทางที่เกี่ยวข้องพยายามไม่เพียงแค่ «ตัดหาง» แต่คำนึงถึงรูปแบบของการแจกแจงและโครงสร้างข้อมูลของขั้นตอนการสร้างข้อความปัจจุบัน<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

ต่างจากการตัดส่วนหางอย่างง่าย วิธีการดังกล่าวพยายามคำนึงถึงไม่เพียงแค่ว่า token นั้น**น่าจะเป็นเพียงใด** แต่ยังว่า**มันเป็น token ที่ปกติเพียงใด**สำหรับบริบทนั้นด้วย สิ่งนี้สำคัญเพราะการเสื่อมสภาพไม่ได้เกิดขึ้นเฉพาะจาก token ที่อ่อนแอเกินไป แต่ยังเกิดจากการเลือก token ที่ «ปลอดภัย» และธรรมดามากเกินไป<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

ตัวอย่างเช่น **locally typical sampling** มุ่งความใกล้ชิดของเนื้อหาข้อมูลของ token กับ conditional entropy ที่คาดหวัง กล่าวอีกนัยหนึ่ง วิธีการนี้พยายามไม่เพียงแค่เลือกคำที่มีความน่าจะเป็นสูงสุด แต่เลือกคำที่สอดคล้องกับระดับข้อมูลที่ «ปกติ» สำหรับบริบทนั้นดีกว่า ซึ่งช่วยลด:

- การซ้ำที่เสื่อมสภาพ
- ความเป็นแม่แบบมากเกินไป
- ความยากจนทางความหมายของข้อความพร้อมรักษาความเชื่อมโยงโดยรวม<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

วิธีการปรับตัวอื่นๆ ใช้ตรรกะที่คล้ายกัน แทนที่จะตัดตายตัวอย่างง่าย วิธีการเหล่านี้พยายามปรับตัวให้เข้ากับสถานะปัจจุบันของการแจกแจง ด้วยเหตุนี้โมเดลมีแนวโน้มน้อยกว่าที่จะตกอยู่ในสองขั้ว:

- กลายเป็นสิ่งที่คาดเดาได้และ «แบน» มากเกินไป
- จับข้อความต่อที่อ่อนแอและมีสัญญาณรบกวนมากเกินไป<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)</sup>

วิธีการดังกล่าวมักซับซ้อนกว่าในการตีความและปรับแต่งเมื่อเทียบกับ Top-k หรือ Top-p แต่สามารถจัดการกับรูปแบบที่ละเอียดอ่อนของการเสื่อมสภาพได้ดีกว่า ซึ่งปัญหาไม่ได้อยู่ที่ความไม่เชื่อมโยงที่ชัดเจน แต่อยู่ที่การสูญเสียความมีข้อมูลและการซ้ำของแม่แบบการพูด<sup>[\[3\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-meister2023-3)[\[5\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shi2024-5)</sup>

**พูดง่ายๆ** วิธีการสุ่มตัวอย่างแบบปรับตัวมากขึ้นพยายามไม่เพียงแค่ลบคำที่อ่อนแอ แต่เก็บข้อความต่อที่เหมาะกับบริบทปัจจุบันในด้านความหมายและ «ความเป็นธรรมชาติ» ดีกว่า

### Contrastive search

Contrastive search และวิธีการที่เกี่ยวข้องพยายามรักษาทั้งความเชื่อมโยงและความหลากหลายพร้อมกัน โดยลดการซ้ำที่มักเกิดขึ้นกับ decoding ที่อิงการเพิ่มค่าอย่างเข้มงวด<sup>[\[4\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-contrastive2022-4)</sup>

แนวคิดของแนวทาง contrastive คือไม่พึ่งพาแค่ความน่าจะเป็นของ token ถัดไป แต่การเลือกตัวเลือกเป็นการประนีประนอมระหว่างสองข้อกำหนด:

- ข้อความต่อต้องน่าจะเป็นพอสมควร
- ข้อความต่อต้องไม่คล้ายกับบริบทที่สร้างไปแล้วมากเกินไปและเสริมการซ้ำ<sup>[\[4\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-contrastive2022-4)</sup>

ด้วยเหตุนี้ contrastive search พยายามยับยั้งข้อความต่อที่ดูน่าจะเป็นมากในเชิงรูปแบบ แต่ในทางปฏิบัตินำไปสู่การวนซ้ำ ความเป็นแม่แบบ หรือการสร้างข้อความที่ซ้ำซากเกินไป ต่างจากการสุ่มตัวอย่างล้วนๆ แนวทางนี้ไม่ใช้ความสุ่มเป็นกลไกหลักในการต่อสู้กับการเสื่อมสภาพ แต่พยายามเลือก**ข้อความต่อที่น่าจะเป็นพอสมควรแต่เสื่อมสภาพน้อยกว่า**อย่างตั้งใจ<sup>[\[4\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-contrastive2022-4)</sup>

วิธีการดังกล่าวมีความสำคัญเป็นพิเศษเพราะกลยุทธ์ที่อิงการเพิ่มค่าแบบดั้งเดิมมักให้ความมั่นใจในระดับท้องถิ่นสูง แต่แลกมาด้วยความเป็นธรรมชาติของข้อความ Contrastive search พยายามรักษาจุดแข็งของการเลือกแบบ deterministic ซึ่งได้แก่ความเชื่อมโยงและความมั่นคง พร้อมกับลดการซ้ำและความน่าเบื่อ<sup>[\[4\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-contrastive2022-4)</sup>

**พูดง่ายๆ** contrastive search พยายามไม่เลือกข้อความต่อเพียงเพราะมันมีความน่าจะเป็นสูงสุด แต่ยัง «ลงโทษ» ตัวเลือกที่ซ้ำซากมากเกินไปเพื่อไม่ให้ข้อความวนซ้ำหรือกลายเป็นแม่แบบมากเกินไป

### การเลือกกลยุทธ์ให้เหมาะกับงาน

งานวิจัยในภายหลังเน้นย้ำว่าไม่มีวิธีสากล ประสิทธิภาพของกลยุทธ์ decoding เฉพาะขึ้นอยู่กับประเภทของงาน ขนาดของโมเดล การ alignment และปัจจัยอื่นๆ สิ่งที่ทำงานได้ดีสำหรับการสร้างข้อความบทสนทนาหรือบทความอาจทำงานได้แย่กว่าในโค้ด คณิตศาสตร์ หรือโดเมน deterministic อื่นๆ<sup>[\[5\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shi2024-5)</sup>

ซึ่งหมายความว่าการต่อสู้กับนิวรัลเทกซ์ดีเจเนอเรชันไม่ใช่เพียงการเลือกอัลกอริทึม «ดีที่สุด» แต่ยัง**จับคู่วิธีการกับประเภทของงาน**ด้วย ในการสร้างข้อความแบบเปิด (เรื่องราว บทสนทนา คำตอบยาว) วิธีการสุ่มตัวอย่างและรูปแบบการคัดเลือกที่ยืดหยุ่นกว่ามักช่วยรักษาความหลากหลายและความเป็นธรรมชาติ ในงานที่เข้มงวดและ deterministic กว่า เช่น การเขียนโปรแกรม คณิตศาสตร์ หรืองานที่มีคำตอบที่ถูกต้องชัดเจน ความสุ่มที่ก้าวร้าวเกินไปอาจลดคุณภาพของผลลัพธ์ได้ในทางตรงกันข้าม<sup>[\[5\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shi2024-5)</sup>

Shi และคณะยังแสดงให้เห็นว่าผลลัพธ์สุดท้ายได้รับอิทธิพลไม่เพียงแค่จากกลยุทธ์การ decoding เอง แต่ยังจาก:

- ขนาดของโมเดล
- ระดับการ alignment
- ความไวต่อ hyperparameter
- สภาพแวดล้อมการ deploy และข้อจำกัดของ inference<sup>[\[5\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shi2024-5)</sup>

จากนี้มีข้อสรุปทางปฏิบัติที่สำคัญ วิธีการเดียวกันอาจดูแข็งแกร่งในการกำหนดค่าหนึ่งและอ่อนแอกว่าอย่างเห็นได้ชัดในอีกการกำหนดค่าหนึ่ง ดังนั้นการบรรเทานิวรัลเทกซ์ดีเจเนอเรชันในทางปฏิบัติแทบจะต้องปรับ parameter และตรวจสอบกับประเภทของงานเฉพาะเสมอ มากกว่าการพึ่งพา «สูตรสากล» ตัวเดียว<sup>[\[5\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shi2024-5)</sup>

**พูดง่ายๆ** วิธีการ decoding ที่ดีขึ้นอยู่กับว่าโมเดลกำลังทำอะไรอยู่ สำหรับข้อความอิสระวิธีการบางอย่างทำงานได้ดีกว่า สำหรับโค้ด คณิตศาสตร์ และงานที่แม่นยำอื่นๆ จะเป็นวิธีการอื่น

### การรวมวิธีการต่างๆ เข้าด้วยกัน

ในทางปฏิบัติ วิธีการบรรเทามักถูกนำมาใช้ไม่แยกกัน แต่ในการผสมผสานกัน ตัวอย่างเช่น การตัดส่วนหางของการแจกแจงอาจใช้ร่วมกับอุณหภูมิ และกลยุทธ์ที่ซับซ้อนกว่าอาจรวมกับข้อจำกัดเพิ่มเติมในการเลือก token<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[5\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shi2024-5)</sup>

ความหมายของการผสมผสานดังกล่าวคือการแยกหน้าที่ออกจากกัน:

- กลไกหนึ่งกำหนดระดับความหลากหลายโดยรวม
- อีกกลไกหนึ่งจำกัด «หางที่มีสัญญาณรบกวน»
- กลไกที่สามลดความเสี่ยงของการซ้ำหรือความเป็นแม่แบบ<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[4\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-contrastive2022-4)</sup>

อย่างไรก็ตาม การเพิ่มจำนวนกลไกที่ต้องปรับพร้อมกันทำให้การควบคุมพฤติกรรมของโมเดลซับซ้อนขึ้น ดังนั้นรูปแบบที่ซับซ้อนกว่าไม่ได้หมายถึงผลลัพธ์ที่ดีกว่าเสมอไป ผลกำไรมักขึ้นอยู่กับว่า hyperparameter ถูกปรับดีเพียงใดและการผสมผสานเฉพาะสอดคล้องกับงานเพียงใด<sup>[\[5\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shi2024-5)</sup>

**กล่าวอีกนัยหนึ่ง** การบรรเทานิวรัลเทกซ์ดีเจเนอเรชันส่วนใหญ่สร้างขึ้นจากการผสมผสานข้อจำกัดหลายอย่างอย่างระมัดระวัง ไม่ใช่วิธีการ «วิเศษ» วิธีเดียว โดยแต่ละข้อจำกัดกำจัดปัญหาของตัวเอง

## ไม่ควรสับสนกับ model collapse

นิวรัลเทกซ์ดีเจเนอเรชันในความหมายของ Holtzman และคณะไม่ควรสับสนกับ **model collapse** ซึ่งเป็นปรากฏการณ์แยกต่างหากที่ศึกษาในงานวิจัยเกี่ยวกับการฝึกโมเดลซ้ำๆ บนข้อมูลสังเคราะห์

ในกรณีแรกพูดถึงการเสื่อมสภาพของข้อความ**ในขั้นตอนการสร้าง** (inference) เป็นหลัก เมื่อกลยุทธ์การ decoding ที่ไม่เหมาะสมนำไปสู่การซ้ำ ความเป็นแม่แบบ หรือการสูญเสียความเชื่อมโยง<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)</sup> ในกรณีที่สองพูดถึงการเสื่อมสภาพของโมเดล**ในขั้นตอนการฝึก** เมื่อข้อมูลที่สร้างโดยโมเดลเริ่ม «ปนเปื้อน» ชุดข้อมูลการฝึกของโมเดลรุ่นต่อไป ซึ่งอาจนำไปสู่การหายไปของเหตุการณ์ที่หายากและการแคบลงของการแจกแจง<sup>[\[6\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shumailov2024-6)</sup>

ปรากฏการณ์เหล่านี้มีความเชื่อมโยงกันในเชิงหัวข้อ เนื่องจากทั้งสองเกี่ยวข้องกับการเสื่อมลงของคุณภาพข้อความของโครงข่ายประสาทเทียม อย่างไรก็ตามในวรรณกรรมมักถูกพิจารณาเป็นปัญหาที่แตกต่างกัน ปัญหาหนึ่งเกี่ยวข้องกับการ decoding อีกปัญหาหนึ่งเกี่ยวข้องกับข้อมูลและการฝึก<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[6\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shumailov2024-6)</sup>

## ความสำคัญสำหรับโมเดลภาษาขนาดใหญ่

ในยุคของโมเดลภาษาขนาดใหญ่ ปัญหานิวรัลเทกซ์ดีเจเนอเรชันไม่ได้หายไป แต่เลื่อนจากพื้นที่ของตัวสร้างข้อความขนาดเล็กมาสู่พื้นที่การจัดการพฤติกรรมของระบบที่มีประสิทธิภาพมากขึ้น แม้ว่า LLM สมัยใหม่มักมีการ alignment ที่ดีกว่าและตกอยู่ในวงซ้ำอย่างรุนแรงน้อยกว่า แต่การเลือกกลยุทธ์ decoding ยังคงส่งผลต่อความเป็นธรรมชาติ ความหลากหลาย และความมั่นคงของผลลัพธ์อย่างเห็นได้ชัด<sup>[\[5\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-shi2024-5)</sup>

ด้วยเหตุนี้นิวรัลเทกซ์ดีเจเนอเรชันจึงถูกมองไม่ใช่เป็นข้อบกพร่องแคบๆ ของโมเดลภาษายุคแรก แต่เป็นปัญหาพื้นฐานของการสร้างข้อความในทางปฏิบัติ กล่าวคือ การสร้างแบบจำลองความน่าจะเป็นที่ดีไม่ได้รับประกันข้อความที่ดีในขั้นตอน inference เสมอไป<sup>[\[1\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-holtzman2019-1)[\[2\]](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_note-finlayson2024-2)</sup>

## ดูเพิ่มเติม

- การสุ่มตัวอย่างแบบ Top-p
- การสุ่มตัวอย่างแบบ Top-k
- อุณหภูมิ
- โมเดลภาษาขนาดใหญ่

## เอกสารอ้างอิง

- Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019; ตีพิมพ์ใน ICLR 2020). *The Curious Case of Neural Text Degeneration*. arXiv:1904.09751.
- Finlayson, M., Hewitt, J., Koller, A., Swayamdipta, S., & Sabharwal, A. (2024). *Closing the Curious Case of Neural Text Degeneration*. arXiv:2310.01693.
- Meister, C., Pimentel, T., Wiher, G., & Cotterell, R. (2023). *Locally Typical Sampling*. TACL 2023.
- Shi, C., Yang, H., Cai, D., Zhang, Z., Wang, Y., Yang, Y., & Lam, W. (2024). *A Thorough Examination of Decoding Methods in the Era of LLMs*. EMNLP 2024.
- Su, Y., Lan, T., Wang, Y., Yogatama, D., Kong, L., & Collier, N. (2022). *A Contrastive Framework for Neural Text Generation*. arXiv:2202.06417.
- Shumailov, I. et al. (2024). *AI models collapse when trained on recursively generated data*. Nature 2024.

## หมายเหตุ

1.  <span id="cite_note-holtzman2019-1">↑ <sup>[1.00](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-0)</sup> <sup>[1.01](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-1)</sup> <sup>[1.02](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-2)</sup> <sup>[1.03](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-3)</sup> <sup>[1.04](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-4)</sup> <sup>[1.05](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-5)</sup> <sup>[1.06](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-6)</sup> <sup>[1.07](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-7)</sup> <sup>[1.08](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-8)</sup> <sup>[1.09](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-9)</sup> <sup>[1.10](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-10)</sup> <sup>[1.11](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-11)</sup> <sup>[1.12](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-12)</sup> <sup>[1.13](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-13)</sup> <sup>[1.14](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-14)</sup> <sup>[1.15](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-15)</sup> <sup>[1.16](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-16)</sup> <sup>[1.17](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-17)</sup> <sup>[1.18](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-18)</sup> <sup>[1.19](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-19)</sup> <sup>[1.20](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-20)</sup> <sup>[1.21](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-21)</sup> <sup>[1.22](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-22)</sup> <sup>[1.23](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-23)</sup> <sup>[1.24](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-24)</sup> <sup>[1.25](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-25)</sup> <sup>[1.26](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-26)</sup> <sup>[1.27](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-27)</sup> <sup>[1.28](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-28)</sup> <sup>[1.29](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-29)</sup> <sup>[1.30](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-30)</sup> <sup>[1.31](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-31)</sup> <sup>[1.32](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-32)</sup> <sup>[1.33](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-33)</sup> <sup>[1.34](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-34)</sup> <sup>[1.35](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-35)</sup> <sup>[1.36](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-36)</sup> <sup>[1.37](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-37)</sup> <sup>[1.38](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-38)</sup> <sup>[1.39](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-39)</sup> <sup>[1.40](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-40)</sup> <sup>[1.41](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-41)</sup> <sup>[1.42](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-42)</sup> <sup>[1.43](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-43)</sup> <sup>[1.44](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-44)</sup> <sup>[1.45](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-45)</sup> <sup>[1.46](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-46)</sup> <sup>[1.47](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-holtzman2019_1-47)</sup> Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019). *The Curious Case of Neural Text Degeneration*. arXiv:1904.09751. <a href="https://arxiv.org/abs/1904.09751" class="external autonumber" rel="nofollow">[1]</a></span>
2.  <span id="cite_note-finlayson2024-2">↑ <sup>[2.00](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-0)</sup> <sup>[2.01](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-1)</sup> <sup>[2.02](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-2)</sup> <sup>[2.03](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-3)</sup> <sup>[2.04](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-4)</sup> <sup>[2.05](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-5)</sup> <sup>[2.06](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-6)</sup> <sup>[2.07](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-7)</sup> <sup>[2.08](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-8)</sup> <sup>[2.09](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-9)</sup> <sup>[2.10](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-10)</sup> <sup>[2.11](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-finlayson2024_2-11)</sup> Finlayson, M., Hewitt, J., Koller, A., Swayamdipta, S., & Sabharwal, A. (2024). *Closing the Curious Case of Neural Text Degeneration*. arXiv:2310.01693. <a href="https://arxiv.org/abs/2310.01693" class="external autonumber" rel="nofollow">[2]</a></span>
3.  <span id="cite_note-meister2023-3">↑ <sup>[3.00](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-0)</sup> <sup>[3.01](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-1)</sup> <sup>[3.02](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-2)</sup> <sup>[3.03](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-3)</sup> <sup>[3.04](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-4)</sup> <sup>[3.05](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-5)</sup> <sup>[3.06](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-6)</sup> <sup>[3.07](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-7)</sup> <sup>[3.08](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-8)</sup> <sup>[3.09](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-9)</sup> <sup>[3.10](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-10)</sup> <sup>[3.11](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-11)</sup> <sup>[3.12](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-12)</sup> <sup>[3.13](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-13)</sup> <sup>[3.14](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-14)</sup> <sup>[3.15](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-15)</sup> <sup>[3.16](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-16)</sup> <sup>[3.17](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-17)</sup> <sup>[3.18](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-18)</sup> <sup>[3.19](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-meister2023_3-19)</sup> Meister, C., Pimentel, T., Wiher, G., & Cotterell, R. (2023). *Locally Typical Sampling*. *Transactions of the Association for Computational Linguistics*, 11. <a href="https://aclanthology.org/2023.tacl-1.7/" class="external autonumber" rel="nofollow">[3]</a></span>
4.  <span id="cite_note-contrastive2022-4">↑ <sup>[4.0](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-contrastive2022_4-0)</sup> <sup>[4.1](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-contrastive2022_4-1)</sup> <sup>[4.2](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-contrastive2022_4-2)</sup> <sup>[4.3](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-contrastive2022_4-3)</sup> <sup>[4.4](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-contrastive2022_4-4)</sup> <sup>[4.5](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-contrastive2022_4-5)</sup> <sup>[4.6](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-contrastive2022_4-6)</sup> <sup>[4.7](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-contrastive2022_4-7)</sup> Su, Y., Lan, T., Wang, Y., Yogatama, D., Kong, L., & Collier, N. (2022). *A Contrastive Framework for Neural Text Generation*. arXiv:2202.06417. <a href="https://arxiv.org/abs/2202.06417" class="external autonumber" rel="nofollow">[4]</a></span>
5.  <span id="cite_note-shi2024-5">↑ <sup>[5.0](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shi2024_5-0)</sup> <sup>[5.1](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shi2024_5-1)</sup> <sup>[5.2](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shi2024_5-2)</sup> <sup>[5.3](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shi2024_5-3)</sup> <sup>[5.4](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shi2024_5-4)</sup> <sup>[5.5](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shi2024_5-5)</sup> <sup>[5.6](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shi2024_5-6)</sup> <sup>[5.7](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shi2024_5-7)</sup> <sup>[5.8](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shi2024_5-8)</sup> Shi, C., Yang, H., Cai, D., Zhang, Z., Wang, Y., Yang, Y., & Lam, W. (2024). *A Thorough Examination of Decoding Methods in the Era of LLMs*. EMNLP 2024. <a href="https://aclanthology.org/2024.emnlp-main.489/" class="external autonumber" rel="nofollow">[5]</a></span>
6.  <span id="cite_note-shumailov2024-6">↑ <sup>[6.0](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shumailov2024_6-0)</sup> <sup>[6.1](https://systems-analysis.info/int/Neural_Text_Degeneration_%E2%80%94_%E0%B8%99%E0%B8%B4%E0%B8%A7%E0%B8%A3%E0%B8%B1%E0%B8%A5%E0%B9%80%E0%B8%97%E0%B8%81%E0%B8%8B%E0%B9%8C%E0%B8%94%E0%B8%B5%E0%B9%80%E0%B8%88%E0%B9%80%E0%B8%99%E0%B8%AD%E0%B9%80%E0%B8%A3%E0%B8%8A%E0%B8%B1%E0%B8%99_(%E0%B8%81%E0%B8%B2%E0%B8%A3%E0%B9%80%E0%B8%AA%E0%B8%B7%E0%B9%88%E0%B8%AD%E0%B8%A1%E0%B8%AA%E0%B8%A0%E0%B8%B2%E0%B8%9E%E0%B8%82%E0%B8%AD%E0%B8%87%E0%B8%82%E0%B9%89%E0%B8%AD%E0%B8%84%E0%B8%A7%E0%B8%B2%E0%B8%A1%E0%B9%83%E0%B8%99%E0%B9%82%E0%B8%A1%E0%B9%80%E0%B8%94%E0%B8%A5%E0%B8%A0%E0%B8%B2%E0%B8%A9%E0%B8%B2%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B8%AA%E0%B8%B2%E0%B8%97%E0%B9%80%E0%B8%97%E0%B8%B5%E0%B8%A2%E0%B8%A1)#cite_ref-shumailov2024_6-1)</sup> Shumailov, I. et al. (2024). *AI models collapse when trained on recursively generated data*. *Nature*. <a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC11269175/" class="external autonumber" rel="nofollow">[6]</a></span>