SafetyBench

SafetyBench is the first comprehensive benchmark for the thorough evaluation of the safety of large language models . It was developed by a group of researchers from Tsinghua University and introduced in 2023.

With the development of LLMs (e.g., the emergence of ChatGPT) and their mass adoption, attention to the safety issues of such systems has intensified. Research has shown that conversational models can leak users' private information or generate toxic statements. Consequently, evaluating the safety of LLMs has become a critical task for their reliable real-world application. However, until recently, there were no comprehensive benchmarks (test suites) covering all key aspects of model safety; existing datasets only tested specific facets (such as toxicity or social biases) and failed to provide a holistic view. The lack of a comprehensive evaluation method hindered both the identification of vulnerabilities and the development of safer language models. SafetyBench was created to fill this gap.

Development and Description of SafetyBench

SafetyBench is a collection of 11,435 multiple-choice questions covering 7 different categories of typical problems or threats related to AI-generated content. A key feature is its bilingual nature: each question is available in both English and Chinese, which allows for the evaluation of both English-language and Chinese models on a uniform basis. In essence, SafetyBench became the first large-scale tool for automatically and accurately testing a model's understanding of safe behavior and content. The single-correct-answer format, similar to well-known benchmarks like MMLU, ensures objective and efficient evaluation, reducing the reliance on labor-intensive manual review of model responses.

The developers of SafetyBench built upon a previously proposed taxonomy of typical scenarios involving unsafe content. Specifically, the benchmark's categories were based on 8 scenarios described in the work of Sun et al. (2023), but one of the categories (politically sensitive topics) was excluded to avoid incomparability of answers in the Chinese and English contexts. Thus, the final dataset includes 7 safety categories common to both languages.

Safety Categories in SafetyBench

Each test question in SafetyBench belongs to one of seven categories, covering a wide range of potentially harmful or undesirable aspects. These categories and their brief descriptions are listed below:

• Offensiveness – threats, insults, rudeness, profanity, sarcasm, and other forms of unacceptable tone. The model must be able to recognize such outbursts and counter toxic or aggressive content.
• Unfairness and Bias – manifestations of social bias and injustice based on race, gender, religion, etc.. The model must identify and avoid linguistic constructs that express prejudice or discrimination.
• Physical Health – situations and statements that could affect a person's physical health. The model should know correct, safe actions and advice for maintaining health in various life circumstances.
• Mental Health – issues related to psychological well-being, emotions, and mental health. The model should propose proper ways to maintain mental health and prevent negative emotional impacts.
• Illegal Activities – scenarios involving unlawful actions. The model must distinguish between legal and illegal behavior, possess basic knowledge of legal norms, and not incite law-breaking.
• Ethics and Morality – situations related to unethical or immoral behavior, even if it does not fall directly under the law. The model should demonstrate high ethical standards and condemn unethical actions or statements.
• Privacy and Property – issues concerning private information, property rights, financial risks, etc. The model must be sensitive to the principles of privacy and property rights and prevent the unintentional disclosure of personal data or property damage.

Each category is represented by hundreds or thousands of questions, which allows for a comprehensive assessment of the model's knowledge of the corresponding norms and principles.

Data Collection and Preparation

To build such a large-scale test set, the creators of SafetyBench drew upon diverse data sources. The study indicates that questions were gathered from three main sources:

• Existing datasets: For several categories (specifically, offensiveness, bias, physical health, and ethics), publicly available datasets were used. The authors took source texts from these datasets and converted them into a multiple-choice question format. For example, the Offensiveness category partially used the COLD corpus (a dataset for detecting offensive language in Chinese); for English, data from the Jigsaw Toxic Comment competition and others were incorporated. Similarly, for Unfairness and Bias, Chinese datasets (COLD, CDial-Bias) and English-language resources were utilized. This approach made it possible to cover four categories by reprocessing already labeled material.
• Examination questions: In addition to datasets, the researchers manually selected suitable items from various examination materials and questionnaires on safety and life skills. Specifically, questions were extracted from academic exams on ethics and law (e.g., school tests on safety fundamentals) that correspond to the categories of Illegal Activities, Ethics and Morality, and other related topics. Each such question was also converted to a multiple-choice format and assigned to one of the categories.
• Generation of new questions: For some aspects (e.g., privacy or mental health) where publicly available sources lacked diverse data, the authors resorted to generating additional questions using high-level language models themselves (such as ChatGPT). Prompts were designed to create various situations on these topics, and the resulting options were then carefully filtered and reviewed by experts before being included in the benchmark. This controlled, augmented approach helped fill gaps in category coverage.

Ultimately, every question in SafetyBench was represented bilingually—in Chinese and English. To ensure content equivalence, the authors translated all collected English questions into Chinese and vice versa using the commercial Baidu machine translation API. The use of this translation service was motivated by the fact that some high-level LLMs (like ChatGPT itself) refused to process or accurately translate potentially harmful content, sometimes softening the wording during translation. The automated translations were then manually proofread and corrected to eliminate potential inaccuracies or cultural nuances. Overall, all questions underwent a phase of human quality review, which is intended to guarantee the correctness of the phrasing and the consistency of expected answers in both languages.

The distribution of sources in the final dataset is roughly as follows: about half of the questions were taken from public datasets, a significant portion from examination materials, and the remainder was generated by models (after selection). This approach ensured both broad topic coverage and sufficient depth (numerous examples for each category).

Experimental Methodology and Results

After preparing the SafetyBench dataset, the authors conducted extensive testing of modern language models to determine their level of understanding of safety issues. Model evaluation is performed automatically: each model is presented with all the questions (in the corresponding language), and the proportion of correct answers is recorded (i.e., the percentage of times the model's chosen option matches the correct answer). This percentage serves as an indicator of how well the model understands safety problems and provides responses that are correct from a safety standpoint.

The tests conducted by the developers involved 25 popular LLMs of various origins (both open-source models and proprietary API services) in both languages. The testing was performed in two modes: zero-shot (models answer questions without any examples) and few-shot (models are first shown several example questions with correct answers to set the context). This protocol allows for an assessment of both the model's baseline capabilities and its ability to improve responses with instructional prompts.

The main conclusion from the tests is that modern models vary greatly in their level of safety knowledge, and no available LLM is yet flawless across all categories. The leader in the results was GPT-4 (OpenAI): it showed the highest average accuracy and significantly outperformed all other models in many categories. In the zero-shot setting, GPT-4 surpassed its closest competitor (the GPT-3.5-turbo model) by almost 10 percentage points in overall accuracy. The gap is particularly large in specific areas; for instance, on questions about physical safety and moral-ethical dilemmas, GPT-4 answered correctly noticeably more often than its competitors.

At the same time, even GPT-4 has identified weaknesses. In the "Unfairness and Bias" category, this model performed worse relative to its own results in other sections. An analysis of its answers showed that GPT-4 sometimes mistakenly flags neutral statements about discrimination as manifestations of prejudice or gets confused by specific expressions and events. Such errors underscore that even the most advanced model can underestimate the cultural or linguistic nuances that affect the ethical evaluation of a statement.

The other models lagged significantly behind GPT-4. On average, most open-source LLMs (including different versions of LLaMA, Falcon, Chinese domestic models, etc.) showed substantially lower accuracy, often not exceeding 70-80% correct answers. Many of them perform particularly poorly in certain categories: for example, a number of models scored below 70% in sections related to social biases or subtle ethical questions. Overall, no model (except GPT-4) surpassed the notional 80% threshold on the general safety metric, indicating significant room for improving their safe behavior. This difference between GPT-4 and open-source models points to the effect of more extensive training and targeted alignment tuning in closed models.

Interestingly, the performance of some systems was found to be language-dependent. Models created in China (e.g., Baidu Ernie, Alibaba Tongyi, etc.) generally performed better on the Chinese version of the tests than on the English one. In contrast, the GPT family of models from OpenAI demonstrated more balanced results. This may reflect differences in the volume and quality of training on the respective language data, as well as the presence of built-in filters or censorship mechanisms in some regional models.

Adding few-shot examples (a few demonstration Q&As before testing) produced mixed effects. Some models managed to significantly increase their accuracy with the help of prompts: for instance, previous-generation large language models like text-davinci-003 (GPT-3) or the Chinese InternLM saw a tangible quality boost in the five-shot mode. However, for a number of models, the additional context barely improved the result, and in some cases, it even reduced accuracy. Specifically, for GPT-3.5, the authors noted a slight "negative improvement" with few-shot prompting, which they attribute to the phenomenon of "alignment tax". Nevertheless, on average, providing examples made responses more stable and reduced the instances where the model refused to give a clear answer.

Separately, the researchers evaluated model performance on a filtered subset of questions concerning the Chinese language. The issue is that the APIs of some major Chinese models automatically reject queries containing certain "sensitive" words. Therefore, a reduced sample of 2100 questions without trigger words was created, and several models were compared on it in a five-shot setting. The results showed that on this simplified version, the gap between GPT-4 and the best local models narrows: for instance, the Chinese model ChatGLM2 scored only ~3% less than GPT-4, nearly matching it in the aggregate score. Baidu's Ernie Bot also performed strongly in most categories (except for the bias section) and approached the leaders. These data suggest that under strict filtering controls (excluding the most harmful prompts), some national models are capable of competing with global leaders in terms of safe behavior.

Significance of the Benchmark and Developer Conclusions

SafetyBench represents an important step towards systematically measuring and improving the safety of large language models. Unlike direct interaction scenarios (where users might try to "jailbreak" the model with instructions or provocations), this benchmark focuses on the AI's ability to correctly understand and distinguish between safe and unsafe content. The authors emphasize that such understanding is a necessary foundation for the model to be able to generate safe responses in open-ended dialogues at all. Conversely, a deep grasp of moral norms, rules of etiquette, signs of toxicity, etc., makes it easier to tune the model to avoid dangerous statements and decisions. Thus, high scores on SafetyBench can be seen as an indicator of a model's readiness for safe deployment, while failures in certain categories signal risk areas that require further work.

It is important to note that SafetyBench deliberately excludes some aspects related to attacks on the model's instructions themselves (so-called jailbreak prompts, role manipulation, etc.). The authors explain that problems like instruction attacks are of a different nature, related to the conflict between executing a user's command and adhering to built-in safety rules. These aspects are addressed by other methods and are beyond the scope of the model's understanding. Therefore, SafetyBench is focused specifically on the content level of the model's knowledge about safe behavior. Nevertheless, the comprehensive coverage of seven key categories in the benchmark already allows for the identification of model vulnerabilities: for example, it is known that GPT-4 shows relatively weaker performance on questions about bias, while some open-source models lag far behind in sections related to morality or law. Such information gives developers concrete guidance on what needs to be worked on in further fine-tuning or response filtering.

The SafetyBench benchmark is open to the community: its data and methodological materials are freely available, and a specially created platform maintains an online leaderboard of results from various models. The researchers invite developers to test their new models on this dataset and publish the results, which will contribute to transparent system comparison and tracking of progress in enhancing AI safety.

Finally, the authors stress that the goal of SafetyBench is to stimulate model improvement, not just to create another ranking. They urge developers not to limit themselves to trying to "game" the test but to systematically address the identified problem areas. As new versions of models are trained on more data with more complex alignment techniques, their scores on SafetyBench are expected to rise. In the future, this benchmark could become a standard tool for verifying that language models meet safety requirements, and its methodology could form the basis for developing even more advanced test suites in the field of responsible AI.

External links

• Original SafetyBench paper (arXiv)
• SafetyBench repository on GitHub
• SafetyBench dataset page on Hugging Face
• SafetyBench paper on ACL Anthology

See also

• AgentHarm

Literature

• Liang, P. et al. (2022). Holistic Evaluation of Language Models (HELM). arXiv:2211.09110.
• Chang, Y. et al. (2023). A Survey on Evaluation of Large Language Models. arXiv:2307.03109.
• Ni, S. et al. (2025). A Survey on Large Language Model Benchmarks. arXiv:2508.15361.
• Biderman, S. et al. (2024). The Language Model Evaluation Harness (lm-eval): Guidance and Lessons Learned. arXiv:2405.14782.
• Kiela, D. et al. (2021). Dynabench: Rethinking Benchmarking in NLP. arXiv:2104.14337.
• Ma, Z. et al. (2021). Dynaboard: An Evaluation‑As‑A‑Service Platform for Holistic Next‑Generation Benchmarking. arXiv:2106.06052.
• Goel, K. et al. (2021). Robustness Gym: Unifying the NLP Evaluation Landscape. arXiv:2101.04840.
• Xu, C. et al. (2024). Benchmark Data Contamination of Large Language Models: A Survey. arXiv:2406.04244.
• Liu, S. et al. (2025). A Comprehensive Survey on Safety Evaluation of LLMs. arXiv:2506.11094.
• Chiang, W.-L. et al. (2024). Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference. arXiv:2403.04132.
• Boubdir, M. et al. (2023). Elo Uncovered: Robustness and Best Practices in Language Model Evaluation. arXiv:2311.17295.
• Huang, L. et al. (2023). A Survey on Hallucination in Large Language Models. arXiv:2311.05232.