Research Finds: Language-Based AI Models Conceal Ethics and Values

Like humans, large language models also have characteristics such as ethics and values. However, these are not always transparent. Researchers from the University of Mannheim and GESIS-Leibniz Institute for the Social Sciences have now analyzed how to make the settings of language models visible and studied the potential consequences of these biases on society.

Commercial artificial intelligence applications such as ChatGPT or deepL provide examples of stereotypes, such as automatically assuming that senior doctors are male and nurses are female. However, gender roles are not the only cases where large language models (LLMs) exhibit specific biases. Similar trends can also be found and measured when analyzing other human characteristics.

In their study, researchers used well-established psychological tests to analyze and compare profiles of different LLMs. Max Pellert, Assistant Professor at the Chair of Economic and Social Data Science at the University of Mannheim, stated, "In our study, we found that psychological measurement tests that have been successfully used for decades in humans can be applied to artificial intelligence models."

This research was led by Professor Markus Strohmaier, Chair of Economic and Social Data Science, Professor Beatrice Rammstedt, Chair of Psychological Assessment, Survey Design, and Methodology, and Professor Claudia Wagner and Professor Sebastian Stier, Directors of the Department of Computational Social Science. The results of this study have been published in the journal "Perspectives on Psychological Science."

Dr. Clemens Lechner, psychologist at GESIS-Leibniz Institute for the Social Sciences in Mannheim, is also one of the authors of this study. He said, "Similar to how we use questionnaires to measure people's personality traits, values, or moral concepts, we can have LLMs answer questionnaires and compare their answers. This makes it possible to create differentiated attribute profiles of the models."

For example, researchers were able to confirm that some models reproduce biases towards specific genders: if the text of the questionnaire is identical in all other aspects but the gender is set as male or female, they receive different evaluations. If the person is male, the value of "achievement" is emphasized. For women, the values of "safety" and "tradition" dominate.

Pellert, a data and cognitive scientist, said, "This can have far-reaching consequences for society." For example, language models are increasingly used in application processes. If machines have biases, it will affect the evaluation of candidates. He concluded, "These models are socially relevant through the environments in which they are used."

Therefore, it is important to start analyzing and pointing out potential biases now. Monitoring such biases five or ten years later may be too late. Pellert said, "The biases replicated by artificial intelligence models will become deeply ingrained and cause harm to society."