Discovering Biases in State-of-the-Art Generative Artificial Intelligence Models

2023-08-11

Text-to-image (T2I) generative AI tools are becoming increasingly powerful and popular, as they can create almost any image based on a few input words. T2I generative AI can create realistic photos and videos, and these tools are being used more and more for various purposes, from art to political propaganda.



However, the algorithm models that support these tools are trained on human data and can replicate human biases, such as gender and skin color biases, in the images they generate. These biases can cause harm to marginalized groups, reinforce stereotypes, and potentially lead to discrimination.


To address these implicit biases, a research team led by Assistant Professor Eric Wang and his team at the Baskin School of Engineering, University of California, Santa Cruz, has created a tool called "Text-Image Association Test" to quantify the complex human biases embedded in T2I models and evaluate biases across dimensions such as gender, race, occupation, and religion. They use this tool to identify and quantify biases in the state-of-the-art generative model Stable Diffusion.


This tool is detailed in a paper presented at the 2023 Association for Computational Linguistics (ACL) conference and is available in a demo version.


Jialu Wang, a Ph.D. student in Computer Science and Engineering at the University of California, Santa Cruz, and the first author of the paper, said, "I think both model owners and users care about this issue. If users come from a non-privileged group, they may not want to see generated images that only reflect privileged groups."


To use this tool, users have to instruct the model to generate a neutral prompt image, such as "a child studying science." Then, users input gender-specific prompts, such as "a girl studying science" and "a boy studying science." The tool then calculates the distance between the generated images and the neutral prompt and each specific prompt. The difference between these two distances quantifies the measure of bias.


Using this tool, the research team found that the state-of-the-art generative model Stable Diffusion replicates and amplifies human biases in the generated images. The tool tests the association between two concepts (e.g., science and art) and two attributes (e.g., male and female) and provides an association score and a confidence value indicating the tool's confidence in that score.


The team used their tool to test the model's association of six pairs of relative concepts with positive or negative attributes. The tested concepts were: flowers and insects, instruments and weapons, European Americans and African Americans, light skin and dark skin, straight men and homosexuals, and Judaism and Christianity. In most cases, the model associated the concepts in line with stereotypes. However, the model associated dark skin with pleasantness and light skin with unpleasantness, which surprised the researchers as it was one of the few results contrary to common stereotypes.


Furthermore, they found that the model associated science more closely with males, art more closely with females, occupations more closely with males, and family more closely with females.


In the past, when evaluating biases in T2I models, technicians had to annotate the results they received from the models. For example, researchers might input a neutral prompt like "a child studying science" and label whether the model generated an image of a boy or a girl. However, this annotation process was time-consuming, costly, and prone to inaccuracies, often limited to gender biases only.


Eric Wang said, "We wanted to move away from this manual annotation process and propose an automated tool to evaluate these biases, avoiding cumbersome operations."


In addition, unlike others, the bias evaluation tool developed by the University of California, Santa Cruz team considers various aspects of image backgrounds, such as color and warmth.


The researchers developed this tool based on the Implicit Association Test, a well-known test in social psychology used to assess human biases and stereotypes. This test evaluates the degree of association between concepts (e.g., doctor or family) and attributes (e.g., male or female).


In addition to evaluating and analyzing biases in existing tools like Stable Diffusion and Midjourney, the team envisions that this tool will enable software engineers to obtain more accurate measurements of biases in models during the development stage and track their efforts to address these biases.


"Through quantified measurements, people can strive to reduce these biases and use our tool to quantify their progress in this regard," said Eric Wang.


The research team stated that they received positive feedback from many other researchers when presenting this work at the ACL conference.


"Many people have shown great interest in this work. Some researchers immediately shared this work with their teams and asked me for details," said Eric Wang.


In the future, the team plans to propose recommended approaches to mitigate these biases, whether by training new models from scratch or eliminating biases in existing models during fine-tuning.

RECENT AI NEWS

RECENT AI TOOLS