How to Evaluate the Performance of AI Models with a New Tool

2023-06-01

When machine learning models are deployed in the real world, it is like marking potential diseases in X-rays for radiologists to examine. Human users also need to know when to trust the predictions of the models. However, machine learning models are so large and complex that even the scientists who design them cannot accurately understand how these models make predictions. Therefore, they have created a technique called saliency methods to try to explain the behavior of the models. With the continuous release of new methods, researchers from MIT and IBM Research have developed a tool to help users choose the most suitable saliency method for their specific tasks. They have developed saliency cards, which provide a standardized document that introduces the operation of the methods, including their advantages and disadvantages, and provides explanations to help users interpret them correctly. Anjali Bogesht, a graduate student and co-chair of the MIT and AI Lab (CSAIL) Visualization Group, explained that with this information, they hope users can selectively choose the appropriate saliency method for the machine learning model they are using and the task it performs. Interviews with AI researchers and experts from other fields have shown that these cards help people quickly compare different methods and choose the right techniques. Choosing the right method allows users to have a more accurate understanding of the behavior of their models, enabling them to explain their predictions correctly. Bogesht said, "The saliency cards aim to provide a concise summary of saliency methods and break them down into the most critical, human-centered attributes. They are really designed for everyone, from machine learning researchers to non-expert users trying it for the first time." In addition to Bogesht, the other co-chairs of this study are Harini Suresh, a postdoctoral fellow at MIT, Hendrik Strobelt, a senior research scientist at IBM Research, John Guttag, a distinguished professor of computer science and electrical engineering at MIT, and Arvind Satyanarayan, an associate professor of computer science at MIT, who lead the Visualization Group at CSAIL. This research will be presented at the ACM Conference on Fairness, Accountability, and Transparency. Choosing the right method Researchers previously evaluated saliency methods using the concept of fidelity, which reflects the accuracy of a model's decision-making process. However, Bogesht explained that fidelity is not black and white. One method may perform well in one fidelity test but fail in another. Due to the large number of saliency methods and many possible evaluation methods, users often choose a popular method or one used by their colleagues. However, choosing the "wrong" method can have serious consequences. For example, a saliency method called integrated gradients compares the importance of features in an image to a meaningless baseline. This method typically uses all zeros as the baseline, but if applied to an image, all zeros are equivalent to black. Bogesht said, "It would tell you that any black pixels in the image are not important, even if they are meaningful, because they are the same as that meaningless baseline. This could be a big problem if you are looking at X-rays, as doctors may be concerned about black areas." Saliency cards summarize the workings of saliency methods based on 10 user-centered attributes, helping users avoid such problems. These attributes capture how saliency is computed, the relationship between saliency methods and models, and how users perceive their outputs. For example, one of the attributes is parameter dependence, which measures the sensitivity of the saliency method to user-specified parameters. The saliency card for integrated gradients will describe its parameters and how they affect its performance. With this card, users can quickly understand that the default parameter (all-zero baseline) may lead to misleading results when evaluating X-rays. These cards may also be useful for scientists as they reveal gaps in the research field. For example, researchers at MIT were unable to identify a saliency method that is both computationally efficient and applicable to any machine learning model. Bogesht said, "Can we fill this gap? Is there a saliency method that can do both? Or are these two ideas conflicting in theory?" Revealing their cards After creating several cards, the team conducted user research with eight domain experts, ranging from computer scientists to radiologists unfamiliar with machine learning. In interviews, all participants stated that the concise descriptions helped them determine the priority of attributes and compare methods. Bogesht said that even the radiologist, who was unfamiliar with machine learning, was able to understand the cards and use them in the process of selecting saliency methods. The interviews also revealed some surprises. Researchers often expected clinical doctors to need a clear method that focuses on specific objects in medical images. However, in this study, the clinical doctor actually preferred some noise in medical images to help reduce uncertainty. She said, "When we broke it down into different attributes and asked people, no one's priorities in the study were the same as others, even if they played the same role." In the future, researchers hope to explore attributes that are currently under-evaluated and design saliency methods for specific tasks. They also aim to gain a better understanding of how people perceive the outputs of saliency methods, which could lead to better visualizations. Additionally, Bogesht said they will host the work on a public repository for feedback and to drive future work. She said, "We really hope that these evolving cards will grow with new saliency methods and evaluations. Ultimately, this conversation about the attributes of saliency methods and their role in different tasks will be a good starting point." Source: https://techxplore.com/news/2023-05-tool-people-method-ai.html

RECENT AI NEWS

RECENT AI TOOLS