XAI - Explainable Artificial Intelligence

2019-12-20

The success of machine learning has promoted a new trend in artificial intelligence applications, which provide extensive benefits to various fields. However, many AI systems are unable to explain their autonomous decisions and actions to human users. For certain AI applications, explanations may not be necessary, and some AI researchers believe that the emphasis on explanations is misplaced, as it may be difficult to achieve or even unnecessary. However, in many critical applications such as defense, medicine, finance, and law, explanations are necessary to facilitate user understanding, trust, and effective management of these new AI partners.


The latest advancements in artificial intelligence are largely attributed to new machine learning techniques that involve internal representation models. These include support vector machines (SVM), random forests, probabilistic graphical models, reinforcement learning (RL), and deep learning (DL) neural networks. Although these models exhibit high performance, they lack transparency in terms of interpretability. There may be an inherent trade-off between machine learning performance, such as prediction accuracy, and interpretability. Typically, the highest performing methods, such as DL, are the most difficult to interpret, while the most interpretable methods, such as decision trees, are the least accurate.



What is XAI?


The goal of explainable AI (XAI) systems is to make human behavior more understandable by providing explanations. There are some general principles that can help create effective and more understandable AI systems: XAI systems should be able to explain their functionality and understanding, explain what they have done, what they are currently doing, and what will happen next, and disclose significant information about their operation.


However, each explanation is contextual, depending on the tasks, capabilities, and expectations of the users of the AI system. Therefore, the definition of interpretability and explainability depends on the domain and cannot be defined independently of the domain. Explanations can be complete or partial. Fully interpretable models provide complete and fully transparent explanations. Partially interpretable models reveal important parts of their reasoning process. Interpretable models adhere to "interpretability constraints" defined according to the domain (e.g., monotonicity of certain variables and related variables), while black-box or unconstrained models may not adhere to these constraints.



User Expectations


XAI assumes that explanations are provided to end users based on the decisions, recommendations, or operations of the AI system, but there may be many different types of users, often at different stages of system development and use. For example, one type of user may be an intelligence analyst, judge, or operator. However, other users who need explanations of the system may be developers or test operators who need to understand areas that may need improvement. Another user may be a decision-maker trying to assess the fairness of the system. Each user group may have preferred types of explanations that can convey information most effectively. Effective explanations will consider the target user group of the system, their background knowledge, and their specific needs for the explanations.



Interpretability - Evaluation and Measurement


Many methods have been proposed to evaluate and measure the effectiveness of explanations. However, there is currently no universal method to measure whether XAI systems are more capable of improving user understanding than non-XAI systems. From the user's perspective, some of these metrics are subjective, such as user satisfaction, which can be measured through subjective evaluations of clarity and usefulness of explanations. A more efficient objective measure may be task performance; that is, whether explanations improve the user's decision-making ability. Reliable and consistent measurement of the impact of explanations remains an open research question. The evaluation and measurement of XAI systems include evaluation frameworks, commonalities, common sense, and argumentation.


From a user-centered research perspective, studying capabilities and knowledge can enable XAI to go beyond explaining specific XAI systems and help its users determine appropriate roles of trust. In the future, XAI may play important social roles. These roles can include learning and explaining to individuals, coordinating with other agents to connect knowledge, developing interdisciplinary insights and commonalities, collaborating with humans and other agents for teaching purposes, and leveraging previously discovered knowledge to accelerate further discoveries and applications of knowledge. From this social perspective of knowledge understanding and generation, the future of XAI is just beginning.

RECENT AI NEWS

RECENT AI TOOLS