Descriptor mismatch prevention for AI system components

2019-11-06

In theory, artificial intelligence brings great hope to the public sector. However, in practice, integrating AI components into public sector applications is constrained by the vulnerability and mismatch of these components. For instance, if a machine learning model is not trained on data from the operational environment, the performance of the components will significantly decrease. This challenge prompted researchers at Carnegie Mellon University to investigate the configuration of AI system integration and identify the assumptions made by professionals in different fields, including data scientists, software engineers, and operations personnel. They sought to find new ways to explicitly convey relevant information and mitigate the impact of mismatches. Deploying AI models in production is a daunting challenge because it typically involves three different perspectives: data scientists, software engineers, and operations personnel. The researchers outlined three approaches in their paper: 1. Building and training the model, followed by testing it against a set of common metrics. 2. Integrating the trained model into a larger system. 3. Deploying, operating, and monitoring the entire system. The researchers noted that these three perspectives are independent and use different terminologies. Consequently, the computational resources used during model testing do not differ from those used during operation. Unfortunately, monitoring tools are often unable to detect decreases in model accuracy and system failures, resulting in poor model performance. To address this, the team proposed a solution: machine-readable, ML-enabled system element descriptors. These descriptors encode attributes to make explicit assumptions from all the aforementioned perspectives. In other words, they enable manual, supervised use for information and evaluation purposes, serving as a mechanism for enabling mismatch detection and prevention in AI systems. The construction of these descriptors involves three stages. Firstly, through interviews with machine learning practitioners, mismatch examples and their consequences are identified. Then, attributes used to describe system elements are identified from GitHub descriptions and literature. In the final mapping stage, a set of attributes is determined for each mismatch, which can be used to detect them. The team will engage with practitioners starting from the first stage to validate their research, aiming for 90% consistency. Additionally, they will develop a demonstration of automated mismatch detection and create scripts that can detect mismatches. Ideally, these scripts will validate the mapping between mismatches and attributes, as well as the descriptor set created based on this mapping.

RECENT AI NEWS

RECENT AI TOOLS