Probabilistic Artificial Intelligence: An Intelligent System Capable of Self-Assessment

2023-05-26

Although today's artificial intelligence systems are large and powerful, they often struggle to distinguish between illusions and reality. Autonomous driving systems may fail to correctly perceive pedestrians and emergency vehicles, resulting in fatal consequences. Conversational AI systems confidently fabricate facts and, after reinforcement learning training, typically struggle to accurately assess their own uncertainty. Researchers from MIT and UC Berkeley have collaborated to develop a new method for constructing precise AI reasoning algorithms that can simultaneously generate probabilistic data explanations and accurately evaluate the quality of these explanations. This new method is based on a mathematical approach called Sequential Monte Carlo (SMC). SMC algorithms have been widely used to calibrate uncertainty in AI by proposing possible data explanations and tracking the likelihood or impossibility of these explanations as more information is obtained. However, SMC algorithms are too simplistic for complex tasks. One of the main issues is that one of the central steps of the algorithm, proposing guesses for possible explanations (before tracking the relative likelihood of different hypotheses), is too simple. In complex application areas, examining the data and proposing reasonable guesses about what might be happening is itself a challenging problem. For example, in the case of autonomous driving, this requires examining video data captured by the autonomous vehicle's cameras, identifying vehicles and pedestrians on the road, and guessing the possible motion paths of pedestrians currently hidden from view. Proposing reasonable guesses from raw data may require complex algorithms, which conventional SMC algorithms cannot support. This is where the advantage of the new method, called Sequential Monte Carlo with Probabilistic Program Proposals (SMCP3), lies. SMCP3 allows for more intelligent inference of possible data explanations and updates the proposed explanations based on new information, as well as evaluating the quality of these complexly proposed explanations. SMCP3 achieves this by allowing the use of any probabilistic program - that is, a computer program that allows for random selection - as a strategy for proposing data explanations (i.e., intelligent guesses). Early versions of SMC only allowed for very simple strategies, to the point where the accurate probability of any guess could be calculated. This limitation made it difficult to use multi-stage guessing processes. The researchers' SMCP3 paper demonstrates that by using more complex proposal processes, SMCP3 can improve the accuracy of AI systems in tracking 3D objects and analyzing data, as well as improve the algorithm's own estimation of the likelihood of the data. Previous research from MIT and other institutions has shown that these estimations can be used to infer how well a reasoning algorithm explains the data relative to an ideal Bayesian reasoner. George Matheos, co-first author of the paper and incoming MIT doctoral student in Electrical Engineering and Computer Science, expressed his excitement about the potential of SMCP3, which makes it possible to use well-understood, uncertainty-calibrated algorithms in complex problem settings where older versions of SMC would not work. "Today, we have many new algorithms, many of which are based on deep neural networks, that can propose what might be happening in various problem domains based on data. However, these algorithms are often not true uncertainty-calibrated algorithms. They only output one possible scenario, without clarity on whether this is the only reasonable explanation, whether other explanations exist, or even whether it is a good explanation! But with SMCP3, I believe it is possible to build uncertainty-calibrated algorithms using more of these clever but hard-to-trust algorithms. As we use AI systems to make decisions in more and more areas of life, having systems we can trust, that know their own uncertainty, will be crucial for reliability and safety." Senior author Vikash Mansinghka added, "The first electronic computer was used to run Monte Carlo methods. Monte Carlo methods are one of the most widely used techniques in computation and AI. But from the beginning, Monte Carlo methods have been difficult to design and implement: mathematical formulas had to be derived manually, and there are many subtle mathematical constraints that users must be aware of. SMCP3 automates the heavy mathematical computations and expands the design space. We have used it to come up with new AI algorithms that were previously impossible to design." Source: https://news.mit.edu/2023/probabilistic-ai-knows-how-well-its-working-0525

RECENT AI NEWS

RECENT AI TOOLS