What is the "Black Box" of Artificial Intelligence?

2023-05-29

A computer scientist explains what it means when the inner workings of ChatGPT and other artificial intelligence systems are hidden. For some, the term "black box" brings to mind the recording devices on airplanes that are valuable for forensic analysis in case of accidents. For others, it evokes thoughts of small, rudimentary theaters. However, the black box is also an important term in the world of artificial intelligence. An AI black box refers to an AI system with internal workings that are invisible to the user. You can provide input and receive output from them, but you cannot inspect the system's code or the logic behind generating the output. Machine learning is a major component of artificial intelligence. It serves as the foundation for generative AI systems like ChatGPT and DALL-E 2. Machine learning consists of three components: one or a group of algorithms, training data, and models. Algorithms are a series of processes. In machine learning, algorithms learn to recognize patterns after being trained on a dataset. Once a machine learning algorithm is trained, it produces a machine learning model that people can use. For example, a machine learning algorithm can be designed to recognize patterns in images, with dog images as the training data. The resulting machine learning model would be a canine observer. You can input an image into it and obtain an output that determines whether a group of pixels in the image represents a dog and its location. All three components of a machine learning system can be hidden, or set as black boxes. Typically, algorithms are public, which weakens the effect of putting them in a black box. Therefore, to protect their intellectual property, AI developers often set the models as black boxes. Another approach used by software developers is to obscure the training data used for the models, effectively making it a black box. Black boxes make it difficult for us to understand how artificial intelligence works. The opposite of a black box is sometimes referred to as a glass box. An AI glass box is a system where the algorithm, training data, and models are all visible to anyone. However, researchers still consider some aspects as black boxes because they do not fully understand how machine learning algorithms, especially deep learning algorithms, operate. The field of explainable AI is working on developing algorithms that may not be glass boxes but can provide better understanding for humans. Why are AI black boxes important? In many cases, there are valid reasons to be cautious about black box machine learning algorithms and models. Suppose a machine learning model has diagnosed your health condition. Would you prefer it to be a black box or a glass box? What about the doctor who prescribed medication based on the model's decision? She might want to know how the model arrived at its decision. Imagine a machine learning model that determines your eligibility for a business loan from a bank. If the model rejects you, wouldn't you want to know why? Knowing the reason would allow you to appeal more effectively or make changes to increase your chances of getting a loan next time. Black boxes also have significant implications for software system security. For years, many in the computing field believed that keeping software in a black box would prevent hackers from examining it, thus ensuring its security. This assumption has largely been proven wrong, as hackers can reverse-engineer software by carefully observing its workings and discovering exploitable vulnerabilities. If software is in a glass box, software testers and ethical hackers can inspect it and inform the creators of weaknesses, thereby minimizing network attacks. Source: https://gizmodo.com/chatgpt-app-what-is-an-ai-black-box-1850481273

RECENT AI NEWS

RECENT AI TOOLS