From Large Language Models to Hallucinations: A Concise AI Terminology Guide

Artificial intelligence is a profound and intricate field. Scientists working in this domain often use specialized terminology to describe their work, and when reporting on the AI industry, adopting these technical terms becomes essential. To assist with this, we have created a glossary defining some of the most critical terms and phrases used in our articles.

Artificial General Intelligence (AGI)

Artificial General Intelligence, or AGI, remains a loosely defined concept. Generally, it refers to AI that surpasses the average human capability across many, if not most, tasks. Sam Altman, CEO of OpenAI, recently described AGI as “equivalent to a mid-level human colleague you could hire.” Meanwhile, OpenAI’s charter defines AGI as “highly autonomous systems that outperform humans in most economically valuable work.” Google DeepMind offers a slightly different perspective, interpreting AGI as “AI capable of performing at least as well as humans across most cognitive tasks.” Confused? Don’t worry — experts at the forefront of AI research share your uncertainty.

AI Agents

An AI agent refers to a tool that uses AI technology to perform a range of tasks on your behalf — extending beyond the capabilities of more basic AI chatbots. These tasks include processing expense reimbursements, booking flights or restaurant reservations, and even writing or maintaining code. However, as we’ve previously explained, this emerging field contains many dynamic components, meaning the term “AI agent” can vary depending on who you ask. The infrastructure to support its envisioned functionality is also still being developed. Fundamentally, an AI agent represents an autonomous system that might utilize multiple AI systems to execute multi-step tasks.

Chain of Thought

When faced with a simple question, the human brain can often respond without much deliberation — for instance, “Which is taller, a giraffe or a cat?” But in many cases, arriving at the correct answer may require pen and paper due to intermediate steps. For example, if a farmer has chickens and cows with 40 heads and 120 legs total, solving this requires creating a simple equation (20 chickens and 20 cows).

In the context of AI, chain-of-thought reasoning for large language models means breaking down problems into smaller intermediate steps to improve the quality of the final result. This process often takes longer but tends to yield more accurate answers, especially in logic or coding scenarios. Reasoning models evolved from traditional large language models and are optimized through reinforcement learning to align with chain-of-thought processes.

Deep Learning

Deep learning is a subset of machine learning where AI algorithms are structured with multilayered artificial neural networks (ANNs), allowing them to make more complex associations than simpler machine-learning systems like linear models or decision trees. The structure of deep learning algorithms draws inspiration from the interconnected pathways of neurons in the human brain.

Deep learning AI models can autonomously identify significant features in data without requiring human engineers to define them. This structure also supports algorithms capable of learning from mistakes and improving outputs through repetition and adjustment. However, deep learning systems require vast amounts of data points to produce reliable results (millions or more). Their training times are typically longer compared to simpler machine learning algorithms, making development costs higher.

Diffusion

Diffusion serves as the core technology behind many AI models generating art, music, and text. Inspired by physics, diffusion systems gradually "corrupt" data structures — such as photos or songs — by adding noise until nothing remains. In physics, diffusion is spontaneous and irreversible — sugar dissolved in coffee cannot revert to its cube form. However, AI-based diffusion systems aim to learn a "reverse diffusion" process, recovering corrupted data from noise.

Distillation

Distillation is a technique for extracting knowledge from large AI models using a "teacher-student" model. Developers send requests to the teacher model and record its outputs. These outputs are sometimes compared against datasets for accuracy. Subsequently, they are used to train the student model, which learns to approximate the teacher's behavior.

Distillation can create a smaller, more efficient model based on a larger one while minimizing distillation loss. This method may explain how OpenAI developed GPT-4 Turbo (a faster version of GPT-4). Although all AI companies internally use distillation, some may leverage it to catch up with cutting-edge models. Distilling from competitors’ models often violates AI API and chat assistant service terms.

Fine-Tuning

Fine-tuning involves further training an AI model to enhance its performance on specific tasks or domains beyond its original training focus — typically achieved by introducing new, specialized (task-oriented) data.