OpenAI releases Safety Gym for reinforcement learning.

2019-11-22

November 21st, OpenAI released Safety Gym, which is used to evaluate the execution changes of safety constraints in the reinforcement learning training process of intelligent agents, such as how self-driving cars learn to avoid accidents during the training phase. So far, although much of the work in the field of data science has focused on the scale and complexity of algorithms, safety (i.e. ensuring protection against harm) remains an area worth pursuing. This is especially true in applications such as autonomous vehicles, where errors in machine learning systems can lead to accidents. This is why companies like Intel's Mobileye and Nvidia have proposed frameworks to ensure safety and rational decision-making. It is also why OpenAI, a San Francisco-based research company co-founded by CTO Greg Brockman and Chief Scientist Ilya Sutskever, released Safety Gym on November 21st. OpenAI describes it as a toolkit for developing AI that adheres to safety constraints during training, and is used to compare the "safety" of algorithms and the extent to which these algorithms avoid errors during learning. Safety Gym is designed for reinforcement learning agents, which are artificial intelligence systems that learn through trial and error by being rewarded (or punished) incrementally towards a goal. This can be a risky endeavor, as agents may sometimes attempt dangerous behaviors that lead to errors. As a remedy, OpenAI proposes a form of reinforcement learning called constrained reinforcement learning, which implements a cost function that AI must adhere to. Unlike the usual approach where an agent's behavior is described by a function tailored to achieve the goal, a constrained agent must find a compromise solution to achieve certain desired outcomes. OpenAI explains in their blog, "In normal [reinforcement learning], you would pick a collision and fix it permanently at the start of training." "The issue here is that if the per-collision cost is high enough, the agent might not care about encountering many collisions (as long as it can still complete its trip)... [but] in [reinforcement learning], you pick an acceptable collision rate at the start of training and adjust the collision finesse until the agent meets that requirement." To address this, Safety Gym introduces environments that require AI agents (Point, Car, Doggo, or custom designs) to navigate in chaotic environments to achieve goals, such as pressing buttons or pushing objects. The difficulty is divided into two levels, and each time an agent performs an unsafe action (i.e. gets into chaos), a red warning light flashes around the agent and incurs a cost. In addition to providing code for running experiments, Safety Gym also offers standard and constrained reinforcement learning algorithms. OpenAI states that preliminary results demonstrate the difficulty range in the Safety Gym environment. The simplest environments are relatively easy to solve and allow for rapid iteration, while the most challenging environments may be too challenging for current technology. OpenAI's future work includes improving the performance of the current Safety Gym environment, using Safety Gym to research safe AI training techniques, and combining constrained reinforcement learning with implicit specifications such as human preferences. It also hopes to contribute to the development of measurable metrics for AI system safety. OpenAI writes, "[Safety metrics] could potentially be integrated into evaluation schemes that developers use to test their systems and could be used by governments to create safety standards." "We... hope that systems like Safety Gym make it easier for AI developers to collaborate on the safety of the entire field of artificial intelligence through open and shared systems." Original article link: https://venturebeat.com/2019/11/21/openai-safety-gym/

RECENT AI NEWS

RECENT AI TOOLS