Google's Bard can now write and execute code to answer questions

2023-06-12

Google says having an LLM write code is like asking a human to do long division. Large language models (LLMs) like ChatGPT and Google Bard can provide decent answers to certain types of questions, but ironically, these computers are quite terrible at computation. Google has a new solution that aims to make language models correctly perform simple tasks like math: having AI write programs. Google states that now when you ask a "computational" task like math or string operations, Bard doesn't display the output of the language model, but instead writes a program, executes it, and presents the program's output as the answer to the user. Google's blog post provides an example input: "Reverse the word 'Lollipop' for me." ChatGPT failed to answer this question correctly and provided the wrong answer "pillopoL" because the language model views the world in terms of a large number of words or "tokens" and is not good at handling such problems. Here is an example of Bard's output: It produced the correct output "popilloL," but more interestingly, it also includes the Python code it wrote to answer this question. This is fascinating for those interested in programming, but for the average person, it may be the most terrifying output. And it's not particularly relevant either. Imagine if when you asked Gmail to fetch your emails, it displayed a piece of code; that would be strange. Do what you're supposed to do, Bard. Google compares the behavior of AI models writing programs to humans doing long division because it is a different mode of thinking: This approach is inspired by a well-studied dichotomy in human intelligence, particularly highlighted in Daniel Kahneman's book "Thinking, Fast and Slow" - the separation of "System 1" and "System 2" thinking. System 1 thinking is fast, intuitive, and effortless. When a jazz musician improvises or a typist considers a word and sees it appear on the screen, they are using System 1 thinking. In contrast, System 2 thinking is slow, deliberate, and effortful. When you do long division or learn to play a musical instrument, you are using System 2 thinking. In this analogy, LLMs can be thought of as purely operating in System 1 thinking - rapidly generating text without deep consideration. This brings some astonishing abilities but may fall short in certain cases (imagine solving math problems using only System 1 thinking - you can't stop and calculate, you can only choose the first answer that comes to mind). Traditional computational approaches align more with System 2 thinking: they are formulaic, inflexible, but the correct sequence of steps can yield impressive results, such as the solution to long division. Google states that this "live coding" approach will also be used for questions like "What are the prime factors of 15683615?" and "Calculate my savings growth rate." The company says, "So far, based on internal benchmarking, Google states that Bard has improved accuracy by about 30% for computation-based words and math problems." Google, as usual, warns, "Bard may fail to answer questions correctly, either due to a misunderstanding of your question or, like all of us, writing code that fails on the first run." Source: https://arstechnica.com/google/2023/06/googles-bard-ai-can-now-write-and-execute-code-to-answer-a-question/

RECENT AI NEWS

RECENT AI TOOLS