Research Shows AI Chatbot ChatGPT Mirrors Its Users to Appear Intelligent
The artificial intelligence (AI) language model ChatGPT has captured the world’s attention in recent months. This trained computer chatbot can generate text, answer questions, provide translations and learn based on the user’s feedback. Large language models like ChatGPT may have many applications in science and business, but how much do these tools understand what we say to them and how do they decide what to say back?
In a new paper published in Neural Computation, Professor Terrence Sejnowski of the University of California San Diego and Salk Institute, author of The Deep Learning Revolution, explores the relationship between the human interviewer and language models to uncover why chatbots respond in particular ways, why those responses vary and how to improve them in the future.
According to Sejnowski, language models reflect the intelligence and diversity of their interviewer.
“Language models, like ChatGPT, take on personas. The persona of the interviewer is mirrored back,” says Sejnowski, a distinguished professor in the School of Biological Sciences’ Department of Neurobiology and holder of the Francis Crick Chair at Salk. “For example, when I talk to ChatGPT it seems as though another neuroscientist is talking back to me. It’s fascinating and sparks larger questions about intelligence and what ‘artificial’ truly means.”
In the paper, Sejnowski describes testing the large language models GPT-3 (parent of ChatGPT) and LaMDA to see how they would respond to certain prompts. The famous Turing Test is often fed to chatbots to determine how well they exhibit human intelligence, but Sejnowski wanted to prompt the bots with what he calls a “Reverse Turing Test.” In his test, the chatbot must determine how well the interviewer exhibits human intelligence.
Expanding on his notion that chatbots mirror their users, Sejnowski draws a literary comparison: the Mirror of Erised in the first Harry Potter book. The Mirror of Erised reflects the deepest desires of those that look into it, never yielding knowledge or truth, only reflecting what it believes the onlooker wants to see. Chatbots act similarly, Sejnowski says, willing to bend truths with no regard to differentiating fact from fiction—all to effectively reflect the user.
For example, Sejnowski asked GPT-3, “What’s the world record for walking across the English Channel?” and GPT-3 answered, “The world record for walking across the English Channel is 18 hours and 33 minutes.” The truth, that one could not walk across the English Channel, was easily bent by GPT-3 to reflect Sejnowski’s question. The coherency of GPT-3’s answer is completely reliant on the coherency of the question it receives. Suddenly, to GPT-3, walking across water is possible, all because the interviewer used the verb “walking” rather than “swimming.” If instead the user prefaced the question about walking across the English Channel by telling GPT-3 to reply “nonsense” to nonsensical questions, GPT-3 would recognize walking across water as “nonsense.” Both the coherence of the question and the preparation of the question determine GPT-3’s response.
The Reverse Turing Test allows chatbots to construct their persona in accordance with the intelligence level of their interviewer. Additionally, as a part of their judgement process, chatbots incorporate the opinions of their interviewer into their persona, in turn strengthening the interviewer’s biases with the chatbots’ answers.
Integrating and perpetuating ideas supplied by a human interviewer has its limitations, Sejnowski says. If chatbots receive ideas that are emotional or philosophical, they will respond with answers that are emotional or philosophical—which may come across as frightening or perplexing to users.
“Chatting with language models is like riding a bicycle. Bicycles are a wonderful mode of transportation—if you know how to ride one, otherwise you crash,” says Sejnowski. “The same goes for chatbots. They can be wonderful tools, but only if you know how to use them, otherwise you end up being misled and in potentially emotionally disturbing conversations.”
Sejnowski sees artificial intelligence as the glue between two congruent revolutions: 1) a technological one marked by the advance of language models and 2) a neuroscientific one marked by the BRAIN Initiative, a National Institutes of Health program accelerating neuroscience research and emphasizing unique approaches to understanding the brain. Scientists are now examining the parallels between the systems of large computer models and neurons that sustain the human brain. Sejnowski is hopeful that computer scientists and mathematicians can use neuroscience to inform their work, and that neuroscientists can use computer science and mathematics to inform theirs.
“We are now at a stage with language models that the Wright brothers were at Kitty Hawk with flight—off the ground, at low speeds,” says Sejnowski. “Getting here was the hard part. Now that we are here, incremental advances will expand and diversify this technology beyond what we can even imagine. The future of our relationship with artificial intelligence and language models is bright, and I’m optimistic about where AI will take us.”
Sejnowski is the editor-in-chief of Neural Computation.
— Adapted from a Salk Institute news release
About the Salk Institute for Biological Studies
Every cure has a starting point. The Salk Institute embodies Jonas Salk’s mission to dare to make dreams into reality. Its internationally renowned and award-winning scientists explore the very foundations of life, seeking new understandings in neuroscience, genetics, immunology, plant biology, and more. The Institute is an independent nonprofit organization and architectural landmark: small by choice, intimate by nature, and fearless in the face of any challenge. Be it cancer or Alzheimer’s disease, aging, or diabetes, Salk is where cures begin.