MIT Historian: AI Hallucinations Trace Back to Randomness Built In on Purpose
Generative AI models grew out of mid-20th century ideas that a thinking machine needs some irrationality, argues MIT historian Benjamin Lindquist. In a new paper in the journal Past and Present, he traces AI hallucinations back to early engineers who deliberately injected randomness into otherwise deterministic algorithms.
- MIT's Benjamin Lindquist argues in Past and Present that early AI engineers deliberately injected randomness to mimic human creativity.
- He traces the origins of AI hallucinations to this early research.
- Early neural network advocates valued unexpected answers, the opposite of what users expect from ChatGPT or Claude today.
- Lindquist says suppressing hallucinations involves tradeoffs, since surprising conclusions and errors are connected.
Randomness as a route to creativity
Benjamin Lindquist is an assistant professor in the MIT History Section. He holds a shared position with the MIT Schwarzman College of Computing and the Department of Electrical Engineering and Computer Science (EECS), and is a principal investigator in the Laboratory for Information and Decision Systems. His paper is titled "The Irrational Computer: Randomness and Unreason in the History of Artificial Intelligence."
According to Lindquist, many figures in science and art have long theorized that chance plays an important part in discovery and creativity. Early computer engineers equated creativity with unexpected leaps of logic, or even a dream-like state that lets a person look at things from a new perspective. To mimic that kind of thinking, AI pioneers intentionally added randomness to algorithms built to solve problems deterministically.
He points to John Pierce, a prominent electrical engineer at Bell Labs, who in the early 1950s argued that math was valuable in art because it could remove order. In a calculated way, math could add a burst of the unexpected, helping to avoid clichés and introduce irregularity that might lead to insight.
Neural networks and two camps
One popular way to bring irregularity into predictable computers was through artificial neural networks. Lindquist says many early networks were built by psychologists, who hoped a model of the brain would let scientists observe what happens inside, including how brains make productive mistakes. Researchers believed these systems could open the black box of the mind and show "all the messy stuff that was really happening."
Another camp held that creative thinking was just normal thinking done better. The debate, Lindquist says, was about what human inventiveness boils down to and how to rebuild it in machine form.
He notes a sharp contrast with today. People now value systems such as ChatGPT or Claude because their answers are mostly correct. Early advocates valued neural networks for the opposite reason: they wanted a system that occasionally gave answers that deviated from expectations, even strange or bizarre ones. Because early systems were confined to laboratories, wrong outputs carried no real-world consequences. Now, he says, there are legal and monetary reasons why OpenAI or Anthropic would not want a chatbot that gives unexpected or creative answers, and there has been a "sanding down" of strangeness. That quality may still sit at the core of the systems, but it is largely suppressed.
How neural networks became dominant
For a long time, Lindquist explains, "symbolic AI" was the dominant approach. It assumes the world is based on rules, and that researchers should reconstruct knowledge as facts and propositions and feed it into a computer. Neural networks gained traction in the 1950s but fell out of favor early on. Frank Rosenblatt, a psychologist and important early figure, made grand promises about what such systems could do. He died accidentally on his 43rd birthday in 1971, and by then a shift toward symbolic AI was already underway.
Neural networks never went away. By the 1980s they had moved more into cognitive science. To grow more powerful, they needed a lot of data and computational power, which did not arrive until the internet era. Around 2012, Lindquist says, neural networks started to outperform a statistical form of machine learning, and they kept growing from there.
A cautionary lesson about tradeoffs
Lindquist draws a cautionary point from this history. Developers can work toward suppressing hallucinations and making AI behave a certain way, but there will be other consequences. In his view, there is a connection between the surprising, creative conclusions AI can offer and errors or hallucinations.
He suggests there are inevitable tradeoffs between creativity and predictability when building machines that mimic human behavior. Maybe it is impossible to be completely at either end of the spectrum, he says, but when engineers work robustly toward one goal, they often sacrifice something else unintentionally.
MIT News: Q&A: How AI came to model the “messy” human mind
