MIT researcher develops AI systems that can explain themselves

MIT Associate Professor Jacob Andreas is developing methods that could help artificial intelligence models explain what they learned and how they work in language people can understand. His research addresses the difficulty of interpreting deep-learning systems in high-stakes applications.
- Jacob Andreas is developing methods for AI models to explain what they learned.
- His group studies the algorithms used by large language models.
- The techniques can describe information-processing nodes in neural networks.
- Andreas joined the MIT faculty in 2019.
- His research aims to support more accurate and robust deep-learning models.
Making complex models easier to understand
Deep-learning models contain enormous networks of internal processing nodes. When researchers deploy them in high-stakes scenarios, understanding what a model knows and how it may behave requires analyzing that complex internal structure.
Andreas, a newly tenured associate professor in MIT’s Department of Electrical Engineering and Computer Science and a member of the Computer Science and Artificial Intelligence Laboratory, is pursuing a different approach. His group develops techniques that enable AI models to communicate accurately about what they learned from training data.
“Rather than needing to use all these external tools to try and understand what is going on under the hood, maybe we can train models to talk about themselves in a way that is accurate,” Andreas says, according to MIT News.
The aim is to give researchers tools that translate complicated neural computation into language humans can interpret. Andreas describes these as “mind-reading tools” for AI systems, intended to support the development of more accurate and robust deep-learning models.
Research spans language, vision and interpretability
Andreas’ interest in the subject began with language and computer programming. He studied computer science at Columbia University, where he worked on machine translation in the lab of Michael Collins, the Vikram S. Pandit Professor of Computer Science at Columbia University.
He later pursued graduate studies at the University of California at Berkeley after earning a master’s degree at Cambridge University in England. His research developed during a major shift in natural language processing, as neural networks increasingly replaced statistical methods for handling language.
At Berkeley, Andreas collaborated with computer vision researchers on systems that combined language and visual information. He completed a thesis project involving neural module networks, which use small neural-network components for tasks such as localizing objects or performing logical reasoning.
Projects involving interpretability and natural-language descriptions of AI models’ internal states later became central to his work at MIT. His group studies the algorithms used by large language models as they learn and process information, while also developing ways to predict how trained models will behave in the real world.
Toward more natural interaction with AI systems
One family of techniques developed by Andreas’ group trains large language models to explain what is happening inside other neural networks. These methods can assign a natural-language description to every information-processing node in a large neural network. One example could describe a node that identifies “patches of grass near animals” in input images.
The group also designs algorithms for training machine-learning models and for predicting how particular datasets or examples will affect different aspects of model behavior. These efforts reflect Andreas’ broader goal of making the training process more natural and efficient, so AI models learn in ways that are more like human learning.
For researchers and developers, the work could provide a way to examine model behavior without relying only on the models’ parameters or external analysis tools. Andreas says that a deep understanding of how the technology works will remain important as access to coding agents becomes widespread.
MIT News: Learning the language of machine learning


