UChicago Hires Mateo Díaz to Study Why Simple Algorithms Solve Huge Problems

A puzzle at the core of optimization

Díaz compares optimization to finding the lowest point of a hilly landscape without ever seeing all of it. According to the university, many optimization algorithms work this way. "At each iteration, they can see their most recent choice or history of choices, but not the function globally," Díaz explains. Even so, he says, "algorithms are still very good at finding useful solutions."

Classical optimization theory explains this well when functions are convex, meaning the problems are tidy. Modern machine learning is different. Its problems are nonsmooth, nonconvex and involve enormous spaces with billions of parameters, and classical theory cannot keep up. "There is a very steep divide," Díaz says, between what is known for nicely structured functions and what happens when pre-training neural networks, for example for large language models (LLMs).

Díaz is developing theory to identify the geometric properties of complicated loss landscapes that let simple algorithms succeed repeatedly. He hopes to turn those insights into algorithms that are more efficient, easier to scale on modern hardware and more reliable for high-stakes uses.

From theory to production solvers

Before UChicago, Díaz was an Assistant Professor of Applied Mathematics and Statistics at Johns Hopkins University. He received his Ph.D. in Applied Mathematics from Cornell University. Most of his work is theoretical and sits at the intersection of optimization, geometry and statistics, but some of it has already reached practice.

One example is linear programming, a class of optimization problems first solved by George Dantzig's simplex algorithm in the 1940s. It is still widely used, but the problems have grown with the world's data. Díaz says classical methods such as simplex "do not scale very well because they rely on subroutines, like matrix inversion, that are hard to parallelize."

Together with collaborators at Google Research, he turned to first-order optimization methods, which are straightforward to parallelize. These had long been seen as too slow to be practical. Díaz says his team showed that "if you enhance them correctly, they can be both practical and a desirable alternative," particularly for problems too big for a laptop. He says the method is now deployed in multiple production-grade solvers, including implementations by NVIDIA, Google and Gurobi.

Why AI raises the stakes

Díaz says the rise of modern AI has not changed the questions he works on, but it has widened the gap between theory and practice. Algorithms can train models at extraordinary pace and scale, yet researchers do not fully understand why the underlying optimization works so well. "We don't understand the dynamics of neural network training," he says. "Not at the scale that we're seeing in practice."

He argues this matters most in safety-critical applications, where algorithms cannot be responsibly deployed without guarantees about when they can be trusted, what conditions their success requires and where their limits lie.

Díaz also says AI can produce in days work that once took months or years, so "the bar now should be significantly higher." He calls for more ambition in the questions researchers dare to ask.

What he plans at UChicago

Díaz says he was drawn to UChicago because it values ideas and discourse. "UChicago is an institution that really values the quality of intellectual ideas over, say, the number of citations," he says.

He hopes to build interdisciplinary collaborations inside and beyond the DSI. Following his work with biomedical engineers at Hopkins, he wants to work with colleagues in the School of Medicine, especially on imaging problems where the underlying mechanism is itself an optimization algorithm. He describes the DSI's students and faculty as "absolutely fantastic."