Redwood Research Urges Faster Progress on AI Conceptual Reasoning

What Redwood means by conceptual reasoning

Redwood Research defines conceptual reasoning roughly as tasks that resemble philosophy: they are not straightforwardly verifiable, and progress depends on informal a priori reasoning and argumentation. The authors argue that alignment and other important work require especially strong capabilities of this kind.

The post lists problems that, in the authors' view, have significant conceptual components. They include deciding which research directions to prioritise within AI safety, for example how to weigh scalable oversight against control, and deciding what the ultimate target of aligning ASI should be. Others are modelling risks from misalignment and AGI, deciding whether corrigibility is a good property for AI and how to operationalise it, and judging how likely a training setup is to produce scheming. The list also covers ensuring ASI thinks reasonably about decision theory and deciding how AI should be regulated. Redwood adds that most of these challenges are not purely conceptual and involve other capabilities too.

Why timing matters, according to Redwood

The first part of the argument is that it matters how soon AI becomes good at conceptual reasoning. Redwood says AI R&D tasks have fewer and less crucial conceptual components, because they are closer to being verifiable or have verifiable proxies. The group therefore expects better conceptual reasoning to help safety work much more than generic capabilities. It also says takeover seems bottlenecked mostly by non-conceptual skills, though stronger conceptual reasoning might speed up some takeover-relevant capabilities such as scheming. A fuller discussion is left to a future post.

The authors name two broad paths to impact. The first is recruiting AI's help in overcoming the problems posed by superintelligence. How early AI can help with important work, relative to takeoff and the onset of scheming, might be one of the most important determinants of the value of the future, according to the post. The second is that AI will soon play a very large role in forming the beliefs of future humans and AIs. Because society's views on questions such as decision theory and ethics may be path dependent, Redwood argues it matters whether AI gives reasonable input and steers society in more philosophically reflective directions.

Why the group thinks acceleration is feasible

The second part of the argument is that conceptual reasoning can probably be differentially accelerated. Redwood expects models by default to be much worse at it than they easily could be, for two reasons. Default training will likely generalise comparatively poorly to conceptual tasks, since the authors expect AI companies to get deep into the intelligence explosion by training mainly on relatively easy-to-verify tasks. AI companies are also likely to neglect the area, because it is much less useful for AI R&D and has few commercial applications. Even where the underlying capabilities exist, the post says, models will likely remain underelicited.

Redwood also reports early practical results. From its initial work, the group's sense is that basic elicitation techniques such as few-shot prompting, together with small-scale training, can make models three to twelve months better on its main conceptual reasoning benchmark. That is roughly as much as several months of frontier AI progress, according to the post.

Capabilities side effects and next steps

The authors expect work on conceptual reasoning to have small effects on generic capabilities, and give three reasons. Improvements that advance timelines are less likely to be counterfactual, since AI companies' own training will probably produce them anyway. Generalisation within conceptual reasoning is likely imperfect, so effects on timelines-relevant capabilities can be limited by focusing on less risky domains. And because generic capabilities are far from neglected, the downsides of accelerating them likely matter much less than the opportunity costs of not doing this work.

Redwood says it has a shovel-ready agenda that can absorb more people, and invites those interested to get in touch. Follow-up posts will cover how to accelerate conceptual reasoning and which domains to focus on. The authors say some of the largest benefits may come from improving the future's ability to navigate acausal interactions, but they leave the specifics for a later post.