GovAI Sets Up Policy Team to Prepare Governments and Labs for Advanced AI

Why GovAI is creating the team

According to GovAI, the AI governance space needs more rigorous work on what influential actors, such as governments and AI labs, should do in the coming years. The new Policy Team is meant to help close that gap.

The team will mainly work on AI policy development from a long-run perspective. GovAI says it will also spend some time advising and advocating for its recommendations, though it expects to rely heavily on other actors for that part. The work is aimed mostly at the governments of the US, UK and EU, as well as AI labs.

Three initial bets

GovAI plans to concentrate on a handful of bets at a time. Initially, it is likely to pursue three areas:

• Compute governance: The team asks whether compute is a particularly useful governance node for AI and, if so, how it can serve different goals. Those goals include monitoring capabilities, restricting access to capabilities, and identifying high-risk systems so they can receive significant scrutiny.
• Corporate governance: The questions here concern what measures frontier labs should adopt. They include what can be learned from other industries to improve risk management, how a board of directors can best oversee management, and how ethics boards should be designed.
• AI regulation: The team will examine which present-day regulation would best help manage risks from advanced AI systems. Example questions include whether foundation models should be a regulatory target, what features of AI systems regulation should mandate, and how to create more adaptive and expert regulatory ecosystems.

Approaches and people

GovAI lists several ways it will approach policy development. These include back-chaining from desirable outcomes to concrete recommendations, such as increasing the chance of effective international treaties on AI in the future, and considering what should be done today to prepare for a particular event, such as the US government making an Apollo Program-level investment in AI.

Other approaches are articulating and evaluating intermediate policy goals, analyzing specific governance levers, evaluating existing recommendations such as increasing high-skilled immigration to the US and UK, and giving concrete advice to decision-makers. One example given is providing input on the design of the US National AI Research Resource. Over time, GovAI plans to assess which bets and approaches are most fruitful and refine its focus.

The team currently consists of Jonas Schuett, who specializes in corporate governance, Lennart Heim, who specializes in compute governance, and Markus Anderljung, the team lead. It will also collaborate with the rest of GovAI and with people at other organizations.

Hiring and what it means

GovAI says it wants to grow the team. It will hire Research Scholars on the Policy Track on a regular basis and plans to work with people in its 3-month Fellowship. It is likely to open applications for Research Fellows in the near future, and expressions of interest can be submitted now. New staff can work from Oxford, where most of GovAI is based, from the Bay Area, or remotely.

For policymakers and labs in the US, UK and EU, the announcement signals a source of research on concrete governance questions, from compute oversight to board structures and regulation. For researchers interested in the field, it opens a route into policy-focused work at GovAI. The post is dated August 1, 2022.