GovAI reports 2020 output: two major reports, 15 academic papers and a bigger team

A growing team and a broad research agenda

In the report's opening note, GovAI director Allan Dafoe writes that, in his view, "the governance of AI will become among the most important global issues." He says 2020 brought continued developments in the field and that it is heartening to see how rapidly it grows.

According to the report, GovAI now has a core team of 9 researchers and a network of 21 affiliates and collaborators. It also welcomed two Visiting Senior Researchers, Joslyn Barnhart and Robert Trager.

The topics covered during the year include a theory of impact for AI governance, the Windfall Clause, Cooperative AI, the logic of strategic assets, national security and antitrust, corporate intellectual property strategies, AI researcher responsibility and impact statements, historical economic growth trends, AI and China, and trustworthy AI development.

Dafoe acknowledges the limits of this scope. Citing his paper AI Governance: Opportunity and Theory of Impact, he writes that the group is highly uncertain about the technical and geopolitical nature of the problem and should therefore acquire a diverse portfolio of expertise. GovAI's work, he says, covers only a small fraction of the problem space.

Selected publications

The report lists a series of papers. Several stand out:

• Open Problems in Cooperative AI: Authored by Dafoe with researchers largely from DeepMind, it argues that AI should explicitly focus on problems of cooperation, which the authors term Cooperative AI. GovAI co-organized a NeurIPS workshop on the topic.
• The Windfall Clause: A policy proposal for a mechanism in which AI developers make ex-ante commitments to distribute a substantial part of profits back to the global commons if they capture an extremely large part of the global economy through transformative AI. According to the report, the project was run by GovAI and inspired the Partnership on AI's Shared Prosperity Initiative.
• The Offense-Defense Balance of Scientific Knowledge: By Toby Shevlane and Dafoe, it asks whether publishing AI research reduces misuse. It argues that the pattern in software vulnerability disclosure, which often favors defense, cannot be assumed to hold for AI research.
• U.S. Public Opinion on the Governance of Artificial Intelligence: Presents results from a survey of 2,000 Americans' attitudes toward AI and AI governance.
• Social and Governance Implications of Improved Data Efficiency: Explores effects on market concentration, malicious use, privacy and robustness if researchers succeed in improving the data efficiency of machine learning.

Policy, law and institutions

Other work looks at the institutional side of AI. A forthcoming article in Nature Machine Intelligence reflects on the NeurIPS Broader Impact Requirement, which the authors describe as a novel governance initiative by one of the world's most prestigious AI conferences.

A GovAI working paper by Nathan Calvin and Jade Leung offers a preliminary analysis of the legal rules, norms and strategies governing AI-related intellectual property, including the practices of select companies and governments. A technical report by Cullen O'Keefe examines how national security considerations may affect antitrust decisions in AI, drawing on historical precedents. The report notes that AI, like railways, the internet and electricity, is likely to affect both national security and market structure.

A forthcoming Security Studies paper by Jeffrey Ding and Dafoe asks what makes an asset strategic. According to the report, clarifying this logic could move policymakers away from especially unhelpful rivalrous industrial policies. The paper applies the analysis to AI.

Funding and what comes next

GovAI thanks Open Philanthropy, the Future of Life Institute and the European Research Council for their support. It is part of the Future of Humanity Institute at the University of Oxford and cites growing connections with the Department of Politics and International Relations, the Department of Computer Science and the new Ethics in AI Institute.

The group says it wants to help new talent enter AI governance through its Governance of AI Fellowship, with applications expected to open in Spring 2021, as well as through hiring, collaborations and hosting senior visitors. Dafoe says GovAI is excited about growing its team and has big ambitions for further progress. Those interested can visit www.governance.ai or contact Markus Anderljung.