Google study: AI sharpens senior lawyers' judgment but not juniors'
A three-month randomized controlled trial with practicing patent attorneys found that AI raises work quality across the board, but only senior lawyers showed stronger unassisted judgment afterward. Junior lawyers showed no average skill gain, according to Google Research authors David Autor and Tanya Rodchenko.
- A trial with 133 lawyers at eleven IP law firms randomized access to a Google Labs patent writing assistant, now part of Gemini Notebook.
- AI raised drafting scores by 0.34 SD at 10 days and 0.38 SD at 90 days, according to the authors.
- On an unassisted redlining task, treated seniors outperformed controls by 0.45 SD, while juniors showed no discernible improvement.
- Junior scores split into more very low and more good results, with no more excellent ones.
- The authors say better performance and durable judgment are different goals that may conflict for juniors.
How the experiment worked
The study, published by the National Bureau of Economic Research under the title "Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting", involved 133 lawyers at eleven intellectual property law firms that do regular, non-exclusive business with Google. The researchers randomized access to a then-unreleased Google Labs AI patent writing assistant, which is now part of Gemini Notebook.
Two-thirds of lawyers at each firm formed the treatment group and received early access to the tool. The rest, the control group, got some training on using AI but were not allowed to use the tool until the three-month study ended.
The authors say that studying how AI erodes or builds expertise requires three elements rarely found together: sustained AI use in ordinary professional work over months, a credible evaluation of unassisted judgment, and blinded grading by domain experts. Third-party patent professionals scored submissions on five dimensions: enforceability, accuracy, strategic ambiguity, completeness and clarity.
Better drafts with AI, especially for juniors
After 10 days, all lawyers drafted a patent from a packet of inventor materials for a hypothetical invention. They repeated the task after 90 days with different simulated materials. AI access raised drafting scores by 0.34 standard deviations at 10 days, equivalent to a 10-point climb in percentile ranking among control group scores. At 90 days the improvement was 0.38 SD, or an 11-percentile point increase.
The gains were consistent across all quality dimensions. They came from fewer poor scores and more good scores, with no change in the frequency of excellent scores. The pattern was particularly clear for junior lawyers, whose quality gains came with time savings: 18 minutes faster than the control group's average of 124 minutes on the 10-day task.
The test without AI
At 90 days the researchers added a redlining task: marking up and correcting an existing hypothetical patent containing many substantive and stylistic errors. The authors say this mirrors the professional judgment senior lawyers routinely use and often cannot delegate to AI. No one was allowed to use AI for it, so the scores measured how well skills had developed.
Lawyers with AI access outperformed controls by 0.32 SD on this unassisted task, a 9-point equivalent climb in percentile rankings. The effect was driven entirely by senior lawyers, defined as those with 7 or more years of experience, who outperformed controls by 0.45 SD, a 13-point climb. Juniors, under 7 years of experience, showed no discernible improvement. Their scores bifurcated: more very low scores, fewer mediocre ones, more good ones, and no more excellent ones.
Why seniors and juniors diverged
The authors examined editing patterns. Across both groups, juniors worked sequentially from top to bottom, often spending their time copy-editing low-stakes introductory sections before reaching the main claims. They focused on surface-level synonym swapping rather than improving commercial scope. Even when they spotted serious flaws, they often left comments diagnosing the defect instead of making the fix. Because this "diagnose-without-execute" habit was equally common among unassisted control juniors, the authors describe it as a baseline junior deficit that three months of AI-executed rewrites did not remediate.
Treated seniors spent longer on redlining, skipped low-stakes prose, rebuilt the claims from scratch, removed language that could narrow legal rights, and paired many edits with explicit legal doctrines. In follow-up interviews, seniors described treating AI output as a "logic auditor" rather than a finished product, which the authors say forced them to articulate the why and how of structural edits.
What it means
The authors conclude that improving performance and building durable professional judgment are different goals, and for junior professionals they may be in tension. They suggest foundational expertise may be the missing link that lets AI-assisted repetition turn into seasoned judgment over a career. For firms and trainers, the findings indicate that higher machine-assisted output from junior staff does not by itself mean faster acquisition of the expertise that makes senior professionals distinctly valuable.
Google Research: Does better work always mean better workers?
