Gen Z was promised that AI would be the great equalizer: a tool that lets a 24-year-old produce work like a veteran’s. David Autor’s latest research suggests it may do the opposite.
In a three-month experiment with patent lawyers, AI improved draft quality across the board. But when researchers took the tool away and tested independent judgment, only the experienced lawyers showed an average improvement. The juniors did not.
“Our results challenge the idea of AI being an automatic skill equalizer,” Autor said in written responses to Fortune. “Our data suggest that it’s a performance equalizer, but a skill-disequalizer, in that only practitioners who already had foundational mental models were able to level up their underlying skill sets.”
The study sorted lawyers by experience, not age. “Senior” meant seven or more years in practice, and the researchers did not study Gen Z as a group. But early-career workers are the people the findings bear on most directly.
From the China shock to AI
Autor is head of MIT’s economics department and a Google Technology and Society Visiting Fellow. He is best known for the “China shock,” developed with economists David Dorn and Gordon Hanson. Their research showed that Chinese import competition did lasting damage to American manufacturing communities, which did not adjust as smoothly as optimistic accounts of globalization predicted. In heavily exposed labor markets, wages and labor-force participation stayed depressed, and unemployment stayed elevated, for at least a decade.
More recently, Autor and Hanson have warned of a “China shock 2.0”: Chinese competition in advanced industries that threatens American technological leadership and the high-wage jobs that come with it. Some prominent economists, including Apollo Global Management’s Torsten Slok, have warned that the “AI shock” could have similarities to the China shock in terms of its impact on the U.S. labor force, with Tufts’ Bhaskar Chakravorti even coining the phrase “the wired belt” as a parallel to the rust belt.
Autor pushed back in an appearance on the Possible podcast hosted by LinkedIn co-founder Reid Hoffman, saying AI “will not be, in any sense, a repeat of the China trade shock” because that was experienced by U.S. firms as a “pure negative competitive shock,” but AI will have a “very different texture” in boosting productivity.
The AI debate rests on a version of the assumption the China shock overturned: that once technology takes over routine work, workers will naturally move up to higher-level skills. In his responses to Fortune, Autor said his worry about AI is not mass job loss. “The danger that we may face with AI is not that it eliminates large numbers of jobs in net, but that it eliminates some areas of specialty while creating new ones simultaneously,” he said. “That sounds like a reasonable deal, but the workers losing those old careers are generally not the ones who will be able to take advantage of the new roles that emerge.”
The patent-law experiment looks at a narrower question: whether workers using a powerful new tool are actually building the expertise they need to advance.
Better drafts, uneven learning
Autor and six co-authors published the study as a National Bureau of Economic Research working paper, which has not been peer-reviewed. According to the paper, it enrolled 133 lawyers at 11 U.S. intellectual-property law firms that have ongoing patent-drafting relationships with Google. Researchers randomly assigned access to a custom AI patent-drafting assistant, a then-unreleased Google Labs tool. Two-thirds of the lawyers got access. The rest went without until the study ended.
Google’s role went beyond the tool. The paper lists Autor’s six co-authors as Google employees. It also states that Google conducted the study and paid its direct costs, and that MIT’s human-subjects committee determined MIT was not engaged in the research. In an email to Fortune, Google representatives described the paper as independent research Autor did as part of his fellowship.
Lawyers drafted patents from simulated inventor materials after 10 days and again after 90 days. Patent attorneys at an independent law firm graded the work without knowing which lawyers had used AI, according to the paper. They scored enforceability, accuracy, strategic ambiguity, completeness and clarity.
After 90 days, AI access raised drafting scores by 0.38 standard deviations, the paper found. In a Google blog post accompanying the paper, Autor and co-author Tanya Rodchenko described that as an 11-percentile-point gain relative to control scores. The paper says the improvement came from less weak work, not more excellent work.
The harder test came at the end. Lawyers had to mark up a hypothetical patent that, according to the blog post, contained many substantive and stylistic errors. The paper calls this a task patent lawyers “routinely perform unaided.” No AI was allowed.
Senior lawyers who had AI access beat their control-group peers by 0.45 standard deviations, according to the paper. Junior lawyers showed no average gain.
The paper also reports caveats. Only 91 of the 133 lawyers finished the AI-free test. The firms declined a skills test at the outset, so the researchers inferred skill gains from random assignment instead of measuring before and after. The authors also flagged 15 of the 91 markups as possibly AI-assisted despite the ban, and the flags turned up about as often among control lawyers. With those 15 excluded, the paper says, the senior advantage shrinks to 0.39 standard deviations and holds up, though at a looser statistical threshold. The overall effect does not.
The ‘illusion of competence‘
The juniors’ scores split, the paper found: significantly more poor scores, fewer mediocre ones, more good ones and no gain at the top. The authors write that AI “served as a springboard for some juniors and a cushion for others.”
The blog post describes how the juniors worked. They tended to go top to bottom, polishing introductory text before reaching the main claims. Some spotted serious flaws but left comments describing them instead of fixing them. Control-group juniors did the same, a pattern the authors call “a baseline junior deficit” that three months of AI access did not fix.
The juniors did notice a difference: they liked the tool. In his responses to Fortune, Autor said AI-assisted junior lawyers reported a surge in task satisfaction, the largest effect in the study. They felt that skipping the “blank page problem” let them step straight into a reviewer role, he said.
“Young professionals need to beware of the illusion of competence,” he said. “The only way you’re really going to know if you’re developing skills is if you do tasks without AI assistance and evaluate your performance.”
Why the older lawyers gained
According to the blog post, the experienced lawyers described AI in follow-up interviews as a “logic auditor” and not as a finished product. The authors write that it weakened their attachment to existing prose and forced them to spell out the why and how of their structural edits.
“Arguably, you can use AI as a ‘logic auditor’ only if you already know the law well enough to spot when fluent text is legally flawed,” Autor told Fortune. He said he believes offloading baseline drafting freed seniors to focus on strategic scope, “the forest not the trees.”
On the unassisted test, the blog post says, treated seniors skipped low-stakes prose, rebuilt claims from scratch, cut language that could narrow legal protection and tied many of their edits to legal doctrine.
That doesn’t mean juniors should abandon the tool. Autor told Fortune that novices should work like seniors: frame the argument unassisted first, then use AI as a critic to pressure-test the logic. He acknowledged that is a lot to ask. “Most of us don’t have the self-discipline to do things by hand when a readily available, low-effort tool is sitting right there, offering to do it for us,” he said.
Don’t cut the apprenticeship
For employers, better output creates a temptation to hire fewer juniors. Autor called that short-sighted. “If firms automate away formative practice without replacing it with some kind of guided learning environment, they sever the apprenticeship pipeline that produces tomorrow’s senior partners,” he told Fortune.
He suggested firms decide which tasks can be permanently completed with AI assistance and which judgment tasks require unassisted mastery. They could also pair AI rollouts with regular unassisted skill checks, such as offline redlining exercises, and hold partners accountable for mentoring juniors through the technology’s mistakes. These are ideas, not proven fixes. “It’s early days in this area of research and practice,” Autor said, and few solutions have been robustly proven.
The paper lists other limits. The sample was small and drawn from firms doing ongoing patent work for a single sophisticated client, and three months is brief compared with the years it takes to build patent expertise. The blog post adds that newer models and wider AI familiarity might change the results if the trial were run today. It does not show that AI keeps juniors from becoming experts over a career.
Autor ties the issue back to his China work. Most technological advantage, he told Fortune, comes less from equipment or blueprints than from human know-how, built through slow, laborious mastery. “If we think that AI will relieve us of the burden of mastering expertise, I think we will be sorely disappointed,” he said. “Human intelligence and machine intelligence will be complements for the long term.”
This story was originally featured on Fortune.com
Recent Comments