A recent, widely circulated report, “Benchmarking Humans & AI in Contract Drafting” (Guo, Rodrigues, Al Mamari, Udeshi, and Astbury, September 2025), made headlines with a striking claim: 97% of lawyers use generative AI for legal work. While it is no secret that AI has become increasingly prevalent in legal practice, this figure suggests a true watershed moment where the technology has essentially reached “universality.” But when we explore the underlying methodology, a number of important concerns emerge and put in doubt the now oft-quoted 97% figure and possible claims of “universality.”
The study, based on research conducted in early 2025, was primarily a benchmarking exercise that compared the outputs of 13 AI tools against a group of in-house lawyers across 30 drafting tasks. This portion of the study, which found that top-performing AI tools matched or even outperformed human lawyers on first-draft reliability while lagging slightly on usefulness, is valuable. It provides concrete findings regarding the areas of law where AI outperforms, or still falls short of, human attorneys.
However, the widely cited “97% of lawyers use AI” statistic comes from a survey of just 72 respondents. Given that recent estimates place the total number of lawyers in this country at over a million, a sample of fewer than 100 may not be representative of the broader population.
The report did not disclose the demographic profile of the law firms/lawyers participating (e.g., the size of the law firm, corporate vs. consumer practice, law firm vs. in-house) or the recruitment methods used to source the 72 respondents included in the study. This left key questions about the reliability of the bold 97% statistic, especially given the small sample size. When contacted by The National Law Review (NLR), the report’s authors stated that the responses were obtained through “direct outreach, LinkedIn, and a practice-community network.” The authors further acknowledged “a risk of selection bias,” noting that lawyers already engaged with AI may have been more willing to respond.
In an email response to NLR, the authors reported their study included a diverse cohort of lawyers spanning 24 jurisdictions, and that a more comprehensive demographic overview of the study participants would be publicly released in the coming weeks, which may provide helpful nuance. Until then, however, a conclusion that AI use among lawyers has reached universality should be treated with healthy skepticism. In an era of frequent claims about technology now surpassing once-thought unreachable benchmarks (e.g., the passing of the Turing Test, reaching AGI, reaching superintelligence, and quantum supremacy), any new claim of this ilk should be based on a strong statistical foundation using best-in-class experimental methodologies. For claims to be accepted, the conclusion should be supported by multiple studies by different research groups and be of sufficient scale. Importantly, the results for any experiment should be demonstrated to be replicable.
The authors also acknowledged that the number of lawyers contacted for the survey was not recorded, pointing to the possibility of self-selection bias (i.e., a disproportionate number of lawyers who do not use AI may have simply chosen to opt out of the study for any number of reasons, including not wanting to appear out of date or due to embarrassment).
Moreover, “universality” itself has multiple dimensions. Does it refer to a lawyer who has tried a generative AI tool once, one who relies on it daily, or one who integrates it into substantive client work? The concept of “universality” has little value without a clear definition of what constitutes use.
While these limitations weaken the claims of universality, they do not diminish the value of the benchmarking work presented in the report. The report offers important and timely findings regarding how AI compares to humans in terms of reliability, usefulness, and workflow support. Still, adoption numbers should be treated with caution. While the study may offer interesting insights into how AI is being used by early adopters, without larger and more representative samples, adoption figures risk overstating the pace of change.