This week, we talk with Gabe Pereyra, President and co-founder at Harvey, about his path from DeepMind and Google Brain to launching Harvey with Winston Weinberg; how a roommate’s real-world legal workflows met early GPT-4 access and OpenAI backing; why legal emerged as the right domain for large models; and how personal ties to the profession plus a desire to tackle big societal problems shaped a mission to apply advanced AI where language and law intersect.

Gabe’s core thesis lands hard, “the models are the product.” Rather than narrow tools for single tasks, Harvey opted for a broad assistant approach. Lawyers live in text and email, so dialog becomes the control surface, an “AI associate” supporting partners and teams. Early demos showed useful output across many tasks, which reinforced a generalist design, then productized connections into Outlook and Word, plus a no-code Workflow Builder.

Go-to-market strategy flipped the usual script. Instead of starting small, Harvey partnered early with Allen & Overy and leaders like David Wakeling. Large firms supplied layered review, which reduced risk from model errors and increased learning velocity. From there the build list grew, security and data privacy, dedicated capacity, links to firm systems, case law, DMS, data rooms, and eDiscovery. A matter workspace sits at the center. Adoption rises with surface area, with daily activity approaching seventy percent where four or more product surfaces see regular use. ROI work now includes analysis of write-offs and specialized workflows co-built with firms and clients, for example Orrick, A&O, and PwC.

Talent, training, and experience value come next. Firms worry about job paths, and Gabe does not duck that concern. Models handle complex work, which raises anxiety, yet also shortens learning curves. Harvey collaborates on curricula using past deals, plus partnerships with law schools. Return on experience shows up in recruiting, PwC reports stronger appeal among early-career talent, and quality-of-life gains matter. On litigation use cases, chronology builders require firm expertise and guardrails, with evaluation methods that mirror how senior associates review junior output. Frequent use builds a mental model for where errors tend to appear.

Partnerships round out the strategy. Research content from LexisNexis and Wolters Kluwer, work product in iManage and NetDocuments, CLM workflows via Ironclad, with plans for data rooms, eDiscovery, and billing. Vision extends to a complete matter management service, emails, documents, prior work, evaluation, billing links, and strict ethical walls, all organized by client-matter. Global requirements drive multi-region storage and controls, including Australia’s residency rules. The forward look centers on differentiation through customization, firms encode expertise into models, workflows, and agents, then deliver outcomes faster and at software margins. “The value sits in your people,” Gabe says, and firms that convert know-how into systems will lead the pack.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

[Special Thanks to Legal Technology Hub for their sponsoring this episode.]

⁠⁠⁠Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript

Continue Reading The Models Are the Product: Gabe Pereyra on Building an AI Associate and Matter-Centric Workflows

This week we are joined by Patrick DiDomenico and Kevin Klein, two longtime builders of knowledge programs and legal tech gatherings. They walk through the evolution of KM&I for Legal, now entering year three, and the debut of its co-located counterpart, Legal Tech Connect. Both events run in New York on October 22 and 23, with a single community, two distinct agendas, and one big goal, stronger conversations across buyers, builders, and backers of legal tech.

Patrick traces the roots, from the ARK KM era to the launch of KM\&I, then to twisting Kevin’s arm to join as a producer. A larger home opened the door to ambitious programming, Ease Hospitality at 3rd and 40th near Grand Central. Think bright rooms, live plants, strong AV, plenty of seating, and an adjacent tenant lounge with coffee, terrace, and breakout nooks. Lessons from last year show up in smart touches, an overflow room streaming the main stage for those who need to handle calls or email without missing core content, longer breaks for real conversations, and, yes, food worthy of repeat trips.

Format matters here. KM&I holds firm on peer-to-peer sessions led by law firm professionals. Providers participate through tight five-minute spotlights between talks, plus optional demo rooms during generous coffee breaks and lunch. Legal Tech Connect flips the lens, product stories on stage, founder journeys, market forces, and regulatory themes. A crossover ticket lets attendees roam freely between both programs. Breakouts return by popular demand, a C-suite roundtable, KM 101 for newcomers, a track for KM attorneys and PSLs, and a managers and directors forum that grew from attendee feedback.

Themes thread across both days. ROI from AI and KM tools appears throughout, from data strategy as a differentiator to co-development case studies. Expect a lively take on the rise of the legal engineer, with skills for scaling tools and driving adoption, plus a frank discussion about where these roles sit inside firms. Professor Michele DiStefano opens with client centricity, drawing on her research and book, with every attendee receiving a copy. She then moderates a session at Legal Tech Connect on how legal tech companies sell to law firms, bridging provider goals with buyer needs. Another panel stages the AI conversation among a partner, pricing director, client, and innovation lead, a timely look at value, billing, and collaboration.

The bigger story is community. Patrick and Kevin highlight the peer network that forms in hallways and over coffee, mentors found by chance, and ideas that travel home in workable form. Legal Tech Connect brings investors and founders into the mix, which raises the quality of dialogue on funding, product focus, and adoption. Looking ahead, they predict fewer conferences, higher quality bars, and a shift toward substance over appearance. Listeners who want more details, including registration, should visit kmniforlegal.com and legaltechconnect.com. The two events sit side by side in October, and the goal is simple, leave with practical ideas, new contacts, and a clearer view of where legal innovation heads next.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

[Special Thanks to Legal Technology Hub for their sponsoring this episode.]

Blue Sky: ⁠@geeklawblog.com⁠ ⁠@marlgeb⁠
⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Continue Reading KM&I Meets Legal Tech Connect: Two Tracks, One Community with Patrick DiDomenico and Kevin Klein

In the latest episode of “The Geek in Review” podcast, co-hosts Greg Lambert and Marlene Gebauer continue their series of interviews from this year’s LegalWeek conference by engaging with Joshua Lenon from Clio. As Clio’s Lawyer in Residence, Lenon delves into the insights derived from the company’s Legal Trends Report, emphasizing the data-driven analysis of law firm practices and trends. The report, which leverages aggregated anonymous usage data from tens of thousands of law firms using Clio, offers a unique perspective on the productivity, billing, and technological adoption within the legal industry, particularly among mid-sized law firms.

Lenon shares intriguing findings from the report, highlighting a significant increase in productivity, billable hours, and revenue across the industry over the past eight years. However, a closer examination of mid-sized law firms (defined as those with 20 to 200 lawyers) reveals disparities in matter handling and productivity gains compared to smaller firms. Lenon explains how mid-sized firms maintain a consistent workload per lawyer by adjusting the ratio of lawyers to non-lawyer timekeepers based on demand. This adaptability showcases the strategic management of resources within mid-sized firms to optimize efficiency and service delivery.

The discussion further explores the impact of financial technology (FinTech) on law firms’ operational efficiency. Lenon illustrates how adopting new payment methods and technologies significantly improves firms’ collection rates and client payment experiences. Specifically, mid-sized firms that embrace FinTech and client-centered approaches see notable improvements in their financial health, underscoring the importance of innovation in enhancing legal services delivery.

Lenon also introduces Clio Duo, an in-house AI tool designed to enhance law firms’ access to and interaction with their own data. By providing a chat-based interface for exploring firm data, Clio Duo aims to streamline tasks and improve efficiency, allowing lawyers to focus on high-value work rather than administrative tasks. This development represents Clio’s commitment to leveraging technology to address the evolving needs of the legal profession.

Lenon predicts some of the challenges and opportunities facing the legal profession, particularly in relation to artificial intelligence (AI) and document creation tools. He speculates on the limitations imposed by traditional word processing applications like Microsoft Word and suggests that the future will likely see a paradigm shift towards more interactive and multi-dimensional tools for legal work. This shift, Lenon argues, could dramatically enhance productivity by integrating AI more seamlessly into the legal drafting process, moving beyond the static, page-focused approach of current software.

Throughout the conversation, Lenon’s insights underscore the dynamic interplay between technology and legal practice. As legal technology evolves, so too does the potential for law firms of all sizes to improve efficiency, client satisfaction, and ultimately, profitability. The episode illuminates the importance of data-driven decision-making and technological adaptation in the legal industry’s future. By embracing these tools and insights, law firms can better navigate the challenges of the modern legal landscape, ensuring they remain competitive and responsive to their clients’ needs.

Listen on mobile platforms:  ⁠⁠⁠Apple Podcasts⁠⁠⁠ |  ⁠⁠⁠Spotify⁠⁠⁠ | ⁠⁠YouTube⁠⁠

Contact Us: 

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert
⁠⁠⁠⁠⁠Threads: @glambertpod or @gebauerm66
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠Transcript

Continue Reading LegalWeek 2024 Special Part Four: Joshua Lenon from Clio

This week we catch up with Jeff Pfeifer and Serena Wellen from LexisNexis to discuss the rapid development of AI tools for the legal industry over the past year. Pfeifer and Wellen give us an insider’s view of what it took to bring their Lexis+ AI tool to the market and the balance between speed to market and getting solid customer guidance on what they need in a legal-focused Generative AI tool. Between the initial version released to a select group of customers and the current version, the product grew from an open-ended chat interface into more of a guided resource that helps users on creating and following up on prompts. As with most AI tools created in the past year, there is still more potential as more and more customers use it and give critical feedback along the way.

In addition to Lexis+ AI, LexisNexis has now launched two additional AI products – Lexis Snapshot and Lexis Create. Lexis Snapshot summarizes legal complaints to help firms monitor litigation. Lexis Create brings AI capabilities directly into Microsoft Word to assist with drafting and research while lawyers are working on documents. The goal is to embed insights where lawyers are actually doing their work rather than separate AI tools.

While the focus of the initial Generative AI tools from LexisNexis were focused on the US market, Serena Wellen and her team are busy expanding that to more of an international reach. This requires adapting the models, content, and interface to different languages and legal systems. This is complex undertaking, but Wellen discusses how LexisNexis has content and editors around the world to help customize the tools. Surprisingly, desired use cases are fairly consistent globally – both simple legal tasks as well as more advanced legal research and drafting.

Greg Lambert brings up a recent LinkedIn discussion that he had with Microsoft’s Jason Barnwell, where Barnwell told him that today’s version of Generative AI tools are “the worst these things will ever be.” In response, Pfeifer says that LexisNexis is focused on continuously improving answer quality to build trust and prove the value of AI to skeptical lawyers. LexisNexis is leveraging relationships with companies like Microsoft to reinforce the stability and progress being made.

Wellen and Pfeifer look into the future and predicted that AI assistants will become highly personalized to individual lawyers. AI agents will also proliferate across platforms to help automate tasks and workflows. Law firms will likely accelerate their adoption of AI tools based on rising expectations and demands from corporate legal departments to work more efficiently.

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠ | YouTube

⁠⁠⁠⁠Contact Us: 

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠
Threads: @glambertpod or @gebauerm66
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

Transcript

Continue Reading Pfeifer and Wellen Give an Inside Look at LexisNexis’ AI Sprint (TGIR Ep. 230)

This week, we cover the highlights of the recent Knowledge Management and Innovation for Legal Conference with our guests, Patrick DiDomenico, founder of Inspire KM Consulting and the organizer of the conference; Tanisha Little, Director of Knowledge Management at Simpson Thatcher; and Sara Miro, Director of Knowledge at Sullivan Cromwell. In a time of Generative AI Hype, the basic ideas behind Knowledge Management have never been more relevant. As much as we believe in the future of a technology that can build upon our current knowledge, how we structure and manage that knowledge will determine just how far we can go. Knowledge management has never been more important than it is today.

The episode begins with an exploration of the organizational complexities behind the inaugural conference. Patrick DiDomenico shares his experiences in orchestrating the event, noting the extensive, months-long commitment it entailed, yet expressing satisfaction with the innovative elements introduced, such as breakout sessions.

We then shift focus to the specific sessions led by our guests. Tanisha Little and Jennifer Mendez conducted a “KM 101” session, effectively orienting newcomers to the field. Additionally, an engaging KM Attorney Roundtable, facilitated by Sara Miro and Patrick Dundas, explored issues such as change management and the implications of generative AI.

Our discussion also covers the most impactful sessions from the conference. Key highlights include Andrea Alliston’s keynote on leadership amidst disruption, Jeff Rovner’s presentation on succession planning, and Mark Smolik’s perspective on aligning law firms with client needs. The consensus underscores the conference’s comprehensive value for professionals across all levels of KM expertise.

A significant aspect of the conference was the notable influx of newcomers to the KM field. Emphasizing the importance of foundational knowledge, Patrick DiDomenico notes that such conferences often attract a substantial proportion of first-time attendees. The provision of introductory content is pivotal for equipping these professionals for more advanced discussions in future gatherings.

For our Crystal Ball Question, there is a unanimous agreement on the transformative potential of generative AI, foreseeing an increase in KM specialization, refined use cases, and enhanced emphasis on data curation. Nevertheless, mastering foundational KM practices, such as change management, is deemed essential for fully capitalizing on these technological advancements. The episode concludes with an optimistic outlook on the continuous growth and evolution in the field of KM.

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠ | YouTube

⁠⁠Contact Us: 

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠
Threads: @glambertpod or @gebauerm66
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

Transcript

Continue Reading The Future of KM is Bright: DiDomenico, Miro, and Little Review the KM&I for Legal Conference (TGIR Ep. 229)

In this episode of The Geek in Review, hosts Greg Lambert and Marlene Gebauer interview three guests from UK law firm Travers Smith about their work on AI: Chief Technology Officer Oliver Bethell, Director of Legal Technology Shawn Curran, and AI Manager Sam Lansley. They discuss Travers Smith’s approach to testing and applying AI tools like generative models.

A key focus is finding ways to safely leverage AI while mitigating risks like copyright issues and hallucination. Travers Smith built an internal chatbot called YCNbot to experiment with generative AI through secure enterprise APIs. They are being cautious on the generative side but see more revolutionary impact from reasoning applications like analyzing documents.

Travers Smith has open sourced tools like YCNbot to spur responsible AI adoption. Collaboration with 273 Ventures helped build in multi-model support. The team is working on reducing dependence on manual prompting and increasing document analysis capabilities. They aim to be model-agnostic to hedge against reliance on a single vendor.

On model safety, Travers Smith emphasizes training data legitimacy, multi-model flexibility, and probing hallucination risks. They co-authored a paper on subtle errors in legal AI. Dedicated roles like prompt engineers are emerging to interface between law and technology. Travers Smith is exploring AI for tasks like contract review but not yet for work product.

When asked about the crystal ball for legal AI, the guests predicted the need for equitable distribution of benefits, growth in reasoning applications vs. generative ones, and movement toward more autonomous agents over manual prompting. Info providers may gain power over intermediaries applying their data.

This wide-ranging discussion provides an inside look at how one forward-thinking firm is advancing legal AI in a prudent and ethical manner. With an open source mindset, Travers Smith is exploring boundaries and sharing solutions to propel the responsible use of emerging technologies in law.

Links:

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠ | YouTube (NEW!)

Contact Us:

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠
Threads: @glambertpod or @gebauerm66
Voicemail: 713-487-7821 Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠⁠TranscriptContinue Reading Deploying Cutting-Edge Legal AI: Travers Smith’s Cautious, But Open-source Approach. (TGIR Ep. 216)

In this episode of The Geek in Review, hosts Marlene Gebauer and Greg Lambert interview Laura Leopard, founder and CEO of Leopard Solutions, about succession planning challenges facing law firms. Leopard explains that many firms have partners nearing retirement age but no concrete plans for transitioning clients and leadership. This lack of succession planning threatens law firms’ futures.

Laura mentions that to make matters worse, the path to equity partnership is getting longer, making it harder to retain promising senior associates and counsel. Firms have added non-equity partner roles, keeping equity partner numbers small to inflate profits per partner. Leadership lacks incentives to retire, with no retirement plans or continued compensation. All this will hamper recruiting efforts, as younger generations prioritize work-life balance.

She recommends that in order to retain mid-career attorneys, firms must rethink policies on remote work, billable hours, and flexibility. Virtual firms with better lifestyle offerings are growing competitors. But firms seem unwilling to change. Leopard argues everything should be on the table for analysis by outside consultants. Phased retirements and succession mentoring could also help transition clients and power.

Though Laura Leopard (and even Bruce MacEwan) cannot point to examples of firms that have executed succession planning well, it is possible with courageous leadership. She advises setting retirement age limits, crafting written plans, and easing older partners’ exits. A too-big-to-fail mentality persists despite serious business vulnerabilities if talent is not retained and recruited.

Looking ahead, Leopard predicts the rise of virtual firms will shake up the legal industry as they encroach on Big Law territory with alternative fee arrangements. The pandemic accelerated dissatisfaction with law firm partnership and policies. As generational divides grow, flexible virtual firms will keep gaining ground over more rigid large firms.

This engaging discussion unpacks the complex dynamics around law firm succession planning and existential threats posed by lack of preparation. As partners cling to power, can bold leaders emerge to implement creative solutions and secure these institutions’ longevity? Tune in for an insightful examination of forces reshaping the legal landscape.

Links:

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠ | YouTube (NEW!)

Contact Us:

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠
Threads: @glambertpod or @gebauerm66
Voicemail: 713-487-7821 Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠⁠Transcript

Continue Reading Laura Leopard on Law Firms’ Current Succession Planning: Step One – Do Nothing (TGIR Ep. 215)

This is part 3 in a 3 part series.  Part 1 questions Goldman’s Sachs data showing that 44% of of legal tasks could be replaced by Generative AI.  In Part 2, we find some better data and estimate an upper limit of 23.5% of revenue that could be reduced by Generative AI. All of our assertions and assumptions will be discussed in further detail in a free LVN Webinar on August 15th.

The Big Idea:  We apply reductions in hours due to Generative AI to a few matters to determine Generative AI’s potential effect on profitability.
Key Points:
  • We establish a baseline sample matter and compare changes to that sample matter when Generative AI is applied
  • We explore how leverage is affected by Generative AI and how those changes may affect profitability in unexpected ways

Determining Generative AI’s effect on law firm profitability requires a bit more than a “back of the napkin” calculation with rough percentages based on keywords in time entries, as we did when roughly calculating the effect on revenue.

As Toby pointed out at the end of the last post, Generative AI is unlikely to hit all timekeepers equally.

We begin with this assertion.

Generative AI will disproportionately impact non-partner hours.

We are comfortable making this assertion for two reasons:

  1. Generative AI, in its current state, is most likely to replace or shorten the time to complete lower complexity and lesser specialized tasks that should be performed at the associate or paralegal level.
  2. Any time legal work hours are reduced, Partners tend to protect their own hours.

With that in mind, Toby began a profitability analysis, beginning with a baseline sample matter that does not factor in any use of Generative AI. We will use this baseline to compare against our AI adjusted matters.


Baseline M&A Sample Matter Data

Our baseline sample matter is loosely modeled on an M&A transaction and includes 5 timekeepers:

  • an Equity Partner
  • a 17th year service partner
  • 10th, 7th and 3rd year associates
TK Hours Rate Realization Revenue Expense Profit
EP 80 $1,000 88% $70,400 $15,200 $55,200
SP17 100 $895 88% $78,760 $48,500 $30,260
10yr 125 $735 88% $80,850 $48,750 $32,100
7yr 90 $660 88% $52,272 $32,400 $19,872
3yr 55 $595 88% $28,798 $18,150 $10,648

Estimated Annual Profit Per Equity Partner (PPEP) – $1,851 X 1400 hrs = $2,591,400

Leverage – 60% Non-Partner Hours


There are, of course, a number of assumptions in this baseline data that could greatly change from firm to firm, including the billable rates, the realization rate, and the expense for each timekeeper. However, we will keep this baseline data consistent across all of our examples in order to make a fair comparison. With different rates, realization, and expenses you will get different results. We strongly encourage every firm to perform a similar calculation for themselves.

Baseline Matter Analysis

The total hours billed are 450. The total revenue is $311k and the total profit in dollars is $163k.

Our model then translates the profit on this one matter into an estimated PPEP number for the firm. This is so we can determine profit margin impact separate from profit dollars.

In this baseline model, the PPEP number is ~$2.6m; meaning that if all work at this firm were staffed and billed like this one matter, the firm average PPEP would be about $2.6m.

Leverage

There’s an old adage in economic circles: “Workers Work. Owners Benefit.”
Continue Reading AI-Pocalypse: The Shocking Impact on Law Firm Profitability

by 3 Geeks (Ryan McClead, Greg Lambert, and Toby Brown)

This is part 2 in a 3 part series. The first part is here. Part 3 is here.

The Big Idea: We found a much better dataset, though still small, from which to extrapolate actual effects of Generative AI on the legal industry.

Key takeaways:

  • We got anonymized and summarized data for 10 corporate legal departments from LexisNexis CounselLink
  • The data showed that almost 40% of time entries, representing 47% of billings, could potentially use Generative AI.
  • We estimate that a realistic initial upper limit for Generative AI would be to reduce that work by half, or 20% of time entries and 23.5% of revenue

In the previous post, Ryan got tired of hearing the Goldman Sachs “44% of Legal is going away” stat being quoted uncritically and decided to actually look into the underlying data used in their report. Ryan’s exploration of the data is an interesting story in and of itself, but the bottom line is that the data is fuzzy at best, the sample size is laughable, and the breathlessly unquestioning reporting on Goldman’s study has been remarkably sloppy.

After writing up his findings, Ryan shared that post with Greg and Toby, and the question quickly arose, “can we find some actual, useful data to better understand the effect that Generative AI might actually have on law firms?” Gregreached out to Kris Satkunas from LexisNexis CounselLink, a recent interviewee on the Geek in Review, to see if CounselLink could share some anonymized benchmark data for us to analyze.

LexisNexis CounselLink Data

As a reminder the Goldman data was using survey questions about how important certain “work tasks” were for their jobs. Those tasks included things like “Getting Information”, “Identifying Objects, Actions, and Events”, and “Scheduling Work and Activities”. These are quite vague and wide open to interpretation.

In an attempt to find more useful data for our purposes, we asked Kris for the percentages of all time entries that included the keywords “Draft” or “Review” in the description. Our assumption is that those two terms will capture a large percentage of actual time entries in which lawyers are likely to use Generative AI. We fully recognize that this simple heuristic will not produce a clean data set from which to extrapolate definitive results, but as a first pass at some real data, we believe this gives us a nice estimate of tasks that could potentially be ripe for automation with Generative AI.
Continue Reading Generative AI Could Reduce Law Firm Revenue by 23.5%