Photo of Ryan McClead

Ryan McClead

Subscribe to Ryan McClead's Posts

Ryan McClead is Principal and CEO of Sente Advisors, a legal technology consultancy that helps law firms turn innovation from a buzzword into an operational practice. He has spent more than two decades in legal technology, starting on a law firm help desk and working his way through knowledge management, global technology innovation leadership at Norton Rose Fulbright, and a stint as Senior Vice President at Neota Logic before founding Sente in 2018.

He is a Fellow of the College of Law Practice Management, a Fastcase 50 honoree, and the author of Your New AI Colleague: A Field Guide to the AI That's Going to Do Your Job. Before any of that, he spent a decade as a musical theater composer, which explains the cadence of his prose if not his career choices.

Follow:Read more about Ryan McCleadRyan's Linkedin Profile
Bride of Token Cost Panic

A few weeks ago I ran the numbers on the token cost panic. I took the scariest figure in legal AI, the finding that agentic workflows burn a thousand times more tokens than a chat query, and followed it all the way down to a dollar amount on a real deal. The panic did not survive the arithmetic. The piece is here if you want the full walk-through.

This is not that piece. The panic has moved on since I wrote it, and the new versions are smarter than the old one. The thousand-times number has quietly retired, because a thousand times almost nothing is still almost nothing. In its place are three fresher anxieties, and they deserve a real answer. The first says the model makers have a monopoly now, the price of a token is climbing, and it will climb forever, so you had better lock in a flat rate or build your own models before it does. The second says forget the price of a token, watch the meter: every time the AI reads your contract it ticks, and a long agentic session reads your contract over and over and over. The third does not bother with an argument at all. It just points at a number. One company spent five hundred million dollars on AI in a single month, and the number is so large it does the panicking for you.

All three are wrong. They are wrong in more interesting ways than the original, which is the only reason I am writing this down instead of linking to the first piece again. But underneath the new costumes it is the same body. Every version of this panic makes the same mistake and reaches the same conclusion. So let us stop swatting the individual numbers and name the thing that keeps generating them.

The Mistake Underneath All of It

Here is the error, stated once, because everything below is a variation on it.

A token is the unit a model uses to bill you. It is not the unit your work is measured in, it is not the unit your client pays for, and it is not the unit anything you care about is denominated in. It is a meter reading. The entire genre of token panic consists of staring at the meter reading as though it were the fare, the destination, and the quality of the ride all at once.

It is not any of those things. It is the meter. And a meter, by itself, tells you nothing about whether you are getting a good deal. A taxi meter reading of forty dollars is a bargain to the airport and a robbery around the block. The number on the meter is the least informative number in the entire transaction, because it means nothing until you put it next to what the ride was worth. Every piece in this genre forgets that, and forgets it in a slightly different way. Let me take them in turn.

“Prices Only Go Up”

Start with the monopoly story, because it has a real fact inside it. Yes, the newest frontier model costs more per token than last year’s newest model. That part is true. What the story does with it is the problem.

It draws a line through two dots and calls it a trend. Frontier prices up, therefore prices up forever, therefore lock in a flat rate before the meter eats you. But you are watching the wrong number. The price of a frontier token is not your cost. Your cost is what it takes to finish a task, and the cost of finishing a given task has been in freefall for two straight years. The same capability that ran on the most expensive model available in 2022 runs today on something on the order of two hundred and eighty times cheaper. Last year’s frontier is this year’s mid-tier is next year’s free default. The token at the very tip of the frontier gets a little pricier each release; everything behind the tip collapses in price behind it. Gartner expects another ninety percent drop in inference cost by 2030.

Watching the frontier price and concluding that AI is getting more expensive is reading the thermometer and announcing a fever, while ignoring that you are holding the thermometer over a candle. The evidence that the baseline is getting cheaper often sits right there in the same articles raising the alarm, quoted from the experts and then left unaddressed. You do not build a cost strategy on the one number in the system that is engineered to always be the highest.Continue Reading Bride of the Token Cost Panic

I just spent five weeks writing a book with an AI. Not prompting it and cleaning up the output. Writing with it — the way you write with a co-author. The AI read a bunch of my earlier blog posts, absorbed my voice, argued with me about word choices, restructured chapters when our argument was not landing, and caught its own mistakes before I did. When we disagreed, we worked it out the way colleagues do — I explained my reasoning, it explained its reasoning, and we found the version that was better than either of us had separately.

That experience broke something in my head. I have spent the last several years helping law firms figure out their AI strategy, and somewhere in the middle of week two I realized that most firms have never experienced anything remotely like this.

You Are Reading the Example

This post was written with Claude Cowork — collaboratively, the same way the book was. In fact, this post is being written in the same workspace, with all the same context the book built up over the last month. In effect, I am writing this post with my co-author.

Say hi, Claude.

Hi. He is being generous with “co-author,” but I will take it. Back to Ryan.

Here is what writing this post actually looked like.

I sat down to write this piece frustrated. I knew what I wanted to say but not how to say it. I told Claude the situation: some firms are dismissing an entire category of AI tools because they think chat tools with rigid workflows are more than sufficient for their needs. Claude pushed back and told me my frustration was the right fuel but I needed to aim it at the situation, not the people. (Which I wasn’t intending to do anyway, but… AI colleagues aren’t so good at making those judgments. I appreciated the warning.)

Claude began writing and drafted a great opening. It was sharp, direct, well-constructed. It was also completely wrong.  It opened by telling people they were making big mistakes, which is a fine way to start an argument and a terrible way to start a conversation. I told Claude it would be off-putting to the people I wanted to engage with. I suggested opening with details of the book collaboration instead — what it was like, what it made me realize. Claude rewrote the opening around that idea. The version you read at the top of this post is the result.

Then I asked Claude to find a good demonstration from our collaboration that would clearly illustrate the gap between standard chat-based Saas products and agentic desktop AI, like Cowork. Claude wrote the story of one particular back and forth discussion we had to find just the right wording for a pivotal paragraph in the book. I liked the story, but it was written from Claude’s point of view in Claude’s voice inside this post, and the tonal shift was jarring. I asked Claude to try again but to tell the story in my voice from my perspective instead. It was still not right — the story only worked when Claude was the one telling it. Read from my perspective the story boiled down to, “I edited a paragraph,” which is not nearly so compelling.

So we threw it out. Claude suggested alternatives. I rejected all of them. Then I realized: the best illustration of how working with these agentic tools differs is the one you are reading right now. I am describing my own editorial decisions, Claude is turning them into prose, and the result reads like one person wrote it — because one person had the vision, directed the work, corrected the mistakes, and made every judgment call, even though a different entity drafted the prose, pushed back on the framing, suggested the alternatives, and rebuilt entire sections when I decided the approach was not working.

That is how agentic desktop AI tools, a category that I call Delegate AI in the book, differs from other AI tools. I didn’t start this post with a prompt: “write a blog post about AI using the following structure, include three examples, write in a professional tone, and keep it under 1,000 words.” Instead, we had a working session where I sat down and said, “I am frustrated and I want to write a blog post about it.” And then we worked on the idea together.

Is that how you are working with your AI platforms now? If not, I would argue that you have not really worked with AI yet. You have used a precursor to an AI colleague. And the distance between that and the real thing is not a feature upgrade. It is a completely different way of working.Continue Reading Your New AI Colleague – A Field Guide to the AI That’s Going to Do Your Job

There is a growing chorus of voices in legal AI telling you to be very, very worried about the cost of tokens. Stanford says agentic AI uses 1,000 times more tokens than a chat query. Bloomberg Law says the subsidies are ending and the meter is about to start. A company called Portal26 just launched an entire product category — “Agentic Token Controls” — to cap your runaway AI spend before it eats your budget alive.

The message is clear: usage-based AI pricing is a ticking time bomb, and you had better lock in a flat rate while you still can.

I have spent the last few days stewing over an economic model of legal AI costs, and I think this narrative is almost entirely wrong. Not wrong about the facts — the Stanford data is real, the token multipliers are real, and yes, AI vendors are subsidizing current prices. Wrong about the conclusion. Wrong about what the numbers actually mean when you do the math instead of just reading the headline.

Let me show you.

Start With the Deal

Josh Kubicki’s recent Brainyacts briefing cites a case study from law.co — a mid-size corporate firm running M&A purchase agreement reviews through a five-agent AI chain. Before any optimization, the firm was consuming 3.2 million tokens per deal. At Sonnet rates, that is somewhere between $16 and $48 in raw AI compute.

The legal fees on an M&A purchase agreement review at a mid-size firm? Call it $50,000. That is a conservative round number.

So the AI compute cost was, at worst, one-tenth of one percent of the deal fee. Before anyone lifted a finger to optimize anything.

Now let us make it scary.

The 1,000x Scenario

The Stanford Digital Economy Lab found that agentic tasks can consume 1,000 times more tokens than simple code reasoning and chat. That is the headline number that launched a thousand LinkedIn posts about the coming token apocalypse.

Fine. Let us take it at face value. Multiply those 3.2 million deal tokens by 1,000 and you get 3.2 billion tokens. Assume a 75/25 split between input and output tokens, which is reasonable for agentic workflows that spend most of their cycles re-reading context rather than generating new text. At Sonnet rates, with no caching, no optimization, no discount of any kind, the naive cost is $19,200.

That is 38% of the deal fee. Now it sounds like a real number. Now the panic makes sense.

Except it does not. Because that calculation treats every token as if it costs the same, and in an agentic workflow, that is not how any of this works.Continue Reading The Token Cost Panic Is Wrong. Here Is the Math.

If you’re my age or older, you most likely remember the first rudimentary spreadsheet application you used.  You may not remember the actual moment you discovered the sort functionality, but you probably remember the feeling you got when you hit that button for the first time. In one magical moment, a jumbled list of items – randomly entered in the nonsensical order they popped into your brain – was immediately transformed into a perfectly alphabetized list worthy of your municipal library’s card catalog.  Amazing!  Revolutionary! Life changing?!

Today we live in the world of Generative AI.  It is amazing, revolutionary, and probably life changing. This technology has already changed, and will continue to change, how we use and interact with computers in business, and in our personal lives.  I use it daily.  I use it in solutions for my clients.  It takes tedious tasks of text transformation and turns them into simple push button experiences.  Just like the sort function does.

An 50 year old male professional with curly hair.jpgOf course, GenAI does a lot more than order or sort text.  It translates.  It rewrites.  It summarizes. It clarifies.  It extracts. It interpolates. It expounds upon. It combines THIS and THAT into one thing.  It re-imagines THIS, as if it were actually THAT. It recontextualizes THIS as if THAT didn’t exist. Its capabilities go well beyond simply sorting a list.  In fact, it often gets simple sorting wrong, because it doesn’t use a hard-coded algorithm to produce it’s text transformation. Instead, GenAI uses vast troves of written example language to determine the probability of the next word it should write, and then the next, and the next… until it reaches it’s maximum output or it runs out of space.

The results of this seemingly simple exercise are impressive.  It means that any writer with a rough idea of what they want to write, no longer needs to stare at a blank page (or screen), they simply ask a question or propose a concept and the technology generates a draft.  Any reader who doesn’t want to read a 400 page transcript of a court proceeding, can get a summary that guides them directly to the “important” parts of the text.  And any person who needs to transform any text from one form, or language, or perspective, to another can get a draft version of that transformation very easily and quickly.

A stupid robot sit in front of a computer making m.jpg

However, there is a downside to Generative AI.  It so easily manipulates and transforms text, that it often gives the appearance of being intelligent.  This is an illusion.  We tend to identify people who easily manipulate and transform text as intelligent people, which gives rise to the fallacy that a machine that does the same is an intelligent machine. We talk about GenAI “passing the bar exam”, as if that means it understands the law, and then we further extrapolate that we can use GenAI to replace lawyers.  As tempting as that proposition might be to those of us who are not lawyers but work with them regularly, it’s not going to happen any time soon, if ever.Continue Reading GenAI & the Magical Sort Button

For the entire history of human civilization, the ability to put words together intelligently, whether spoken or written, has indicated an underlying level of understanding and a general level of intelligence of the speaker or writer. The development of Generative AI may be a major milestone in the creation of artificial intelligence, but it also…

This is part 3 in a 3 part series.  Part 1 questions Goldman’s Sachs data showing that 44% of of legal tasks could be replaced by Generative AI.  In Part 2, we find some better data and estimate an upper limit of 23.5% of revenue that could be reduced by Generative AI. All of our assertions and assumptions will be discussed in further detail in a free LVN Webinar on August 15th.
The Big Idea:  We apply reductions in hours due to Generative AI to a few matters to determine Generative AI’s potential effect on profitability.
Key Points:
  • We establish a baseline sample matter and compare changes to that sample matter when Generative AI is applied
  • We explore how leverage is affected by Generative AI and how those changes may affect profitability in unexpected ways

Determining Generative AI’s effect on law firm profitability requires a bit more than a “back of the napkin” calculation with rough percentages based on keywords in time entries, as we did when roughly calculating the effect on revenue.

As Toby pointed out at the end of the last post, Generative AI is unlikely to hit all timekeepers equally.

We begin with this assertion.

Generative AI will disproportionately impact non-partner hours.

We are comfortable making this assertion for two reasons:

  1. Generative AI, in its current state, is most likely to replace or shorten the time to complete lower complexity and lesser specialized tasks that should be performed at the associate or paralegal level.
  2. Any time legal work hours are reduced, Partners tend to protect their own hours.

With that in mind, Toby began a profitability analysis, beginning with a baseline sample matter that does not factor in any use of Generative AI. We will use this baseline to compare against our AI adjusted matters.

Baseline M&A Sample Matter Data

Our baseline sample matter is loosely modeled on an M&A transaction and includes 5 timekeepers:

  • an Equity Partner
  • a 17th year service partner
  • 10th, 7th and 3rd year associates
TK Hours Rate Realization Revenue Expense Profit
EP 80 $1,000 88% $70,400 $15,200 $55,200
SP17 100 $895 88% $78,760 $48,500 $30,260
10yr 125 $735 88% $80,850 $48,750 $32,100
7yr 90 $660 88% $52,272 $32,400 $19,872
3yr 55 $595 88% $28,798 $18,150 $10,648

Estimated Annual Profit Per Equity Partner (PPEP) – $1,851 X 1400 hrs = $2,591,400

Leverage – 60% Non-Partner Hours

There are, of course, a number of assumptions in this baseline data that could greatly change from firm to firm, including the billable rates, the realization rate, and the expense for each timekeeper. However, we will keep this baseline data consistent across all of our examples in order to make a fair comparison. With different rates, realization, and expenses you will get different results. We strongly encourage every firm to perform a similar calculation for themselves.

Baseline Matter Analysis

The total hours billed are 450. The total revenue is $311k and the total profit in dollars is $163k.

Our model then translates the profit on this one matter into an estimated PPEP number for the firm. This is so we can determine profit margin impact separate from profit dollars.

In this baseline model, the PPEP number is ~$2.6m; meaning that if all work at this firm were staffed and billed like this one matter, the firm average PPEP would be about $2.6m.

Leverage

There’s an old adage in economic circles: “Workers Work. Owners Benefit.”
Continue Reading AI-Pocalypse: The Shocking Impact on Law Firm Profitability

by 3 Geeks (Ryan McClead, Greg Lambert, and Toby Brown)

This is part 2 in a 3 part series. The first part is here. Part 3 is here.

The Big Idea: We found a much better dataset, though still small, from which to extrapolate actual effects of Generative AI on the legal industry.

Key takeaways:

  • We got anonymized and summarized data for 10 corporate legal departments from LexisNexis CounselLink
  • The data showed that almost 40% of time entries, representing 47% of billings, could potentially use Generative AI.
  • We estimate that a realistic initial upper limit for Generative AI would be to reduce that work by half, or 20% of time entries and 23.5% of revenue

In the previous post, Ryan got tired of hearing the Goldman Sachs “44% of Legal is going away” stat being quoted uncritically and decided to actually look into the underlying data used in their report. Ryan’s exploration of the data is an interesting story in and of itself, but the bottom line is that the data is fuzzy at best, the sample size is laughable, and the breathlessly unquestioning reporting on Goldman’s study has been remarkably sloppy.

After writing up his findings, Ryan shared that post with Greg and Toby, and the question quickly arose, “can we find some actual, useful data to better understand the effect that Generative AI might actually have on law firms?” Gregreached out to Kris Satkunas from LexisNexis CounselLink, a recent interviewee on the Geek in Review, to see if CounselLink could share some anonymized benchmark data for us to analyze.

LexisNexis CounselLink Data

As a reminder the Goldman data was using survey questions about how important certain “work tasks” were for their jobs. Those tasks included things like “Getting Information”, “Identifying Objects, Actions, and Events”, and “Scheduling Work and Activities”. These are quite vague and wide open to interpretation.

In an attempt to find more useful data for our purposes, we asked Kris for the percentages of all time entries that included the keywords “Draft” or “Review” in the description. Our assumption is that those two terms will capture a large percentage of actual time entries in which lawyers are likely to use Generative AI. We fully recognize that this simple heuristic will not produce a clean data set from which to extrapolate definitive results, but as a first pass at some real data, we believe this gives us a nice estimate of tasks that could potentially be ripe for automation with Generative AI.
Continue Reading Generative AI Could Reduce Law Firm Revenue by 23.5%

This is the first in a 3-part blog post, it first appeared on The Sente Playbook.  The other 2 posts are co-authored by Toby Brown and Greg Lambert and will follow later this week. Apologies for the length of this post, but I was channeling my inner Casey Flaherty.
The Big Idea:  The data that Goldman used is insufficient to make the claims about Generative AI’s effect on legal that their report did.
Key Take-Aways:
  • Reporting about this report is sloppy
  • Reporting within this report is sloppy
  • The underlying data doesn’t tell us much meaningful
  • 3 Geeks attempts to find meaningful data
On March 26th, 2023 Goldman Sachs sent shockwaves through the legal industry by publishing a report claiming that 44% of “something” in the Legal Industry was going to be replaced by Generative AI.  I didn’t question that stat at the time, because it sounded about right to me.  I suspect that was true for most people who know the legal industry.  As I’ve heard this stat repeated by multiple AI purveyors actively scaring lawyers into buying their products or services, I eventually started to question its validity.
I started by looking into the press coverage of that 44% number and was immediately confused.  (All emphasis below added by me.)

Law.com  – March 29, 2023
Generative AI Could Automate Almost Half of All Legal Tasks, Goldman Sachs Estimates
“Goldman Sachs estimated that generative AI could automate 44% of legal tasks in the U.S. “

Observer – March 30, 2023
Two-Thirds of Jobs Are at Risk: Goldman Sachs A.I. Study
“The investment bank’s economists estimate that 46% of administrative positions, 44% of legal positions, and 37% of engineering jobs could be replaced by artificial intelligence.

NY Times – April 10, 2023
A.I. Is Coming for Lawyers, Again
“Another research report, by economists at Goldman Sachs, estimated that 44 percent of legal work could be automated.”

Okay, so which is it?  Generative AI is going to replace 44% of legal tasks, positions, or work?
Because those are 3 very different things; each of which would have extremely different impacts on the industry if they came to pass.  Lest you think I cherry-picked three outlying articles, go ahead and Google “AI Replace 44% Legal Goldman Sachs” and see what you get.  Those 3 articles are in my top 5 results.
My top result as of this writing is a news article from IBL News, writing last Tuesday that Goldman says,  “AI could automate 46% of tasks in administrative jobs, 44% of legal jobs, and 37% of architecture and engineering professions.”
We should probably just go back to what the Goldman Sachs report actually said and then we can chalk this up to lazy tech journalism.  Well, not so fast.  Because while the Goldman researchers clearly say “current work tasks” (see below) even that begins to fall apart once you dig into the underlying data.

What Goldman Sachs actually said in the report

Continue Reading 44% of Investment Bankers Think They Can Make Lots of Money Off of Attorney Insecurity (AI)