We talk with Michael Bommarito, CEO of 273 Ventures and well-known innovator and thinker in legal technology and education. Bommarito and his colleague, Daniel Katz were behind GPT-3 and GPT-4 taking the Bar Exam. While he and Katz understand the hype in the media reaction, he states that most of the legal and technology experts who were following the advancements in generative AI, expected the results and had already moved on to the next phase in the use of AI in legal.

Michael Bommarito on his farm in Michigan, alongside his trusty friend Foggy.

While we talked to Michael a couple of days before the news broke about a lawyer in New York who submitted a brief to the court relying upon ChatGPT to write the brief and not understanding that AI tools can completely make up cases, fact pattern, and citations, he does talk about the fact that we are falling behind in educating law students and other in understanding how to use Large Language Models (LLMs) properly. In fact, if we don’t start teaching 1Ls and 2Ls in law school immediately, law schools will be doing a disservice for their students for many years to come.

Currently, Bommarito is following up his work at LexPredict, which was sold to Elevate Services in 2018, with 273 Ventures and Kelvin.Legal. With these companies, he aims to bring more efficiency and reduce marginal costs in the legal industry through the application of AI. He sees the industry as one that primarily deals with information and knowledge, yet continues to struggle with high costs and inefficiency. With 273 Ventures and Kelvin.Legal, he is building solutions to help firms bring order to the chaos that is their legal data.

AI and data offer promising solutions for the legal industry but foundational issues around education and adaptation must be addressed. Bommarito explains that decades of inefficiency and mismatched data need to be adjusted before the true value of the AI tools can be achieved. He also believes that while there might have been many false starts on adjustments to the billable hour through things like Alternative Fee Arrangements (AFAs) in the past, the next 12-36 months are going to be pivotal in shifting the business model of the legal industry.

Listen on mobile platforms:  Apple Podcasts |  Spotify

Contact Us: 

Twitter: ⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠, or ⁠⁠⁠⁠@glambert⁠⁠⁠⁠
Voicemail: 713-487-7821
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠

⁠Transcript⁠:

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert 0:15
And I’m Greg Lambert. So Marlene, we have been brainstorming and setting up Yes, us for some really great shows coming up over the next few weeks and months and lots of calendar sharing lots of calendar sharing this week. So I’m really excited about the just the variety of guests that we have set up, have agreed to talk to us.

Marlene Gebauer 0:37
Yeah, I think it’s gonna be a really good set of guests like today’s guests, for example.

Greg Lambert 0:43
Yeah, so we kind of rolled the dice here and asked today’s guests to jump on. It’s actually the Friday before Memorial Day, we were lucky to reach out and get get get a hold of them. And he immediately responded and we got them on like the next day. I know

Marlene Gebauer 0:58
unheard of. We’d like to welcome Mike Bommarito, the CEO of 273 Ventures Fellow at Stanford Codex, and about 100 Other projects going on in his life. So Michael, welcome to The Geek in Review.

Michael Bommarito 1:14
Thank you for having me. And I guess I have the the honor of being the guy who’s still around and not on a boat or sailing it. Alright, so I barn. That is true. And that’s why in part, I’m available, right? It’s because this is a this is a weekend when we got to get a lot of stuff in the ground and seeds planted. And I think there’s more than a metaphor or two in there for for the industry. And not just the actual soil here. But more on that I think to come.

Marlene Gebauer 1:41
Yeah. So Michael, before we dive into all of the latest in generative AI, let’s talk a little bit about your work at places like Michigan State University School of Law, Stanford and Chicago, Kent, you’ve taught courses on legal analytics and other unique data and tech driven courses. Do you mind telling us about the benefits you get from these collaborations with the academic world? And what are the law students get from courses like this?

Michael Bommarito 2:06
Yes, sir. It’s a great question. And it’s one sometimes my wife asks, right, when we’re in the middle of a game, or in a paper doesn’t pay money, or I’m teaching? Yeah, I’m teaching a class and I don’t not going to surprise anybody. But adjunct teaching doesn’t exactly pay many bills, if any bills and so so I think the truth is that it it is probably a little bit of, of insanity around learning, right? I mean, to continue to want to learn, especially in light of how much is changing right now might seem like a crazy desire or Herculean effort, but I think I just love to learn in the best way to learn is to keep yourself in an environment where you’re surrounded by people who also want to learn and where you are forced to distill what is going on around us in something that that you can, you can teach. And so that’s what teaching is for me. Right? It’s it’s a way to stay engaged in ideas to force myself to synthesize and communicate them. And then you obviously learned from students so many different things, they force you to learn and they teach you things. So it’s really incomparable, right in terms of ensuring that that we stay stay with what’s happening. And if you like learning, then no better place to do it then in school with students who want to learn to

Greg Lambert 3:28
do you have any courses coming up? That would be interesting. Well,

Michael Bommarito 3:33
I mostly guest lecturer these days, right? So I guest lecturer in a class that Amani Smathers former student from the old reinvent lattes at MSU. teaches there are I sometimes guest lecturer for the eDiscovery class at MSU to both of those areas, that kind of the legal process and data and then ediscovery let’s just say the syllabus has required some updating and light of the last couple of months. I bet that’s an understatement that you have other guests that I’m sure will will talk to. And then there’s a number of other classes we’ve kind of taught over the years some some funnier in retrospect, like like the Cryptocurrency and blockchain class at Chicago Kent. But we are hoping to put on an LLM focused, maybe one, maybe two session course at at Kent next year. So that’s probably the one that I’m most interested in. Because I think as we’ll get to it’s, it’s unlikely that the cohort of one ALS or two L’s in school right now will come out into a practice that is absent LLM day to day life, whether it’s in back office or kind of front office substantive work, I guess we’ll see. But these students who are one ALS and two L’s right now, really, we owe it to them for a wide variety of reasons to make sure they know what these tools are. They understand that the legal issues they understand the practical issues and as educators what Would we be doing if we didn’t at least give them the option of learning about this stuff right now? That’s the one I really want to make sure we get out there in front of students.

Greg Lambert 5:07
Are you finding that the the administration at law schools are? Because you know, we’re in the the large law firm world, and we’re kind of jumping through every hoop we can to make sure that we’re staying on top of what’s what’s going on day after day, which is unusual for us. Right? We’re usually kind of yeah, let’s see, see what the market does kind of sit back and wait, what are you seeing? Or what are the demands that you that you’re seeing from school administrators that are saying we also get a stay on top of this? Or are or do you

Michael Bommarito 5:42
is? Yeah, I mean, I’d say I dropped out of my PhD a long time ago to work at a hedge fund. And in part of that was because I decided, I think I didn’t really want to do the tenure track thing. So I’m I am luckily, and unluckily not tenured, and so therefore, don’t really have to interface so much with the administrative elements of teaching, which means I just get to teach and interact with students. But from what I do know, I think the the elephant in the room, which will probably connect conversation later is that the ABA still controls the accreditation guidelines. And you can only deviate so much with a fixed budget and all the other constraints. There’s a lot of legacy. And so the problems that have been facing that the school administrators are still problems that are still probably larger than whether or not we add a courser to it. I’m not really a betting man. But I would bet you in the next couple of years schools that don’t move on LLM curricula will regret it. But right now they have other fish to fry in terms of like, what do we do post COVID? What do we do in light of student lending? What do we do in light of LSAT trends? So, so they’re not the only ones barking at the moon here? I guess you could say so now we

Marlene Gebauer 6:57
gotta leave it in.

Greg Lambert 6:59
Well done well done.

Michael Bommarito 7:02
Well, I have a rooster in the background, so I’m just hoping the rooster doesn’t call comes

Marlene Gebauer 7:07
to talk to us. Yeah. Never had a rooster on the show.

Greg Lambert 7:11
Does a rooster have

Michael Bommarito 7:12
the name? His name is foggy, foggy? Like. Yeah, yeah. Do say I do say.

Greg Lambert 7:20
But listen to me when I’m talking son.

Michael Bommarito 7:25
After the senator right. I mean, he was a senator. I think he was a state Supreme Court Justice prior to his time in the Senate. So this is not

Marlene Gebauer 7:33
a fun fact here on The Geek in Review.

Greg Lambert 7:35
Yeah, yeah. Well, speaking of fun facts, let’s let’s jump into the you know, the big topic, which is the discussion of GPT. And the bar exam. So you were you were heavily involved? And both the 3.5 and the GPT, four versions taking the bar exam? I think it was you and Dan Katz and some others. Yep. And the CPA exam and the CPA exam.

Michael Bommarito 8:04
don’t leave out anything that’s that. I know. I mean, this is like, there’s so much to talk about this stuff, right? So we have been tracking I think, at a high level, what can a systematized process of thinking that’s a really vague way of saying like, let’s just say, Large Language Models or generative ai do when presented with legal language for a while. So a couple years ago, we published a paper called Lex NLP, kind of old school NLP, about a year and a half ago, we published a paper called Lexis glue, with a big group of people. That glue is a reference to something else that people in academia have done. It’s about general language understanding and evaluation. So kind of what will these models do? If you ask them a bunch of basic facts, if you ask them about arithmetic problems, that kind of stuff that might be on the LSAT, and over the last, say, three to five years, there’s been a steady but but still disappointing trend of marginal improvements, maybe a little bit of a jump. And the things would get better at let’s say, regurgitating material from Wikipedia, but they wouldn’t really demonstrate much creativity or spark or ingenuity or complex reasoning. And then clearly, that I’ll change. So what I’ve been using Large Language Models one way or another, and neural networks broadly for 20 years, I guess, in the latter case, and let’s say, three to five years, In the former case, GPT two was nothing to write home about. It could write a really bad blog post, it would struggle to write a LinkedIn post, although I guess the standard on that is fairly low these days. Then all of a sudden, Tex DaVinci three, which is kind of GPT three hit, and it was a thing, but you couldn’t really sell it because nobody would believe it. It was weird. So, yeah, we, we saw the GPT, three, five stuff hit. And then we saw that first week. And then I was still a little bit of a pessimist. I’ll be honest. And around Christmas like the week of Christmas, Dan Katz called and was like, Dude, I think we should try it, right. Like we’ve tried all these other tests and exams and formal assessments from the Natural Language Processing communities, I just try it right, it won’t take that long. Just hack it out in a day or two, it’s quiet this week, no client work over Christmas week, or whatever. And so I did it. And it was so much better than any prior models, I think, which didn’t mean passing clearly in the case of three, five, tech 73. But it meant that it was within a clear path to being able to pass whether that’s through combining it with other sources of knowledge through like retrieval, augmentation, which is kind of what a lot of the information providers do, like everybody’s seen co counsel. So that’s like kind of how that stuff works, or whether it’s just through these base models, like the thing that you actually get from open AI or Microsoft, knowing in quotes about legal. And then I guess, again, under NDA about some of the stuff. But then then we started working with Pablo. And in Sean, one of the other engineers at CaseText. On the GPT, for version of the paper in Pablo was i He said this himself publicly to Pablo was never really in doubt that this thing was gonna pass based on what he’d seen already. And as he said, like, what the the reaction that that many managing partners at firms had had to what they had seen too, on the screen. So. So it was more of an exercise in terms of like, putting this all together. And there’s just, there’s a lot of this stuff that went into this, both from what can we do with commercial restriction perspectives? What can we do? Because it’s got to be totally behind the scenes prior to GPT for being publicly announced. And what can we do because the ncbe, and the state bars don’t make a bunch of this stuff actually public and available in kind of an official format. So it’s a it was a long, crazy, fun road, at the end of the day to basically have a lot of people nod their head and not be very surprised, because they had all kinds of use ChatGPT are seeing GPT for and weren’t surprised by the time it was all said and done. So that was one of the most fascinating things and open mumbling for like five minutes straight here. But by the time we did all of this, nobody cared when we launched the GPT four thing, because nobody was surprised when we talked to like managing partners, or GCS or whoever. Nobody cared. Even the ncbe, who we had talked with, wasn’t surprised when we published that paper. It was like a non issue to people. And they all moved on and, and it’s just like, in the zeitgeist now, and we’ll talk to people and they don’t even know, we’re authors of the paper. And it’s kind of funny, and honestly, I’m happy about that, because it makes the conversation a lot easier. Just nod my head and move on. And that’s it. Yeah.

Marlene Gebauer 13:06
I mean, it’s interesting. It’s like, like, on Jeopardy when, you know, Watson, you know,

Michael Bommarito 13:13
OG Watson. Not Watson. x.ai?

Marlene Gebauer 13:17
Yes, yes. Let’s be clear. You know, people, people were just kind of, they were very, very surprised. And they were just like, Wow, that’s amazing. And, you know, so, and even you were saying, you know, you weren’t quite a believer for GPT. Three. And, you know, you’ve talked about others expectations for GPT. Four. And you know, what, it seems that, you know, in the past, we’ve kind of been, you know, we’ve been a little skeptical of different new technologies. But you know, now, it just seems like you said, Well, no one was surprised at this point, because they’ve all been been using it, you know, but what were your expectations? You know, going in, what did you think of the results? And, you know, why do you think suddenly, like, everybody’s like, okay, that’s okay. Now, you know, it’s like, of course, of course, it did. Well, you know,

Michael Bommarito 14:00
yeah. I mean, you know, kind of borrow a Pablo anecdote. You just sit down in there some area of law or some work product that you have personally experienced? And you ask it, especially in a retrieval, augmented contexts, like the information providers are able to put together when they combined with primary law, you ask it to draft a motion, or a contract or a demand letter with some specifics to it. And in less time than it takes you or takes the managing partner to email the associate. It’s got a response on the screen, that’s pretty darn good. That’s probably better than your first year or second year, or maybe even your third year. And then you say, Okay, any veil that had stood between belief and an MI is now gone. Which is funny because I’ve been using neural networks in one form or another for 2020 ish years. I like what a feed forward network can see and for Tran once upon a time. So it’s like this is not none of this stuff is technically new to me. But to see it on the screen, like the moment when Pablo showed it to me the first time was when, when all of the theory and math and conjecture became just, I guess, the state of the world today. Yeah.

Greg Lambert 15:18
And I think one of the things that, and we’ve had other guests that that mentioned, this may have been Pablo himself is that not only did it the GPT, four pass the bar exam, but it did it in something like seven minutes.

Michael Bommarito 15:37
The MBE. Yeah, so he’s probably quoting the MBE portion, and GPD, three, five is different, like Text DaVinci. Three is different than turbo boost three, five is different than four 8k Is there. And then there’s a bunch of caveats to some of that stuff. The point is, depending on the day, and how many people are trying to use ChatGPT, the thing can do it in under five minutes. If everybody’s trying to do their homework, then it’s all going to crash and the API will return errors and you won’t pass the exam. But somewhere in there, we have a bunch of data that shows that the thing could pass every like, answer all 200 Some MBE questions, including the experimental ones, in under whatever number of seconds, and according to the latest JD advising.com table, take it with a grain of salt or whatever, at a 97 percentile national average. And you’re just like, Okay, well, that was never a good assessment of the real practice of law anyway. So what’s next?

Greg Lambert 16:33
Yeah, that’s well, and let me let me expound upon that, then. What do you think, are there any other Tet? I know that you, you said, Now, everyone’s just kind of it’s not a big deal. But you still see that and you hear it, it’s hard to hear a podcast or read an article that some somewhere in it doesn’t say, you know, this thing passed in the, you know, the top 10% Blah, blah, blah,

Michael Bommarito 17:02
or center percentile or whatever, the 90th percentile to get it? Right.

Greg Lambert 17:06
And, and so I mean, it’s definitely a, you know, it’s a line that’s being used to show how far advanced This is. This has gotten, I would say, Are there any other types of exams that it could be taking to, to show or do you think now it’s time it’s going to start writing the exams? So

Marlene Gebauer 17:29
I mean, maybe it’ll figure I don’t know, if it’ll figure out a better way to test people on on, you know, their actual knowledge in terms of being practitioners? We can ask that as well.

Michael Bommarito 17:40
Yeah. I mean, I don’t know how deep you want to go. Right. But it’s like, I think it has revealed to us to some degree, the, the chance that there was a fleeting moment in time when knowledge work looked like this. Because if we’re being honest, there’s a lot of what we do as, quote, knowledge workers, that isn’t really a lot of knowledge. And so is tacit experience is strategy is the ability to understand in a very deep way, whoever you’re representing, or their opposing counsel, or counterparties, or the environment in which they operate. Is that what you’re doing? Or is it simply that you know, which sections of the IRC turn into a decision tree that you can then turn into? Yes, this applies or No, it doesn’t that, like the ladder doesn’t seem like something humans like doing. My wife’s a CPA, CIP big for, right? Like, it’s part of why we did the CPA exam, I can tell you for a fact that even people who go to school and get all these degrees and work in big four, don’t really like doing a lot of that kind of knowledge work. It’s not much different for most attorneys, right? I mean, what is it that we’re doing as knowledge workers today isn’t necessarily what we want to do or should be doing or can sustainably do from an economic perspective, and boy has this thing, put all that stuff front and center?

Greg Lambert 19:05
Well, I want to talk about some of the other work that you are doing, which includes the 273 Ventures and and another, I guess this is part of that with Kelvin Legal.

Michael Bommarito 19:18
Yeah, it’s a geeky thing, right? And I like talking to talking to a guy who’s got some of the coolest comic books around, right. It’s like I feel safe. But the the 273 and Kelvin stuff is all an illusion to like chaos and order and entropy. And in really, in a deep sense, the problem that we all are our knowledge or information engineers in this field. Obviously there’s the strategic stuff, but much of what we do from an effort our perspective is engineering around information. Like I want to change the day’s pay from 30 to 90 or I want to change an assignment restriction or insert an assignment restriction is forms of information or knowledge or whatever, and we just push bits of knowledge around. Why is there so much chaos? When what we really are as knowledge and information managers? Why do our systems? Why is why are our processes? So written with chaos? Why is it that the marginal cost of drafting the next contract of doing the next eight asset purchase agreement, right of even basic things like billing the same client with the same timekeepers the next month on doing roughly the same thing? Why is it so chaotic? Why are the marginal costs so high? So Kelvin and 273 are all about this, like absolute zero from a physics or thermodynamics perspective, where for better or worse down there, you get, you get some degree of order, you don’t have this kind of entropy or chaos. And the marginal cost, in some sense goes to absolute zero. So Kelvin is about turning the things that we do, which is not everything, but the things that we do in this profession, that are information or knowledge engineering into low marginal cost propositions,

Marlene Gebauer 21:09
and what about 273 Ventures?

Michael Bommarito 21:13
Yeah, so that that is we started out so we, we had Lex predict, find that 14, we exit in 18, we sold it to elevate an LSP outsourcing company, we we did three years, whatever, you know, the story was this stuff. And then last year, we came out in incredible time right to to start fresh with clean slate, and all the LLM stuff hit. And so we started with his 273 brand with that kind of the the metaphor that I mentioned a minute ago, and then searched around for a little bit of a metaphor that we liked with this chaos and entropy stuff. And that’s where this Kelvin thing came from. So 273 Ventures is the entity. And Kelvin is the brand that we’re going to market with in terms of the what we’re calling the legal data operating system, which is just a really fancy way of saying like a data stack or a data warehouse or a knowledge management platform.

Greg Lambert 22:08
Now I was I was looking over the website. And and thanks to you before, right before we got on, you showed me some additional examples and some use cases of what what you’re looking at. And you got things like automating litigation workflow, looking at the billing analytics. And so, I mean, is this, for lack of a better better way of saying, is this just automation of certain tasks that humans do now? Is that what your efforts are? Or something beyond that?

Michael Bommarito 22:46
Yeah, it’s it’s one of those things where it’s easy to get very overwhelmed very quickly from like a single point, sale or point solution. So what we’re building is a bunch of what we think of as Lego blocks or the pieces from a software or data or, quote, AI perspective that you need to do just about everything that can be automated. So we started from the foundations really boring stuff at the bottom right. It’s like moving data around between deal rooms in OneDrive, it’s OCR in documents, when the words that appear in legal documents are not like the words that Adobe expects, it’s spell checking the same kind of thing, right? The your your dictionary, and word doesn’t know most of the words we use, it’s a problem. When you let an AI model go to town on a bunch of text, it’s come through a bad OCR process that has been spelled corrected in the wrong way, then you get bad outputs. And so whether you’re going to do some fancy AI stuff or not, you got to start with the foundation. And the foundation for documents is typically are the words on the page, the words they need to be if it came through digital or, or scanned process, so and then there’s a bunch of fancier stuff, right? Like we plug into all the different AI models, we have all these kinds of vector database things. And that’s how, for example, CaseText can combine the primary law, which is many more tokens than fits in GPT, three or four or any Large Language Models with your query to get back the relevant portions of case law or statutory or regulatory material? To answer your question. So there’s a bunch of this really boring kind of technical stuff at the bottom. And yes, for most firms, that’s not what they’re gonna buy, right? Like maybe the amlaw 10 or magic circle firms or fortune 50 have legal departments with data scientists, you know, how to use that stuff. But the use case section, which I think you’re you’re alluding to, and I had put in that chat is what the the end goal of that stuff is so you with us, you essentially buy these Lego blocks, and you can make them in anything right? You can you can turn them into any of these use cases, and instead of buying five or 10 different points solutions that are SAS solutions that may or may not aren’t as we’re seeing today, go through business combinations and get repriced or not get their series A or B or C and continue viably or whatever. Right. So like, instead of being beholden to all of the vendor counterparty risk that buying point solutions creates, you can kind of buy it, if you’re ready for it from a maturity perspective, you can buy one platform with all the tools you need, and have all five or 10 of those point solutions. And another differentiator for us is that we were old school, all of our stuff, you can license and you get source code and escrow and you could put it on your own servers physical or VMware cluster, like an hour and a half hour and a half call earlier today with em law 1025 firm, right? And they’re all focused on how do we get this stuff on our VMware cluster? Because we really want to use Large Language Models, but we’re not going to send any of these documents up to GPT. So what can we do?

Greg Lambert 26:04
Yeah, and I think you’re seeing a lot of the topics right now, especially with if you have CaseText. Co counsel, you have harvey.ai, that really talks about applying generative AI to the practice of law. But it looking at the list you have, and I’ll make sure I put a link in the show notes to the list. This is more on the operations in the business side of law. And I think we may not be as far on that curve on the operation side was with generative AI, as we are on the practice side. At this moment. Maybe maybe not. Marlene has kind of given me a little side eye on that one.

Marlene Gebauer 26:48
Just thinking about like, you know, Microsoft and co pilot and I think that there’s there’s talk about sort of the operative type of capabilities for that product.

Michael Bommarito 26:59
Like the syntax demo yesterday. Yeah.

Greg Lambert 27:03
Yeah. Well, what I want to see, and I think I saw it kind of on that list is an easy way to take, you know, the taxonomies, or the meta metadata from a product or, you know, an open source product like SALI, and have that more automated on on our side. I know there’s a lot of talk about it. But I want to I, for one want to see a little little bit more action going forward.

Michael Bommarito 27:29
Yeah, I know. And that’s where it’s like that the we all have the same dream, or vision about future state, right? And obviously, there’s, there’s all the commercials and trust issues to get there. But why do we not have more interoperability of data? Whether it’s between vendors or between firms or corporate from a pooling perspective, it doesn’t really help anyone to hoard privately, the vast majority of what we hoard privately. And so the only way to really do so many of the things that we’ve talked about, I mean, you’ve been talking about for decades, right? Is to to share at least a taxonomy. I’m not saying we like not saying these two firms have to merge, although there’s clearly going to be a lot of that this year. But can we at least agree on what the areas of law and practice groups are? So what are some of the boring examples of stuff we’re doing our I mean, this is incredibly boring, but a good example of it’s not substantive, it’s low hanging fruit, it would help at MIT. It’s a real thing. We’re going through law firm directories for personnel, and trying to help them standardize the areas of law or practice groups, or how they refer to languages or where they’re admitted so that all of that stuff is tagged. Yes, with SALI. Thank you to Damien and Toby and Jim Hannigan for doing all the hard work to put all the tags in one place that we can consume them. But we’re tagging all of those law firm profiles that we all know in love with standards, that then mean the law firm can send to the corporate in the RFP, a standards based digest of who their actual experts are so that corporate can consume in batch, all those experts in it’s not like a bad thing, because the next time with the tools at least right? The next time the firm has an issue in wants to say out of our panel, who should we call all that standardized data means they might have a system where your name is going to show up at the firm, right? And you’ll get that call, if only the corporate had a way to reliably search across different firms for the industry or matter type or whatever it is that that they’re trying to, to narrow in on. So it’s a lot of the boring stuff, right? It’s um, it doesn’t have to be drafting a motion that that’s going to determine whether or not a business continues to sit going concern. It could just be fixing all of our data in the systems we rely on. And I think that gets lost in a lot of the conversations about generative AI? Yeah.

Marlene Gebauer 30:02
So you’re talking a little bit about like what we can do with kelvins? Ai? So how does the capabilities? How does that differentiate it? Because, you know, I’m hearing some of the things you’re talking about. It’s like, okay, I’ve heard that from some of the other sources, too, is like, you know, so how is this differentiated from the other solutions in the legal industry?

Michael Bommarito 30:24
It’s an interesting question, because it’s like, to some degree, we’re not differentiated. Other than that, we do all of this stuff, kind of under one platform. So instead of

Marlene Gebauer 30:32
me buying five, or 10, which is also a little bit different. It’s not like, Oh, hey, here’s this big thing. And we can you can do whatever you want with it. It’s like, here’s, like, you know, this chunk, this chunk, this chunk?

Michael Bommarito 30:42
Yep. So we have like, I mean, you want to talk buyer persona, and everything. Law firms tend to be buyers of urgency of point solutions. And so they buy one module here, they buy the NLP module, they buy the Billing module, they’ve got an urgent matter, or transaction, they buy the source module, that they end up spending more and would have been better off developing the capability upfront. And Greg, you’ve been talking about this for, for a while, to put it lightly. The corporates might be more strategic buyers who invest in larger scale synergistic capabilities, but then they also have different timescales and their own set of projects. So it’s, it’s a weird market, right? We all know, we all live this market. So we all know the pain points, but it’s like, in a lot of it does come back to share data, right. So like having fewer data migration or data quality issues, would I think facilitate less hesitant because we’ve all been through the prior generations of AI the prior promises of the Next Gen II LLM that fell flat on its face. And a lot of it comes back to data, data data, right, the quality of the data, the taxonomies, the degree to which you agree the taxonomy so

Greg Lambert 31:52
well, that and I think there’s kind of a secret hope that the AI will just fix all the crazy stuff we’ve done for 30 years. And I don’t know if

Marlene Gebauer 32:03
it’ll fix it’ll fix. Smart. Yeah, it’ll fix enterprise. Sorry.

Greg Lambert 32:07
There we go. Yeah.

Michael Bommarito 32:09
It’s getting I mean, it’s like, I want to be, I want to be both optimistic, but realistic, but also not under underestimate what’s possible is like, if you really wanted to sign up for something like the SALI standards, well, I do think you could do for example, as we are kind of doing right now, an ALM, or a practice, taxonomy update, where you go through every single matter in foundation, or elite or team connect, or, or whatever, and update every single one of those matter areas or sub areas or whatever. And as long as you’re okay, with the SALI standard, I think we do kind of have that dream. Today. Now, there’s obviously a lot of people who are going to want something different than what’s in the legal matter standard specification today, or specialized practice areas. So a lot of stuff you could talk about, and I don’t know if you’ve had Damien RIehl or anybody else from SALI on, but recently at least, but but there’s a lot of stuff in the kind of in the pipeline right now. So on the GitHub, for the LMSs standard, we got a lot of stuff in the pipeline, and hopefully some of that will get get people jazzed up. Hopefully, I’ll find some more time to help get that over the line too. But the like this weekend, if all goes well, but the Damien’s listening, I’m sure. So I gotta say that the hope is still a valid hope. And I think there’s more. The flame has been kindled more in the last six months than it deserves to have been kindled in the last six years or so. Let’s not snuff it out yet. Now,

Greg Lambert 33:42
I agree. So if you could give, say one piece of advice for law firms out there, who are still trying to wrap their heads around the potential for these generative AI tools and LLMs. You know, what would you tell them? Can you say,

Michael Bommarito 34:02
because we do. I mean, we do like we talked with managing partners, or, or the innovation committees or councils or whatever name is described to the one or more individuals who

Greg Lambert 34:14
are urging tech responsible users is ours.

Michael Bommarito 34:17
And then there’s like 17, Task Force variants that are all with overlapping scope. And it feels like the federal government by the time we’re done, but the reality is, I don’t think there’s any way out of this, other than being honest about delivery and value proposition. And again, I know everybody’s been saying this for longer than anybody wants to admit. And it’s been wrong every year over and over again. But I do think that the corporates from just an awareness perspective over the next 12 to 36 months are not going to let this be the moment especially in light of like rising interest rates, the potential for a little bit of economic dislocation from other issues. Who’s abroad and domestic. So I just don’t think this is going to be the one that people always talk about. Like I spend a lot of time in finance and alternative investments. You can short a stock, or short a market like real estate, mortgage backed securities residential for a very long time and be right about the fact that residential mortgage backed securities will eventually experience a dramatic destruction in value. But being wrong for seven years is tough. And so I don’t know, this is where, as a betting man, I don’t know what I’d say. But I do, I do think this is the year right for the next two to three years feels like it when the AFA are gonna have to be an honest thing, not just like a shadow billing of hourly time, and you still give it to the guy you went to school with. But like, what value are you creating? What risk? are you reducing? How are we seriously going to measure all of that, and I’m okay compensating you in a way that gives you a material part of the value you create or the risk you reduce. But we all gotta be honest with each other. And so in some sense, it just starts with that honesty. And it has to be Chatham House rules at a Chatham rules it, some private forums, and so be it. And that is happening to some degree. But it’s, it’s got to happen. It’s got to happen fast, because the thing is, we all are friends to some degree, the GCS and the managing partners, but the CFO is not necessarily your friend, CEO is not the friend, the shareholders are not your friends. And they know you’re not going to put this genie back in the bottle.

Marlene Gebauer 36:37
Mica we have reached the end of the podcast. And at this point, we ask all of the guests the crystal ball question. And that is what changes or challenges do you see for the industry over the next two to five years?

Michael Bommarito 36:52
I’ll focus on one that that I feel is, let’s say normatively most important, I feel like we need to get out ahead of the education. Elephant, right, we need to be at the highest level possible at the schools on the ground, figuring out what we’re telling the students who are coming into these next cohorts. And, and making sure that what’s actually available to them, whether it’s in house or not, is a truthful representation of the things that that we think right? Like, I think there’s this sense if you will have a detrimental reliance of all these cohorts going in, and I I don’t think we are all going to look back in five years and feel good about not being honest with the students who are coming into this system right now, in light of the things that we’re seeing in terms of delete, associate onboarding, or total associates required or, or just overall demand. So that to me is the biggest thing. And we could talk obviously shop about technology or AI, but from what’s nagging me and my heart perspective, it’s, it’s I don’t want a student deciding to go to school under a set of assumptions that we all know they should know better are,

Marlene Gebauer 38:08
you know, are we are we preparing them and informing them adequately? Yeah.

Michael Bommarito 38:13
Right. detrimental reliance aside, and the tort sense or whatever it’s like, we need to make sure that the curriculum that we put them through that they pay for, reflects what we actually need them from an industry perspective to be doing when they come out. It’s always been an issue to some degree, but it is going to be more of an issue over the next three to five years, and I think it has ever been before.

Greg Lambert 38:33
What a unique perspective on legal education.

Michael Bommarito 38:38
I can say that because I’m not tenured, firm. So I again have this, this, my crystal ball is not clouded by any of that.

Greg Lambert 38:48
All right, well, Michael Bommarito, CEO of 273 Ventures, thank you very much for hopping on the call with us. This has been this has been great. Thank you all.

Marlene Gebauer 39:00
And of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found at @gebauerm on Twitter,

Greg Lambert 39:13
And I can be reached @glambert on Twitter. Mike, if someone wanted to reach out or learn more, where can you be found online?

Michael Bommarito 39:21
I embarrassingly am not very active online anymore, but I am on LinkedIn, given the age and all and on Twitter at MJBOM m a r

Greg Lambert 39:31
or professionals of a certain age.

Marlene Gebauer 39:34
I know I was like trying not too loud or whatever. And speaking of professionals at a certain age listeners, you can also leave us a voicemail on our geek and review Hotline at 713-487-7821 and as always, the music you hear is from Jerry David DeCicca Thank you Jerry.

Greg Lambert 39:52
Alright, thanks Jerry. Alright Marlene, happy to have a happy holiday.

Marlene Gebauer 39:56
Alright you as well.