In this episode of “The Geek in Review,” hosts Greg Lambert and Marlene Gebauer celebrate the one-year anniversary of CoCounsel, the pioneering Generative AI Legal Research Tool launched by CaseText. They are joined by Pablo Arredondo, Vice President of CoCounsel at Thomson Reuters and co-founder of CaseText, to discuss the significant strides and challenges faced in developing and implementing generative AI in legal research. Pablo shares insights into the early days of exploring generative AI and the transformative potential it held for overcoming the limitations of traditional keyword-based search methods in legal research.
The conversation delves into the technical and strategic journey of bringing CoCounsel to market, highlighting the team’s rapid pivot to leverage GPT-4 technology and the collaborative effort that ensured its successful launch. Pablo emphasizes the importance of quality control, trust, and addressing the nuanced requirements of legal research to ensure that CoCounsel met the high expectations of its users, including law librarians and legal professionals.
Pablo also reflects on the broader implications of generative AI for the legal industry, including the rapid adoption by law firms and legal departments seeking to leverage this technology to enhance their research capabilities and workflow efficiencies. The discussion touches on the ongoing challenges and opportunities presented by generative AI, such as regulatory considerations, ethical concerns, and the need for continuous education and adaptation within the legal profession.
The acquisition of CaseText by Thomson Reuters is discussed, with Pablo sharing his perspective on the strategic move and its potential to further expand and enhance CoCounsel’s capabilities and reach. He highlights the synergy between CaseText’s innovative approach and Thomson Reuters’ extensive resources and market presence, which together aim to drive the next wave of advancements in legal technology and research tools.
Finally, the episode explores future directions for generative AI in legal research, including the expansion of CoCounsel’s capabilities to encompass a wider range of legal tasks and its potential to transform the practice of law. Pablo’s enthusiasm for the possibilities ahead underscores the significant impact that generative AI is set to have on the legal industry, promising to revolutionize how legal professionals interact with information and conduct research.
Listen on mobile platforms: Apple Podcasts | Spotify | YouTube
Contact Us:
Twitter: @gebauerm, or @glambert
Threads: @glambertpod or @gebauerm66
Email: geekinreviewpodcast@gmail.com
Music: Jerry David DeCicca and Eve Searls
Transcript
Marlene Gebauer 0:09
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.
Greg Lambert 0:16
And I’m Greg Lambert. So it has been a year since the official launch of case Texas CoCounsel generative AI ai legal research tool, and a research tool that was not just the first to market, but one that many of my colleagues still say is the best AI legal research tool that is out there. So in order to mark this anniversary, we have asked our longtime friend, and I think guest number two, yeah, after Zena Applebaum, who are now working together, by the way, we asked Pablo Arredondo, who’s the vice president of CoCounsel. And many of us remember as the co founder of CaseText, we asked him to come back onto the show and talk to us about this anniversary. So Pablo, welcome back to The Geek in Review.
Pablo Arredondo 1:05
Guys, thank you so much for having me back. Great to be here.
Greg Lambert 1:08
So probably you and Jake Heller And Laura Safdie, and the rest of the team there CaseText Were always known as these trailblazers and, you know, when it came to legal research and advancements, in fact, you always seem to be like the very first one to mark it with with all these great ideas, and then everyone would come in behind and copy those.
Marlene Gebauer 1:34
Yeah, and this there’s no difference when it came to applying generative AI ai functionality to legal research when you lost when you launched CoCounsel on the CMS March 1 2023. So, you know, take us back, if you don’t mind to the War Room at CaseText. And your decision to jump into the deep end of Gen AI ai, and how you and the CaseText team were able to get CoCounsel launched real quickly.
Pablo Arredondo 2:00
Okay, well, yeah, it’s just crazy to think it’s only been a year since it came out. And you know, for us, the dive into generative AI, or these Large Language Models actually goes back a few years, you know, that was on your guys, I was in back here with you guys, I think April of 2021, talking about all the things you can do with search by using neural nets and language models, you know, using precursors, like Bert read these earlier models. And so we kind of fell in love with these things very early on. Because we saw that they could allow this whole new level of interacting with information where you weren’t suffering from what I theatrically call the tyranny of the keyword, right, for the first time, the system had language encoded in such a way that it could get the gist to kind of understand the meaning, even if there was no overlap. And so, you know, sometimes we were like, Oh, you guys had such, you know, sort of vision to see where the, you know, the truth is, like, we were take, we didn’t have to take a lot on faith, like we were seeing this stuff create a tremendous amount of value, right away, right from our earliest applications of doing it. And what really sort of changed, everything was September 16 2022. So about, you know, six months before, the actual launch of CoCounsel is when Jake and I under an NDA saw the first demo with GPT-4. And that was one of the craziest weeks of my life. I mean, we were given access to a Slack channel, where literally the Slack channel was like just being able to call GPT-4 and ask questions. And remember, this is six weeks before ChatGPT had come out. Nobody had seen any of this stuff at all, you know, outside of the folks working on it. And we immediately saw that it was this qualitative leap, and that it could do so much more than what we had seen before much more than we expected. Even our you know, our brightest AI scientists at the company were sort of like damn, like, this is actually happening. Well, it’s it was not a given. And so, you know, we pivoted the entire company to doing it to the focusing on it. And that’s where startups kind of have sometimes an advantage, which is we can be nimble like that, right, it’s relatively easy for us to pivot the entire company. And what’s so great about that, you know, every team at the company had something to do, right, you know, how do you market these now these things that make people afraid, sometimes and how do you were branding it this mysterious black box AI, and then sometimes we had to create new teams like the, you know, quality control and trust trust team who can evaluate these to see if it’s actually working the way it’s intended to work and to do the kind of control on that. And so what we liked were we had in beta and then some you guys seen this red was a much broader set of skills than what we actually launched with red part of what we did is we prune down to just those skills that had passed muster. And legal research I think was but you know, absolutely sort of the anchor tenant, if you will, for a few reasons. One. It’s such a great way to show off the power of these language models that it’s able to understand the nuance have an input into the system. And the nuance of the cases that come back now, not infallible certainly, but doing it well enough that well, let me put it this way. You know, I was a co author on the GPT-4 passes the bar paper. And I was like, that’s a cool, that’s cool. It’s neat. It’s a great, cool milestone kind of marketing. But what really convinced me this was real was when the law librarian community didn’t rip it to shreds, right. And they said things, and I may have said this before, but you know, they called it solid. That is, that was gushing praise I’ve ever heard from a law librarian, right. So right, when suddenly this thing was actually, you know, like, you know, and again, not infallible by any stretch, but you know, like, able to handle that, in a way the librarians are saying, it’s actually doing the thing. To me, I was like, Well, this is actually happening. And the other thing with legal research is there weren’t as many security concerns, right, security, and privacy is obviously a huge part of of any, anytime you’re practicing law. And while I think some of the most exciting use cases for this technology is uploading your own documents, right, we were talking before we started about, you know, pointing it at all of the documents in your litigation, the transcripts, the correspondence, discovery, etc. That, you know, you have to go through some hoops on security for folks to feel comfortable doing that. Whereas with legal research, it was like, you know, here’s the case law, have at it. And so we launched with legal research, the ability to create your own databases, some of skills are on contract analysis, and you know, does this contract comply, some depo prep, you know, sort of this sort of battery of different skills. And you know, what more testament to how different it’s been than the venue we chose to launch it on, which was morning, Joe, on an MSNBC. And I will never forget the faces of the hosts, as here come sleep deprived Jake and I, there, Jake was covering a lot better than I was carrying a laptop. They’re like, you see them, like hitting the security button under the desk. Like, they think that they’re gonna come on national TV and demo illegal tech product. It was like, um, and it’s just been, it’s just been wild. I mean, it’s just been such an amazing year. Just on all fronts.
Greg Lambert 7:09
Yeah, I just remember it because Pablo was wearing a tie. Yeah. And it was one of the few times I think I’d seen you wearing a tie. Right?
Pablo Arredondo 7:18
Right. Although jeans, because they’re only gonna see me, I wear jeans, cuz I’m like, they’re only going to see above the table, but then they have the shot where it’s like, Nope, he’s wearing T shirts. Anyway. I tried.
Marlene Gebauer 7:26
I tried. So that was my that was very Startup of You.
Greg Lambert 7:28
Yeah, that’s how they were startup. So well, you know, you, you kind of mentioned this with the, with the library, the law librarians themselves, kind of jumping in and seeing this as a a, you know, a big advancement. But it wasn’t just the libraries. Marlene and I have talked for the past 18 months on this, that we have never seen the legal industry, especially big law, and even fortune 500 legal departments, you know, just absolutely move as quickly as they have with this advancement than legal tech when when it comes to generative AI. So, you know, as you were taking this new tool, this kind of kind of crazy thought of the technology can understand the language. And not just the you know, not not just the keywords. How, you know, how surprised were you with the reaction that you were getting when you were showing the product?
Pablo Arredondo 8:28
So the six weeks where we were showing it to people under NDA, before they had seen ChatGPT at all, was some of the strangest experiences I’ve ever had full stop with like a fellow humans, because they were going from like zero, to suddenly seeing you know, and there’s something language is kind of uniquely human, there’s something a little unnerving about seeing a machine have language. And, you know, immediately their minds went to cyber dime, you know, that the Terminator scenarios to sort of the end of legal practice. You know, and then we’ll perhaps later talk about that, obviously, there was a lot of hyperbole around this stuff, you know, early on, and that that was honestly like, it was beyond just legal tech and felt like just this moment, as our species was like encountering this intelligence, and did have like, the chance to be the one to show that to certain people was profound, you know, I mean, it was a really an amazing moment that I don’t think of in terms of normal legal tech sales, right? In terms of like the the normal legal tech sales process. I mean, I’m, you know, I’m a geek, I’m a nerd, I love you got the law librarians are always the one I’m trying to impress the sales reps. Were very happy with how like the managing partners, were reaching out to us saying the clients are asking about this. I think that’s one of the huge differences, right? The clients, you know, last decade, we’re not asking them, What are you doing about the blind spot and current sighters where it’s not catching soft relationships? Certainly something like Kara wired up, right, that was never coming from the clients, you know, in this heated way at the bread. This is something where the public rank or around it and ChatGPT has been it’s sort of mixed blessing in a way because it was misused. was in so many ways, right? It had the hallucinations like the consumer Chatbot. But on the whole, what it just did is it just elevated and magnified the public discussion around this in such a way that you had something quite unusual and LegalTech, which is like partners reaching out, you know, certainly to folks like yourselves sometimes directly to us, asking, you know, we need to know what’s going on here, we need to see this. So it’s just such a night and day difference compared to, you know, what it’s been? And well, you know, somebody will say, Oh, that’s how lucky these new startups that are born into this, I don’t know, I think the real joy is 10 years in the old world, and then experiencing that, because then you can truly, truly savor it.
Greg Lambert 10:40
Yeah, I just wanted to follow up on that, because, you know, this was early 2023. And you mentioned hallucinations, we still almost weekly now, you know, it’s some story that comes out where some law firm has submitted, I guess the latest one is, there was a law firm that submitted a brief to the court on how it should be paid, that use ChatGPT to write the entire document on how white why it should structure its legal fees the way that it does. But what are you, you know, we had some what I call March 2023. Problems with the hallucinations of not understanding how to use the product. What are you seeing as like March 2024, for problems that, that users are running into this?
Pablo Arredondo 11:34
Right. Yeah. So I think that’s such a great way to put it, I think, you know, initially just understanding the difference between a chatbot. And using technology like, you know, that uses retrieval augmented generation that uses these other architectures, that you can really use it, that was the big problem. I think now we’re at a place where some of those old or old friends in legal tech like workflow, and access to documents and integrations are starting to come up a bit more right. And more and more, I think now it’s like, okay, when the rubber hits the road, how do we actually apply this in the real world? And I think those are solvable. And I think, you know, just as this technology caused all this fanfare, with new average to learn about legal tech, I’m hopeful that this technology will galvanize a bunch of integrations between you know, a lot of these these systems, consequently, oh, sorry, please, go ahead.
Marlene Gebauer 12:23
No, you go ahead. I thought you’re done.
Pablo Arredondo 12:25
Let’s do one at a time.
Marlene Gebauer 12:27
Well, no, it’s mine is a totally different question. So finish up.
Pablo Arredondo 12:30
Oh, well, then I’d say also, cost is another issue. I think, right now, the best models are still pricey. And there’s different things that are going on in that front, right. So first is a lot of people are working very hard to make them less expensive. And that’s, you know, that is the Lord’s work as far as I’m concerned. And then we’re also evolving how we can do prompting to better leverage and make it more efficient. You know, the fact that these models, it costs a lot more for them to write something down, then for them to read something. Okay, well, if you do that, then you can kind of designed certain flows that are less write intensive, and can bring the cost down and the speed up. I think that’s an issue. And then I think, yeah, you’ve just got this range. I mean, for the, I don’t know, Greg, you got you guys tell me. But has there ever been a time where firms have dealt with the open the first letter from the client, and it says, You better not be using this tech? And then the next letter says, You better be using this tech? I mean, have you ever seen that before? With anything? I mean, this is?
Greg Lambert 13:24
Yeah, well, it was worded differently. It was, we want you to handle all of our legal work. But we want you to do it on the cheap. But we want it to be really good. So I mean, it’s just a different application of the same and, and quite frankly, I mean, it’s a little a little off topic for for this is I think that’s the the bad relationship between clients and law firms, is they don’t talk to each other. And so this is just one more tool where there’s different expectations, sometimes different expectations from the same people on how it should be applied. But how it, you know, I want you to apply it, and I want you to reduce cost. But I don’t want you using any of my stuff to do that.
Pablo Arredondo 14:10
So and then relatedly literally less one, and then let’s jump but another 2024 problem we’re seeing is like how do we regulate this as a profession, right? Do we need new rules? Do we need new duties, and you’re starting to see Task Force form, California release their roles, which is pretty laissez faire? You know, California was basically like, let’s go over all the rules that already exist and think to ourselves like these don’t stop existing just because new technology is here, which generally is what I like. Contrast that I think with Florida, which started getting into like, specifically how you should build this back to your clients. And I think, you know, cause some quizzical eyebrows among folks have been like, what first? Why are you even purporting to tell me how to do that? Secondly, what are you saying Does it make sense? So I think that’s another thing we’re going to be seeing is, you know, how do we regulate this stuff? If it Do we use existing rules? Do we create new ones, etc. Anyway, Marlene?
Marlene Gebauer 15:05
No, I just I wanted to revisit a thread that you just mentioned about sort of education. And you know, where do you think it’s critical, you know, in terms of education for users on these types of tools? Like what what are what are the things that we should be focusing on most?
Pablo Arredondo 15:26
So education has come up again and again, in different contexts. So I think starting with start with law school for a moment, right? So my view is a lot of law school education is absolutely unbothered by this, right? Like the torts, the doctrines of torts or contracts, this stuff goes back to like William the Conqueror, and how you think about these relationships in these rights and sort of think analytically, is not really tech specific. So I think that the danger there is just making sure that students aren’t like using this stuff to write their essay, and not thinking about it as rigorously as they could have, right. And that’s where you might see a thing. But then for things like advanced legal research, or writing or legal research and writing, of course, you need this stuff to be taught, right? It’s going to become just, you know, part of how you’re doing and and so I think it does need to fold in there. The problem is just getting the cost in a place where we can do it, right. It’s, it’s just a lot more expensive. Then comes this more subtle stuff about the training of junior Associates, and my thinking on this is evolved. So when the partners first used to say, what about all the things that they learn when they’re doing that tedious work? I was like, What a shocker. The person who’s building out that guy for 700 bucks an hour, is worried about the like, pedagogical cut. Right? And so at first I admit, I looked at it cynically, right. But as I think more about it, you know, when I was at Kirkland, Ellis, I had no idea how corporations worked at all, I had no exposure to them, right? I didn’t have family members, right? Doing those long nights of Doc review, right? Whether I knew it or not, I was actually was sort of through osmosis, ingesting certain aspects about how, you know, corporations run and those things were useful in sort of understanding litigation to some extent. So the analogy I make is sort of the Karate Kid, you know, Daniel said, like, you know, wax on wax off, he’s got to paint the, you know, wax the car, paint the fence, and you’d like why how can anyone learn anything? Well, it turns out that those are like components of something bigger. So I actually do think so basically, my evolution for whatever it’s worth is to go from cynical dismissals, like, please, you’re just saying that you want to be willing to realizing that there, there actually are some things that are learned during those tedious processes that we need to think about. How do you just balance that out? Right? How do you how do you make sure that that is still learned in those ways? In a world where increasingly, Large Language Models can do like a lot of that, that listing?
Marlene Gebauer 17:38
So the news dropped in in August of 2023, that Thomson Reuters acquired CaseText. And so Greg, and I’ve known you for a long time, and you know, we were very happy for you yet, we were we were a little surprised, you know, our favorite scrappy startup was painful to get into the car business. So what were your thoughts during this time, you know, as the decision was made to start working with Thomson Reuters?
Pablo Arredondo 18:05
Yeah, so the few things went into that. So of course, he was like, what, why is Pablo not sad about it? Right? That’s what you’re basically like indirectly saying, give it like, you know, he’s always grabbed for it. And the truth is, I’m not I’m not, I wasn’t, we weren’t beforehand. And now I’m really, really not. So I think a couple of things first, the the reality of unique content to point this out, right. And we had gotten as far as we could, trying our hardest to reproduce content. And whereas with our earlier technology, it was sort of annoying that we didn’t have you know, certain aspects content. When we started showing this stuff. Again, we were the only ones who had it for awhile, right? People were, you know, grabbing us by the shirt on, you’re holding up the wall be like, You better get this content. Right. I don’t have that because I need it. Right. So the demands for contents, you know, we can so intensified. And at the same time, our maturity about what it really takes to reproduce this content, right, you know, not just the primary sources, but all the editorial stuff, right? And, yes, when you’re competing, we’re like, nobody needs editorial stuff, nobody needs come on, you guys are just, you’re just making up that you want that administrative body, you know, really what you want is whatever state law you know, the truth is, is that, you know, it helps tremendously to have not just the primary sources, but also a lot of this stuff that’s folded into it. So that’s part one. And then the idea is we wouldn’t just spend 10 years reinventing the wheel we’d spend 10 years inventing like hassle wheel with cracks in it that wasn’t as good so I think especially with this new technology, you just feel this mandate to go and just bring it to its best embodiment as fast as you can. I mean, this is if you work in legal tech, this is it this is like you’re never going to have more of an opportunity to to bring good to the profession. So you know what, why not do the things that are just going to lead to that right. And then the other thing and this was something you know, a little bit maybe on faith we took but now we’re seeing you realize is that Thomson Reuters really is all in on this AI stuff. Like they really do see CoCounsel I think we’ll talk about it in a bit. it not just let’s take all of our product lines and enhance them with with generative AI, although by God means do that that’s a wonderful thing. But what’s this new category that’s going to exist where this AI assistant can follow you from surface to surface from Microsoft Word to your DMS, to your legal research? Right? And that category, I think, is they believe it in enough that they, you know, paid for us and we were more cheap, right, you know, we, you know, they paid for us. And, and we’ve seen everything we’ve seen since we’ve joined has been that they really do buy into that. And they really are supporting that. And so those are the two reasons why I don’t worry, I’m actually quite happy with it. And I hope in time that anyone who had any lingering doubts, will see that it was a very good thing that we joined.
Greg Lambert 20:46
On a side note, I will say before the announcement, Marlene, and I had a private conversation where we were wondering, did, did they miss out on being acquired? Did they wait too long? So that shows you why we’re on the microphones and not actually doing startup? So you know, but from what we’re seeing, and, you know, in this whole integration that, you know, it seems to be the it’s pretty smooth integrating with Thomson Reuters, the rest of the platform, you know, and there are multiple General Journal of AI tools that as users that we see moving forward, so you know, we’ve got the CaseText path, we’ve got the Thomson Reuters precision path, the practical law path, I’m sure there, there’s other big ones that we probably don’t don’t even know about. So what I want to know is, how do you and Mike Dane and Kriti Sharma and others, who are the AI bigwigs within within the company? How are you making both these separate paths work so that you all succeed? But, you know, at the same time you benefit from each other, but you continue to kind of have that CaseText personality, as well, which seems to still be there. How do you make that work?
Pablo Arredondo 22:09
Well, I mean, this CaseText personality is just so intense, I don’t know if anything diluted, I mean, at this point, you know.
Greg Lambert 22:15
You’re influencing you’re influencing TR rather than TR influencing you.
Pablo Arredondo 22:20
But it goes both ways, though, too. I mean, there’s there’s plenty that we’re learning from them as well. I mean, tr, tr has, you know, a lot of experience with a lot of stuff. And you know, you know, actually things so we talked about testing a bit, right? Oh, my goodness, for Thomson Reuters starts testing, it is the most beautiful, extensive, like, you know, they write up like actually what’s going on, right, which is more than we had our resources to do, right, we’re choosing this kind of crude doesn’t flag and things like that. So I think the key is to take the best from both cultures and and combine it together. And what I think is gonna, what makes that so important. And what I think will facilitate it is that we all have this vision now of this AI that kind of floats across the different product lines, right? Even product lines. It those silos existed because they had to because it was too hard to have things talk to each other. And there wasn’t, you know, how much knowledge do you really gain if you do it? Well, now, thanks to this new technology, the gains are going to be tremendous. And, and I don’t want to make it sound easy. And I can see all the engineers rolling their eyes like oh, look at public talking about how easy it’s going to be. You know, a lot of work goes into doing it well. But I do think that this technology also makes it a lot more feasible to have a system that could go talk to like six different platforms, or products, and then and then bring it together.
Greg Lambert 23:33
So it’s not just putting a chat bot in there and making it all work.
Pablo Arredondo 23:36
No, no, I mean, if we here’s the thing is like, you want to slap you want to slap AI and everything to mix with you everything it touches better. But again, to reiterate, I think there’s that that sort of bigger picture, right, this new category of legal AI assistant, that’s really where the competition is now and in where it shouldn’t be. And I you know, I welcome any and all folks who are competing with us in that space, because that is going to be I think legal Tech’s biggest contribution to the practice of law is whatever comes out of this, this competition.
Marlene Gebauer 24:07
So switching gears a little bit, so just a week ago, CaseText for Canada, and Australia just just launched. And that’s it just like I’m sort of seeing a trend this way to where, you know, we’re seeing a lot more for an international, you know, types of, of, of access. So how’s the team adjusting to expanding the resources beyond the US and what are plans for possibly additional areas?
Pablo Arredondo 24:38
Yeah, I mean, my colleague, Laura Safdie had hurt over last year. She’s like Herculean work, dealing with all the contract stuff just to the United States. And now it’s like Oh, joy, Laura, Laura, look, here’s a globe Do you mind just can we just get it everywhere? So so a lot of very cheap cheese, you know, quickly having to learn all the regulation and get up to speed on that along with, you know, her colleagues in that area, but this is to my point right now. Within months of joining firstly, with tons of writers, our stuff is now being sent around the globe, right? So if you’re sincere about wanting to do this stuff, right? Do you see how that’s a better path than even though you lose that kind of scrappy individuality which stuck around? Yeah, who doesn’t love feeling that, you know, it’s a good feeling. But like in terms of just actually making the world a better place, this is the kind of stuff that we can do join forces with Thomson Reuters, where literally, it would be 10 years before we got to Australia or Canada, in the in the other world. And so, you know, there’s a lot of competing regulations right now around AI, which is going to be interesting, right, England’s taking one stance, the European Union is taking another. So we’ll be sort of on the will be affected by that it’ll be really interesting to see like, what places can we go to? And what places do we need to do this or that group? But, you know, the fundamental technology seems to work on on all major languages. Although, you know, with the caveat, so we’re working with Harvard Law Library Innovation Lab, translating French statutes, and then having experts go and look and say, like, is this capturing the nuance, right, because it’s one thing to translate a poem or like a birthday card, or kind of, you know, generic things. What happens when you try to translate actual statutory language, and we’ll be, you know, publishing something about soon and making the translation available. But for the most part, this technology is ready to roll and ready to rock and roll in certainly every English speaking country and a lot of non English countries as well.
Greg Lambert 26:26
So probably let me let me ask him on a personal level, how’s your work life integration changed? From moving from the startup who, you know, I would get emails from you, you know, in the middle of the night, are you because you’re on the West Coast? And I was, I was in Central time, you know, I would see them in the morning, but they’d be like, you know, three o’clock? Yeah. Are you still having the late late days? Or are you able to kind of take take a breath and and not have to answer every question that comes your way?
Pablo Arredondo 26:59
Yeah, I mean, that’s, you know, not when you’re a co founder, every it’s everything in anything. And I think for many hats, you did this last year, honestly, this last year, you there’s no, you don’t want to glamorize it. It was tough. It was tough on my family, it was tough. I’m, you know, I think my wife’s credit was like Ken GPT-4 automatically divorce hear from somebody and I Yeah, but um, but um, you know, that that, you know, your startups are pretty all consuming. And then the nature of this technology. I mean, this is something where you can finish your day of work, and then go see an essay about what is the impact on humanity, right or on the society, right. And it’s also changing so quickly, so that you’re constantly like, Oh, my God, this is somebody miles. So it is it is extremely consuming. area to be working in right now. But that said, Yeah, with tons of writers like, it’s, it’s not, you know, they’re not, it’s different than CaseText, in terms of things that they feel that I should be directly involved in. And I welcome that, as well. We have very good people handling it. Can I divert it to one year, I want to try this out with you guys, and tell me what you think. So if you think about like other big moments, where we harnessed stuff to then create value, so think of like oil, right? So you had millions of years of all these prehistoric little ocean animals. And then you had the force of geological pressure on that for a second. And now we have oil, and it’s like, Oh, my goodness, right. You know, nuclear power, we harness the atom power from like, the threat with these Large Language Models are what’s so interesting to me about them is we’re harnessing our own words here, where we really did was take everything ever written, all of the combined effort of anyone who wrote anything down, and then basically figured out how to, like compress that into this new thing. Right. And so we’re harnessing our own work for for the first time. And I think a lot of the issues, we’re having the bias and the this and that, right. It’s because now this is us, like this is our collective work as a species that’s really going to I don’t know if that adds any value to anything anyone’s thinking about. But do you see what I mean? It’s, it’s a qualitatively different.
Greg Lambert 29:04
Yeah, resource. Yeah. And I think that was probably the reason that you saw the legal industry jump as quickly as you had, because this is the first time that we’ve been able to kind of have this model based on our on language, not not on keywords, not on database structure, not not on organization of the information, but just, you know, taking it as messy as it is, and finding out you know, the results can also be messy because of it. And so, to me, and maybe this is a little off from from your point. The thing that I’m hearing over and over and over again, is that for this to really work, we we you know, this has been the overnight 25 Year problem. Law firms data is super messy and and In order for us to really take advantage of this with our own information, we’ve got to start cleaning it up. And my hope is that we’ll be able to leverage the resources that are coming out from the LLMs. And the way to integrate that, and again, in going back to your, it’s a lot easier for it to input information than it is to export information. You know, being able to leverage that in a in a, in a way to standardize to normalize the information is going to be huge. And I don’t think it’s going to be easy, I think it’s, it’s going to be a struggle, somebody’s going to figure it out. But right now, everyone knows, it knows we need to do it. But we don’t know, really how to start. So maybe that’ll be the next startup for us.
Marlene Gebauer 30:56
And I think it’s given us the opportunity to be a little more introspective, because, you know, we’ve seen, you know, based on these models that, you know, what all of our colleges is, you know, it’s not always so great. And, and, you know, there’s there’s bias, and, you know, maybe these hallucinations are kind of based on, it’s based on the information that it’s getting. And, you know, I think there’s a real opportunity to sort of look at this, you know, it’s like, yes, it’s incredibly powerful, but, you know, are we doing it the right way? Is the output, right? You know, are we taking care of everybody’s rights? Who might be touched by this? And, you know, sort of looking at is like, okay, we can do it? Should we do it? Or if we should do it, how should we do it?
Greg Lambert 31:50
Well, and to take your oil analogy.
Pablo Arredondo 31:54
We did it, what happens if we do it 10 times bigger, let’s find out. And that that’s, that’s our version of that.
Greg Lambert 32:00
Yeah, and just to kind of to wrap it up, going back to the the oil analogy. I mean, you can get, you know, you can get oil out of the the Sand Tars, there’s tar sands in Canada, but it’s, it’s really complicated and hard to clean that up. Whereas, you know, if everything was a nice, light, sweet crude that you get from, you know, from the North Sea, or from Saudi Arabia, be a lot easier there. But those are limited resources. So we’ll see what we do with the with our resource, right.
Pablo Arredondo 32:34
Great. we are talking to a Texas, man, you know,
Marlene Gebauer 32:36
That’s how you can tell.
Pablo Arredondo 32:39
I think that that’s right. And and you know, that we don’t know what the most powerful models are trained on. Right? We don’t. But I will say that, at some point, once it’s enough, it’s like, are things that different if like, Oh, we did this enough, right. bracketing, like the copyright and all that stuff, right. I think at some point, it’s such a huge mass of of language, that I don’t know how much it matters, what the precise contours are, I guess, but so we are. Yeah, so there’s a few things going on, we’re building out the ability to build out AI, if that makes sense. And, you know, it’s one of these things where creating like a skills factor, if you will, right, the system by which you can start to do that. Now, what’s frustrating about that is that it’s not user facing. And so there’s this delay, where people are like, What’s going on over there, what’s going on is we’re refining and iterating, the process by which we can then create these new skills and these new functionalities. So that’s a key key aspect of it. We are then looking to, you’re going to see generative AI ai enhancing current product lines. And that that is, you know, wonderful in its own right. But we’re already you’re also going to start to see an assistant that can span multiple platforms. And that’s what I think we’re all most excited about. It’s not an overnight system, but it really represents I think, this like paradigm shift and what a piece of legal technology is, and that’s where you know that that’s that’s a lot of work is getting it right every time there’s a new task it does, there’s like a risk of failure for that aspect of it. But I think that’s the big picture. That’s what that’s what’s coming next. And don’t get me wrong along the way. I mean, we’re gonna this is gonna be you guys are no carers did a dear to my heart brief analyzers, right? So like, you can drag and drop a brief the very earliest ones right? Were quote checker right, you could check your quote, then Kara, our stuff, it wouldn’t suggest new cases. Now you have the idea is that what you can do is, are the fundamentally misrepresenting the case that they’re citing, right? Or the transcript that they’re citing the idea of having a computer that can code through a brief and all of the supporting exhibits and case law and flag instances where they just don’t see the line? That I mean, that is, you know, with this AI, I think there’s two mistakes people make some people think, Oh, it’s a robot lawyer, and it’s going to take all the jobs like nonsense, but the other mistake is, it’s just going to be used for pedestrian things. It’s just gonna be used for administrative thing you know, it’s just going to make declar and stuff. And the truth is in the middle, right, you know, finding substantive discrepancies between what a lawyer is represented to the court and the evidence is supported. That is at the heart of litigation. I mean, that is people do go to law school to get to do that over and over, right? It’s the most pure motivations to want to smack someone else down. But here we are. So I think that, you know, products like that, like when I when I talk about enhancements to existing product lines, it’s not just going to be like a chat bot, that is a nicer interface, you’re going to see, like, you know, enormous expansions and capability of existing flows like quick check, and other things like that.
Greg Lambert 35:38
Very cool. So, Pablo, we can’t let you get out of here without asking our crystal ball question. And I know we’ve kind of touched on on things. But let’s pull it into one coherent response if possible. So what do you see over the next couple of years, when it comes to advancements in legal technology and legal research tools?
Pablo Arredondo 36:03
I think over the next couple of years, I would think you’re going to, it’s going to become much more ubiquitous, because price is going to fall, speed is gonna go up, right? So it’s sort of the underlying hardware is going to make this something that’s that’s much, you know, in a lot more places, you’re going to see entirely new client offerings, things that law firms never used to do, but are suddenly now involved in doing part of that, perhaps, to us to compensate for erosion of certain billing, you know, for around certain tasks. And I think you’re going to see some but certainly not like sci fi, versions of agentic behavior, right, where the AI is able to go and complete tasks for you. I think that that’s one of these areas where like, it’s very easy to do an early demo and say, Oh, look, it’s here. You know, look at it. It went through Zillow for me, and you know, yeah, I think to really get that to a reliable place will take time. But two years from now, I’d be surprised if you’re not, you know, seeing some some precursors of that kind of behavior. And then obviously, the biggest caveat, which is like, so much of this depends on who’s right about the plateau of scaling up this technique. There are those who think we basically hit the plateau, GBT five won’t be that different than GPT-4. And there are those who think that we’re gonna scale it up and see the same sort of quantum leap. And I like, tell me because disagree. Like, what would what would it look like to have as big a jump? To GBT five as what four was to three? Right? Think about just how insane it was when you first started seeing what GPT-4 could do. Now imagine in two months, there’s another one. That’s right. It could throw a lot of predictions out the window. But, you know, so, yeah, that would be the most I’d hazard on a crystal ball.
Greg Lambert 37:47
Yeah, and what I’m what I’m seeing on on the big advancements are more in the multimodal, and so more on audio more on video. Multimedia. That’s, that’s a huge jump. And I don’t think people really appreciate how big that jump is. At the moment
Pablo Arredondo 38:06
We prototype something, you can fit it an image of factory workers being negligent in various ways, and it’s scanning it now. And then we use petricola content. So it’s finding the precise GFR violations. So it’s literally like like policing the factory for OSHA violations very,
Greg Lambert 38:21
You’re using that picture of the guy that using a forklift to change a light bulb? Is that okay?
Pablo Arredondo 38:26
Well, this is I’ve got I’ve got a forklift lifting another forklift. And this is, you know what, like, so many great things with Evan shake minute Fisher. And then near term, the context window is gonna get a lot bigger, we’re starting to see flickering of that, which really matters, right, because then you’ll be able to feed a lot more information at once and have the AI synthesize and analyze it, you know, like 10, different 100 Page merger agreements, you can look at it all at once, which can then you can start to do like trends and things like that, that aren’t really as easy to do when you have to chunk everything. So it’s just kind of something like I was, I was so excited about this technology in April of 2021. But all it was doing was making it a little better to find, you know, the cases and documents you’re looking for. You can imagine sort of like supernova of excitement. And so, you know, we’re all in this together. It’s again, not overnight, but it is decidedly not a fluke. We’re like a free domain, something we’re like, oh, it just feels about it turns out it wasn’t really there. It is emphatically not that and it’s just now a matter of doing the hard work to put the guard rails and to bring it to bear.
Greg Lambert 39:34
Awesome. Well, Pablo Arredondo, Vice President CoCounsel at Thomson Reuters. Always a pleasure. Thank you very much for coming in and celebrating your one year anniversary of CoCounsel with us.
Marlene Gebauer 39:46
Thanks, Pablo.
Pablo Arredondo 39:47
Thank you guys. Always a pleasure.
Marlene Gebauer 39:50
And of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoyed the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I could be found on LinkedIn, or on X at @gebauerm and on Threads at @mgebauer66.
Greg Lambert 40:07
And I can be reached on LinkedIn or on X @glambert or @glambertpod on Threads. Pablo, if someone wanted to learn more, we’re where’s your preferred place online for them to go?
Pablo Arredondo 40:21
Yeah, my email got a little bit longer. It’s Pablo.Arredondo@ThomsonReuters.com. I did ask them could I just have Pablo at Thomson Reuters and they sort of looked at me and blank they were like, did you just you just don’t know what’s going on.
You Yeah, Pablo.Arredondo@ThomsonReuters.com always happy to follow up talk shop demos, whatever you guys want.
Marlene Gebauer 40:46
Okay. That is always the music you hear is from Jerry David DeCicca Thank you so much, Jerry. Thanks, Jerry.
Greg Lambert 40:52
Alright, thanks everyone.