Law Firm BS – Tier 2 (Chihuahuas, Muffins, ROSS, LTC4)

By Casey Flaherty on November 5, 2017

“Every time you call something a ‘robot lawyer’, God turns a chihuahua into a muffin.”

– Ed Walters at the 2017 COLPM Futures Conference

As CEO of Fastcase, Ed Walters is a true leader in the application of AI to the legal market. He created the AI Sandbox. He speaks all over the world in addition to teaching at Georgetown and
Cornell.

But Ed is a horrible hype man. Simply terrible. He suffers, inter alia, three glaring deficiencies:

Ed knows of what he speaks
Ed is accountable to the market for delivering functional products
Ed is constitutionally incapable of bullshit

What a wretched combination. It is a wonder we permit this man to ascend the dais. Instead of cheerleading for robot magic, he offers a balanced assessment of what various forms of AI are good for while identifying immediate, practical applications of AI to law, and the limits thereof. It’s all so painfully responsible. And boring!

Well, not quite boring. Ed is hilarious. One bit that always gets a laugh is when Ed explains classification errors (and their implications) by reference to the difficulty in distinguishing a chihuahua from a muffin:

If you take delight in such silliness (and I do), there are similar memes, like labradoodle or fried chicken, dalmatian or ice cream, and corgi or bread. Ed, however, deploys humor to make salient points about the challenges of subtle differences and the potential dangers of adversarial examples. Ed is also the best presenter I’ve encountered in explaining the errors in extrapolating from microworlds. I’ll leave the AI to Ed.

I want to talk about puppies or bagel. Or, more pointedly, the distance between having similarities and being the same. Accurate identification should be a neutral act. Correctly categorizing an item as a bagel rather than a puppy is not to disparage bagels or puppies. You can adore both without confusing one for the other.

Which brings me to Tier 2 law firm bullshit (parts 1, 2, 3). In my taxonomy, Tier 2 law firm bullshit consists of statements that present as positive but ultimately prove non-responsive to the question posed. On the surface, the responses resemble attempts to answer but closer inspection reveals only superficial similarities to contextually useful information. I ask for a puppy. They send me a bagel.

I selected two examples—ROSS and LTC4—precisely because I am predisposed to react positively to their inclusion in an RFI response. The bullshit is not in the initiatives themselves but the way law firms use them in answers. An exemplar question:

Technology: How does the firm integrate technology into the delivery of legal services to clients like [organization]? How has technology measurably improved performance? Which recent technology investment have made the biggest, measurable impact?

How does the firm ensure that its lawyers and staff are taking full advantage of the basic technology tools at their disposal (e.g., competence-based learning curricula and micro-certifications)?

Impact. Measurable impact. Measurable impact specific to the client. Good in general is not the same as useful in the particular. I therefore include introductory instructions like:

While an initiative may be truly innovative, that in and of itself does not mean it is valuable to [organization]. An automated private placement document generation system iterated near perfection is worthwhile accomplishment but of little practical use to [organization] if you only support us on employment litigation.

ROSS

It was intellectually interesting that Watson won Jeopardy! It was welcome news when ROSS Intelligence was founded to bring the power of Watson to the legal market. As a legal tech junkie, I can applaud. But with my outside counsel management hat on, I’m not permitted to care until lawyers use ROSS to benefit my clients.

Multiple times, I have received excruciatingly long (so long you’d think I’d written them) responses to questions like the above. They start quickly with “We have licensed ROSS” and then slog on for paragraphs about Watson, Jeopardy!, IBM, cognitive computing, and the coming revolution in legal service delivery.

Let’s unpack some layers of bullshit.

First, “we have licensed product X” borders on being a content-free statement. The quantity of software firms license and then do not deploy or deploy in a limited fashion—e.g., to the one lawyer who requested it—is astounding. The fact of a license, absent additional information, has almost no probative value.

This is particularly true when the subject area is sexy and law firms have recognized that the best publicity comes at the beginning of initiative when potential is limitless. If I had dominion over one of those growing law firm marketing budgets, I would be tempted to earmark funds to invest in AI (or blockchain or whatever is currently hot) for the purpose of press without any intention of following through (which requires real resources and reckoning with reality).

Second, at the various times I encountered these responses, ROSS had only released their bankruptcy product. This is fine, for ROSS. Bankruptcy is a legitimate area of legal practice and a solid silo for ROSS to attack first. But I’ve never done an RFI for a bankruptcy panel firm or matter.

Translated into something English adjacent, my Tier 2 exchanges with law firms are akin to:

Q: Talk to me about how you are using technology to improve toxic tort defense.

A: We’ve purchased but not deployed software to improve bankruptcy research.

I’m bullish on Andrew Arruda and crew. I’m excited for their expansion into other areas (labor & employment is up next). I was giddy at the news of their Series A and Andrew’s TED Talk and their A2J partnership with Northwestern and their partnership with Evolve the Law and…..I, for one, welcome our new Canadian overlords.

Law firms should talk to ROSS. Law firms should test ROSS. Law firms should license and deploy ROSS where its offerings enhance their practice. Law firms should do everything they can to integrate useful tools like ROSS into the delivery of legal services. But, in the context of an RFI, don’t talk to me about ROSS, or anything else, unless you can connect it to a concrete benefit for the client.

LTC4

By contrast, LTC4 may well offer immediate, concrete benefits. But a third party is in no position to evaluate those benefits.

For background, LTC4 is the Legal Technology Core Competencies Certification Coalition. “LTC4 is a non-profit organization, that has established legal technology core competencies and certification that all law firms can use to measure ongoing efficiency improvements.”

As evinced by the Technology question above (as well as a big chunk of my personal history and professional evangelism), I am deeply invested in technology training and certification. Getting better with the tools already at our disposal is among the lowest hanging fruit in improving legal service delivery. I was ecstatic when I learned a coalition of law firms was promulgating core competency standards.

But the LTC4 standards are proprietary and private. Their standards are only accessible to members. This, again, is absolutely fine, for LTC4 and its members. I’m sure the members thought long and hard about how to structure their organization and protect their IP. Tradeoffs were made.

Yet the deliberate opacity does not matter for the most common LTC4-related response I receive. The firms simply tell me “we are LTC4 members.” Like we licensed product X, this statement is virtually meaningless. In short, the firm has access to learning plans.

But I already know that. Everyone has access to learning plans. Whether from Ivy Grey, Deborah Sandoval, Richard Heinich, et al., there is a bevy of free, quality resources in the what-lawyers-should-know-about-X-technology genre on top of a wealth of already great books, classes, trainers, and tech.

Our primary technology training challenge is not determining what legal professionals should be trained on. Nor is it making training available—the internet has ready answers for almost every core technology conundrum. Our primary technology training challenge is getting people into training and ensuring that they learn. Access to training plans by itself has essentially no informational value in indicating that this challenge is being met.

That’s where certifications come in. On this front, LTC4 might still solve the problem. Our legal professionals are LTC4 certified is definitely something. But it is not quite enough. The transparency trouble remains. A third party is in no position to evaluate what certification means. Certified in which subject area? What did the legal professionals do to demonstrate competence and earn certification?

From some personal exposure and given who is involved, I have confidence that LTC4’s offerings are solid. And I can’t wait to see how they continue to evolve and enhance the legal ecosystem. But that fact does not make their standards any less private. I would hope that firms could find a way to provide more information about their LTC4 certification without violating the terms of membership.

In RFI responses, however, firms have a bad habit of creating shorthand and then treating it as a shortcut.

Firms seem to really enjoy naming programs. Apparently, the act of naming renders the seriousness of the program self-evident. The Geek Firm, for example, might have GeekLPM©®™, GeekKM©®™, and Geek360©®™—the firm’s project management, knowledge management, and customer relationship management programs, respectively. And, like LTC4, these programs have the potential to be excellent. That is, the programs themselves are not necessarily bullshit. What is bullshit is the way the shorthand is substituted for meaningful description. As in:

GeekLPM©®™ is the firm’s proprietary legal project management system. Synthesising Lean, Six Sigma, Design Thinking, TQM, TOC, BPR, KFC, QE2, and B2B, we’ve optimized project management to be fit for purpose and deployed it throughout the firm.

This may be true. This may be great. Or it might be marketing claptrap devoid of underlying substance. The burden of proof is on the firm. And statements like the above fail to satisfy the burden.

Where Tier 3 bullshit is obvious nonsense, Tier 2 bullshit may obfuscate something real.

As always, this is not about law firms = bad, clients = good. Rather, the poor quality of RFI responses is symptomatic of the situation. Neither clients nor law firms have much experience or comfort with data-driven dialogue. And law firms have recognized that, for the time being, bullshit offers the path of least resistance because it is often more effective than transparency. Transparency means conceding you are a work in progress—anathema to a profession of atelophobes allergic to admissions of imperfection.

Law firms should invest in improvement even if said improvement is not accessible and appealing to every client. License and deploy ROSS because it improves your bankruptcy practice. If you also want to weave ROSS into a broader story about the firm’s willingness to experiment with cutting-edge technology, go right ahead. Just don’t make it too much more than it is.

Law firms should market. And marketing should respect attention spans (even if I don’t). Hire experts in branding and copywriting. Use GeekLPM©®™ or whatever as shorthand on your websites, on your flyers, and in your brochures. Soundbites and happy talk have their place.

But brandspeak and chit chat should give way to real conversation at some point. My issue isn’t that law firms bullshit. We all bullshit. My issue is that law firms have grown so reliant on bullshit that many can’t switch modes even when the context demands a different kind of communication. Bullshit becomes standard operating procedure.

Next post, I will talk about the crème de la crème of law firm marketing bullshit. Tier 1 is bullshit so smooth even I can’t be sure it’s bullshit.

The full bullshit arc:

______________________________________
D. Casey Flaherty is a legal operations consultant and the founder of Procertas. He serves on the advisory board of Nextlaw Labs. He is the author of Unless You Ask: A Guide for Law Departments to Get More from External Relationships, written and published in partnership with the ACC Legal Operations Section, and the Service Delivery Review Primer, written for the Buying Legal Council. Find more of his writing here. Connect with Casey on Twitter and LinkedIn. Or email casey@procertas.com.