[Ed. Note: Please welcome guest blogger Noah Waisberg, CEO of Kira Systems, and a good friend of the 3 Geeks. Noah was on an ILTA Panel with me last year, and will participate in the follow up to that panel this year called, Legal Technology Innovation: Bolstering AND Destroying Legal Work. This post originally appeared on the Kira Inc. Blog. – RM]
Watson is almost certainly the most significant technology ever to come to law, and it will give lawyers permission to think innovatively and open up the conversation about what is possible in a field that has been somewhat “stuck.”
–“10 predictions about how IBM’s Watson will impact the legal profession“, Paul Lippe and Daniel Martin Katz, ABA Journal
IBM’s Watson AI has received a lot of attention for how it might change law practice. Should it? Or should commentators expecting “Watson” to change the world instead refocus their attention on “artificial intelligence” or “machine learning”?
Recently, legal market observer Ron Friedmann wrote, in a post on potential business models for Watson in the legal space:
if we aim to improve the efficiency of the legal market, there is no lack of technology to choose from. Whether Watson is the best place to bet remains an open question.
- What is Watson currently good it, and is it even the best avenue for automating tasks where it is strongest?
- Is Watson strong in the most promising areas for legal automation?
- Will Watson grow to become leading machine learning or AI technology across the board, or will it remain high quality only for question answering?
From Wikipedia: “Watson is an artificially intelligent computer system capable of answering questions posed in natural language”. Today, it appears to be a leading machine learning offering for question answering of a very specific sort (as we will cover below). IBM looks to be attempting to build Watson out into leading general purpose artificial intelligence software, but there is no consensus that it is at or better than the state of the art in areas beyond question answering. Indeed, early reviews on released Watson APIs have been underwhelming. As Ron Friedmann points out, it is not even clear whether Watson is a better technological approach for legal question answering tasks than, say, Neota Logic.
Apparently, one vendor is currently using Watson to extract data from contracts. I have yet to see any data suggesting that identifying contract provisions is in Watson’s sweet spot (by data, I mean information such as, say, published provision extraction accuracy numbers for a system built using Watson; the vendor claims their “goal today is to deliver a 20% cost reduction for a law firm in a typical diligence exercise”, which would not stand out relative to claims from other contract review software vendors (e.g., our clients tell us they find from 20–45% time savings on page-by-page review using our contract review software, and 60–90% time savings when they rely on it more heavily)).
Is Current Watson Right for Legal Problems?
Watson is currently a leading technology for question answering tasks. Are most legal tasks that could be impacted by software question answering tasks?
“Question answering” could be very broad, and most or all legal tasks could be interpreted as giving answers to questions. However, today Watson only stands out for performance on a narrow definition of question answering.
Each of the following questions illustrates a type of legal problem technology could help solve:
- Is it illegal for individuals to have ferrets in the state of California? (Watson-type question answering)
- Which of these 1 million documents are relevant to determining if anti-competitive behavior occured in this specific case? (eDiscovery technology-assisted review)
- How long does it tend to take for cases to get to trial in front of Judge Vernon Broderick? (Lex Machina)
- Who will the Supreme Court decide for in King v. Burwell? (Katz/Bommarito/Blackman algorithm)
- Can you draft a brief for us to submit to the court for this case? (NarrativeScience, Automated Insights (neither appear to be currently targeting legal))
- Do any expense items on this legal bill seem inappropriate? (SimpleLegal)
- Which of these contracts have change of control or exclusivity clauses? (us (Kira) and others)
Will Future Watson Be Better Than Alternatives?
But as Paul points out, Watson’s R&D investment is probably 100x all these companies combined, and so has the potential to ride a much steeper performance curve.