As I mentioned yesterday, a group of bloggers traveled to ThomsonReuters (TR) in Eagan, Minnesota earlier this week to get a first-hand look at WestlawNext (WLN) and talk with the Project Cobalt team, meet briefly with TR’s CEO of Legal, Peter Warwick, and discuss the the functionality of WLN with Westlaw’s Reference Attorney staff. There are a number of articles that are out from other bloggers that a range of issues from Lisa Solomon’s discussion of Product & Pricing; Jason Eiseman’s video interview of
myself me, Tom Boone and Jason Wilson; Robert Ambrogi’s discussion of West Search functionality; Betsy McKenzie’s view of WLN from an academic perspective; Ken Adam’s survey on CALR value in contract drafting; David Bilinsky’s Top 10 list about WLN, and; Simon Chester’s discussion of WLR from a Canadian perspective. I wanted to take a different approach and talk about the back-end structure of the new West Search Engine and how they have used Knowledge Management theories to create an algorithm that looks to be much better than the current Westlaw.com search results.
WestSearch – Leveraging 100+ Years of Knowledge
Westlaw has compiled millions of pieces of value-added information through editorial analysis of its research attorneys, but that information has been compartmentalized into individual databases and has had a slow transition from the traditional print uses of this information to the computer database search engines. Even when West created KeyCite as a competitor to Shepard’s, it was pretty much still a stand-alone citation system that added-value to the individual cases and statutes, but not really a great enhancement to how searchers retrieved results from their searches.
Now WLN has finally seen the value of four distinct products/processes that not only help as individual value-added products, but actually determine search results rankings, pushing better results to the top based on past knowledge rather than by simple algorithms of term location or dates. Here are the four products/processes that now affect the ranking of search results:
1. West’s Key Number System
The first three categories are really no brainers when it comes to leveraging related pieces of ‘information’ on a similar topic. Now, instead of just getting results that have a term frequency that matches the words in my search query, WLN literally runs an algorithm of over 60 queries that determine alternative words that may apply to my search, top cited resources on that topic, and resources that are deemed to be authoritative through secondary resources. I like the fact that the Project Cobalt team stuck to the term “algorithm” and shied away from “artificial intelligence” (although Peter Warwick did let the AI term slip in his talk.)
Perhaps the most intriguing part of the change in algorithm of WestSearch is number four – “User Searches & Resulting Actions.” This is something that those of us interested in improving Knowledge Management resources see as the holy grail of KM. If we could create a system that leveraged the experience of everyone in our firm in a way that allows our KM tool to become “smarter” we’d jump at that chance. From what I saw, it looks as if WestSearch is making a strong run at making their search algorithm smarter through adapting results based on previous users key actions. In a way this is crowdsourcing on a very high level.
Crowdsourcing the Researchers
I’m a fan of crowdsourcing, and of knowledge management. Combining the two is like putting peanut butter and chocolate together and coming up with a two great things that work well together. WestSearch crowdsources the WLN users by monitoring key actions after the search results are returned. WestSearch takes actions like ‘print’, ‘save’, ‘folder’, and ‘view’ that a searcher performs and logs that information for future reference. If another researcher runs a similar search later, the actions from the previous users is taken into account and the results are influenced by those previous user actions to potentially rank items higher or lower in the results list. The thoughts behind leveraging previous user actions against search results is that the ‘crowd’ will tend to identify and use the same documents when searching specific legal issues. When pressed on this, the Cobalt team said that actions like ‘print all’ or ‘save all’ are not logged because they are looking for specificity over generality. Also, because it is a ‘crowd’ based view, the law student that doesn’t understand the issue and picks less valuable documents will have little influence because the majority of the crowd will choose the ‘best’ documents over and over again, thus improving the algorithm.
What’s the Future?
It will be interesting to watch as this rolls out and WLN adds more and more databases to the system. I will be really interested to see what happens when the “news” portion is integrated and how that will affect the WestSearch algorithm on ‘new’ or ‘hot’ topics that pop up from time to time. I look forward to watching this approach of leveraging existing knowledge in an advanced algorithm to see if it does get ‘smarter’ over time. I’ll also be interested in seeing if WLN can improve its existing West km product (which will still work with WLN), and the ‘foldering’ features that could actually make WLN an additional in-house KM resource.