9/14/11

Supercomputers: The Ultimate Knowledge Management Pipe Dream!

Image [cc] travelin' librarian
I'm going to paraphrase an Archimedes quote – "Give me a big enough database, and I'll forecast the world." In a way, that is the ultimate goal for those of us in the Knowledge, Information, and Intelligence profession. We have a deep craving and expectation that if we just had all available information in a database, combined with acquired knowledge of how to interpret that information, plus an advanced algorithm to pull it all together, we could predict what is going to happen next. For those of us in the legal span, we think of it in smaller chunks of "who do we need to hire?" or, "what can we forecast as legal exposure for our clients?" However, we are not the only ones that believe that "if we could just compile all of the data, we could actually predict the future."

According to an article on the BBC News website, that idea may not be as unobtainable as you might think. The US government, through its Open Source Centre, combined with BBC Monitoring and numerous online news resources may be blazing a trail in finding a way to predict the future through data analysis. Although it is much more complicated than this, here is the basic formula they are feeding into a supercomputer:
  Supercomputer
x 100 million articles
x automated sentiment
x geocoding            
  Predicting Trouble
The Supercomputer (Nautilus at the University of Tennessee), processed all of these articles (legally obtained through agreements, or available on the open web), added in a factor of "sentiment" (positive or negative tone), and put a geographic location on it and voilĂ … trouble can be predicted.

Now, at this point, I've got two competing ideas jumping around in my head.

  1.  Is this simply a "Monday Morning Quarterback" Issue?
  2. Could this be a huge deal and produce the next "WestlawNext 2.0"?
Let me take the Monday Morning QB issue on first. Does this simply allow us to have an "aha!!" moment by looking back on the data, and then plugging in the additional pieces of information that we didn't have before the major event happened? Kind of like what we all did when the Enron fiasco happened, and then we all started looking back at the "signs" that something bad was about to happen. In other words, it all looks so clear when you look back on it, because you now know what happened. 

On the other hand, let's say that this really is something significant, and we finally figured out a way to predict events with about the same accuracy as we predict the weather. If that's the case, then think of all that information that the big publishers and news outlets like Thomson Reuters, Reed Elsevier, McGraw-Hill, Wolters Kluwer, etc. are sitting on, and just think of the possibilities they could do with the "Predicting trouble" formula. That thought makes me concurrently excited, frightened, and discouraged. I'm excited because there is an amazing potential in helping forecast events, trends, legal issues, etc. I'm frightened because this puts a lot of potential power in the hands of these corporations. I'm disappointed because I know that none of these corporations will actually do anything with it unless they can figure a way to corner the market and profit substantially from their efforts. 

I'm actually leaning toward believing this is all a Monday Morning Quarterback issue. It just seems that without the actual information of what the "event" is, then what you're left with is an incomplete formula that simply helps you narrow down a guess to an educated guess. As a person that still holds on to that dream that with enough information, and the right tools to process it, we really can predict the future… it would sure be cool to see if we could test this formula out on some massive information databases and test out this hypothesis.



Bookmark and Share

2 comments:

Scott Preston said...

Interesting post Greg. What if you apply this to stock trading? This is not a stretch at all and there are financial companies that have the data, the intellectual property, the money and the drive to make this happen. I would venture to say that it is happening now (at least the development phase). Think about the impact this will have on our economy.

Frank Schipani said...

This reminds me a lot of Asimov's Foundation series. Now the computing power exists to actually make it happen. See this article on Wikipedia

 

© 2014, All Rights Reserved.