Stupid A.I.

Since IBM is in the news this week for all the wrong reasons, I thought I would take a look at their marquee product...The latest Wonder of the Modern World...The Trebuchet that threw Alex Trebek... The Future Savior of the Legal Profession... of course, I am referring to Watson AI.

A couple of months ago IBM announced that they were starting up Watson as a Service, they call it Watson Analytics, but I prefer WaaS.  ('Cuz you know some IBM marketing people totally bounced that one around for awhile before settling on the boring name. Probably dropped it because it sounded too much like the guy on the right.)

I signed up, logged in, and started to explore some of the sample data sets that Watson Analytics had available.  I chose the "SportsDataLLC NFL 2014 Offensive Stats" and I set about trying to stump the great Watson. I wanted a question that was a simple calculation, but was sufficiently unusual to have not likely been preprogrammed by Watson developers. After a moment I typed...
What is the average number of first downs in the second quarter by teams that led after the first quarter?
Click to see results
Stumped baby! 
Question number one and I busted The Watson.  
Dude didn't have a frickin' clue what I was talking about! 
So much for natural language querying. 
Goodbye AI!  

Once my ego subsided (which took much longer than I care to admit publicly), I checked the actual data set and realized that there were no fields tracking First Downs or Score, let alone Score by Quarter.  So, I didn't actually stump Watson, I just asked him a question for which he had no data.

Which raises some interesting questions on its own. First, why did he suggest the alternative query "How does the number of Week [sic] compare by Team?"  And secondarily, what the hell does "How does the number of Week [sic] compare by Team?" mean anyway?

Intrigued, I clicked through to see exactly how the number of weeks compared by team.

The number of Week by Team.

Um, yep...
All teams played 16 weeks in 2014...
Absolutely true...
Not terribly helpful...
Or relevant.

Of course, the correct answer in this case was, "Yo moron, I don't have any data on that!"  Or even, "Sorry, dude. Not a clue."

Out of curiosity, I dropped the same exact query in Wolfram|Alpha, a computational engine that uses publicly available and scientific data to easily answer natural language queries like, How many teaspoons of water are in Lake Michigan? or What's the average circumference the planets in our solar system? or What's the 205468 decimal of pi?

And you know what?  Wolfram|Alpha couldn't answer my stupid question either, but at least it gave an appropriate response.

This post is not an attack on IBM, Watson, or Artificial Intelligence. I am a huge proponent of AI in the delivery of legal services and, assuming IBM survives, I think Watson will likely be a big player...eventually.  However, given that IBM has posted 11 consecutive quarters of losses, a reasonable person could conclude that WaaS is a premature attempt to monetize the one thing everyone is desperately waiting for IBM to deliver.

My concern is not that firms will start signing up for WaaS in droves to provide Business Intelligence - "You asked for profit margins on work performed for your largest client last year.  Would you rather know the average weeks per month last year?" - but that many people have the wrong idea about what AI is, and what it can do for a law firms. Ironically, IBM may have done a disservice to those of us pushing for the use of AI. They have fostered this idea of an all powerful intelligence that will outperform its human counterparts in whatever field it tackles next.  In reality, the AI that currently exists, Watson included, is best seen as a performance enhancing tool for Biological Intelligence. While that is amazing, and exciting, and truly awe inspiring on its own, it is sadly something less than the magical omniscience that so many are now expecting. We may get there soon enough, but if you are waiting around for that kind of AI, you're going to miss the real AI revolution going on all around you.

Bookmark and Share



© 2014, All Rights Reserved.