Thoughts on Jeopardy Analysis
Blog 3 of 3 in the Due Diligence Often Discovers Discrepancies series
I expect that there are many folks out there who will challenge our analysis. I’ve anticipated some of the objections and have addressed what I think are the three major ones below.
1. Shouldn't some of the points that we reallocated from Watson to Ken have gone to Brad, lowering Ken's revised total? While that is true, Brad would have also taken additional points from Watson. If we had data from Brad, we expect that the gap between Watson and Ken would be narrower, but that Ken would still enjoy a solid lead.
2. What about Game 1? Watson did even better in Game 1 than it did in Game 2. Wouldn't that have kept Watson the winner? Probably not. The reason Watson racked up such a huge total on Game 1 was that it answered 29 of 32 questions correctly in Double Jeopardy. I didn't have a tape, but I believe Ken and Brad also knew many of those answers and were shut out by the buzzer. Allocating those responses across players would have put one or both players within striking distance when they got to Final Jeopardy. Watson blew Final Jeopardy with a comically bad answer to an easy question. So, what would likely have happened is it would have been in second if not third place heading into Game 2
3. What about the humans' own "unfair advantage". Humans tend to ring in before they know the answer and then have several seconds to figure it out. If they had to answer right away like Watson, wouldn't Watson cream them? While this is true, I take exception to the notion that this represents an advantage for the humans. Instead, this represents a fundamental difference in how computers and humans process information. While it can take humans a few seconds to work out the right answer, we can intuit nearly instantaneously whether or not we will be able answer the question. Great Jeopardy players have great intuition and rarely get questions wrong after they ring in, as Ken Jennings demonstrated by getting just 1 question wrong in Game 2. Watson on the other hand seemed to either come to an answer very quickly or never got there. It doesn't have intuition and more time didn't appear to help it significantly. Changing the rules to take out the intuition factor would shift the advantage to Watson but would be counter the goal of the contest - figuring who is better at answering questions.
Let us hear your objections and observations.
__________________________________________
This article was contributed by Jon Klein. Jon is the founder and general partner of The Topline Strategy Group, a strategy consulting and market research firm specializing in emerging technologies. Jon brings a unique blend of strategy consulting and hands on operating experience to The Topline Strategy Group and works closely with Semaphore on a variety of engagements.