A common aphorism among artificial intelligence practitioners is that A.I. is whatever machines can’t currently do.
Adam Gopnik, writing for the New Yorker, has a review called Get Smart in the most recent issue (4 April 2011). Ostensibly, the piece is a review of new books, one by Joshua Foer, Moonwalking with Einstein: The Art and Science of Remembering Everything, and one by Stephen Baker Final Jeopardy: Man vs. Machine and the Quest to Know Everything (which would explain Baker’s spate of Jeopardy!-related blog posts). But like many such pieces in highbrow magazines, the book reviews are just a cover for staking out a philosophical position. Gopnik does a typically New Yorker job in explaining the title of this blog post.
Gopnik describes his mother as “a logician, linguist, and early Fortran speaker” and goes on to add that she worked on an early machine translation project in Canada. I’m guessing she’s the Myrna Gopnik behind this 1968 COLING paper (LE PROJET DE TRADUCTION AUTOMATIQUE A L’UNIVERSITE DE MONTREAL).
As a child, Gopnik’s mom explained to him why a tic-tac-toe-playing computer is not very smart. I also had a run-in with an early tic-tac-toe computer at a science museum that left a deep impression on me. I wanted to build one in 6th grade, but only got as far as writing the game tree out on two sheets of posterboard, being stumped at how to code the game tree on my Digi-Comp, the instruction manuals for which described game trees clearly enough a 10-year old could understand them.
Coincidentally, John Prager just gave a very insightful talk today on the natural language processing (as well as strategy and hardware) behind IBM’s Watson system for playing the televised quiz show Jeopardy!.
I was hugely impressed with Watson. It answered almost every question correctly and grammatically. As Prager pointed out, some of the language is hard. Some of the categories (like “Rhyme Time”) had special components built for them, but most were totally general.
Given the buzz-in versus precision graphs Prager showed, I’m guessing Watson might not have won without the buzzer advantage. That’s because the humans Watson was playing had a history of getting almost every question right. Empirically, the champions answered roughly 70% of the questions they faced and answered 90-100% of them correctly. And that’s with competition to the buzzer from the other players. Watson’s ROC-type curve showed them in the winner’s territory without competition to the buzzer. My former CMU colleague Peter Spirtes was a Jeopardy! winner and he said it was very much about the buzzer because everyone was so good.
There would be two ways to make the questions harder. By making the underlying data more obscure, it’d probably favor Watson. By making the wordplay more obscure, I’m still voting on the human contestants.
Coming back to the over-arching subject of this blog, Prager emphasized the classification component of the system that used hundreds of features of a question, the category and a possible answer to estimate the probability that the answer is correct. Exactly that same opinion is what led me into Bayesian stats, where characterizing uncertainty is the name of the game.