IT WAS not quite a foregone conclusion, but all the smart money was on the machine.
Since the first rehearsal over a year ago, it had become apparent that Watson—a supercomputer built by IBM to decode tricky questions posed in English and answer them correctly within seconds—would trounce the smartest of human challengers. And so it did earlier this week, following a three-day contest against the two most successful human champions of all time on “Jeopardy!”, a popular quiz game aired on American television. By the end of the contest, Watson had accumulated over $77,000 in winnings, compared with $24,000 and $21,600 for the two human champions. IBM donated the $1m in special prize money to charity, while the two human contestants gave half their runner-up awards away.
IBM has a long tradition of setting “grand challenges” for itself—as a way of driving internal research and innovation as well as demonstrating its technical smarts to the outside world. A previous challenge was the chess match staged in 1997 between IBM’s Deep Blue supercomputer and the then world champion, Garry Kasparov. As shocking as it seemed at the time, a computer capable of beating the best chess-player in the world proved only that the machine had enough computational horsepower to perform the rapid logical analysis needed to cope with the combinatorial explosion of moves and counter-moves. In no way did it demonstrate that Deep Blue was doing something even vaguely intelligent.
Even so, defeating a grandmaster at chess was child’s play compared with challenging a quiz show famous for offering clues laden with ambiguity, irony, wit and double meaning as well as riddles and puns—things that humans find tricky enough to fathom, let alone answer. Getting a mere number-crunchier to do so had long been thought impossible. The ability to parse the nested structure of language to extract context and meaning, and then use such concepts to create other linguistic structures, is what human intelligence is supposed to be all about.
Four years in the making, Watson is the brainchild of David Ferrucci, head of the DeepQA project at IBM’s research centre in Yorktown Heights, New York. Dr Ferrucci and his team have been using search, semantics and natural-language processing technologies to improve the way computers handle questions and answers in plain English. That is easier said than done. In parsing a question, a computer has to decide what is the verb, the subject, the object, the preposition as well as the object of the preposition. It must disambiguate words with multiple meanings, by taking into account any context it can recognise. When people talk among themselves, they bring so much contextual awareness to the conversation that answers become obvious. “The computer struggles with that,” says Dr Ferrucci.
Another problem for the computer is copying the facility the human brain has to use experience-based short-cuts (heuristics) to perform tasks. Computers have to do this using lengthy step-by-step procedures (algorithms). According to Dr Ferrucci, it would take two hours for one of the fastest processors to answer a simple natural-language question. To stand any chance of winning, contestants on “Jeopardy!” have to hit the buzzer with a correct answer within three seconds. For that reason, Watson was endowed with no fewer than 2,880 Power750 chips spread over 90 servers. Flat out, the machine can perform 80 trillion calculations a second. For comparison’s sake, a modern PC can manage around 100 billion calculations a second.
For the contest, Watson had to rely entirely on its own resources. That meant no searching the internet for answers or asking humans for help. Instead, it used more than 100 different algorithms to parse the natural-language questions and interrogate the 15 trillion bytes of trivia stored in its memory banks—equivalent to 200m pages of text. In most cases, Watson could dredge up answers quicker than either of its two human rivals. When it was not sure of the answer, the computer simply shut up rather than risk losing the bet. That way, it avoided impulsive behaviour that cost its opponents points.
Your correspondent finds it rather encouraging that a machine has beaten the best in the business. After all, getting a computer to converse with humans in their own language has been an elusive goal of artificial intelligence for decades. Making it happen says more about human achievement than anything spooky about machine dominance. And should a machine manage the feat without the human participants in the conversation realising they are not talking to another person, then the machine would pass the famous test for artificial intelligence devised in 1950 by Alan Turing, a British mathematician famous for cracking the Enigma and Lorenz ciphers during the second world war.
- Watson The Confusion over an Airport Clue (asmarterplanet.com)
- Watson plays ‘Jeopardy!’ well, but what else can it do? (usatoday.com)
- Are Mathematicians In Jeopardy? (rjlipton.wordpress.com)