Computer Quiz Bowl System Ties with Team of Top-Ranked Humans

Competitions between man and machine always seem to hold more than purely objective significance. Whether as a result of alarming science-fiction conjectures about robots overthrowing humanity or of a more reactionary desire to assert humanity's superiority to its creations, people often tend to assign a great deal of societal importance to innovations in automation—IBM's Watson system, which recently won against two Jeopardy champions in 2011, stands out as an example of this behavior.

QANTA, a question-answering system based on deep learning principles, was developed by a team of researchers from the University of Maryland and University of Colorado Boulder led by Jordan Boyd-Graber of Colorado, a former Postdoc at the University of Maryland and a former Assistant Professor in UMIACS and the iSchool. The team included Associate Professor Hal Daumé III and PhD students Mohit Iyyer, Varun Manjunatha, Leonardo Claudino, and He He from the University of Maryland. At the 2015 High School National Championship for Quiz Bowl in Chicago, Illinois last Saturday, May 30, QANTA competed in an exhibition match (a video of which is available here c/o Jordan Boyd-Graber) of tossup questions against a team of four top-ranked human contestants. The match ended in a tie, with a final score of 200-200.

While contestants in Jeopardy must wait for the host to finish reading a clue in order to “buzz in” with an answer, players in quiz bowl can interrupt the question at any time. The questions are generally worded such that “harder” clues are located near the start of the question and “easier” clues are located near the end—in essence, rewarding players who know more about the answer. Per its project overview, QANTA therefore “...must learn to balance incorrect translation versus timely translations, and must use those predictions to translate the sentence.” If the system believes it has enough data to know the answer to the question even without knowing the full question itself, it will interrupt with an answer. The challenge lies, then, in simultaneously guessing the question's composition and applying known data to answer it.

QANTA's applications to problems like quiz bowl can hold significant ramifications for other areas of natural language processing, perhaps most evidently synchronous machine translation (e.g. guessing at an incomplete sentence in languages like German, in which verbs occur at the end of sentences). Though quiz bowl may seem an isolated problem scope, solutions to its questions can be very applicable to other problems in Computer Science.

The successes of QANTA, Watson, and other computational solutions to traditionally “human” dilemmas may seem disconcerting to many, but they also offer valuable perspective. They serve as a harbinger for progress in the field, and stand to revolutionize computational language processing and decision-making.