Arguing With AI

IBM’s artificial intelligence technology has foiled chess masters and Jeopardy! champions, but it hasn’t won a debating competition against a human -- yet.

Project Debater, a two-meter-tall black box powered by artificial intelligence, lost to champion debater Harish Natarajan last month in the first machine-versus-man debate competition of its kind.

It was a loss but not a failure for Project Debater. She (yes, she) was able to construct arguments and make rebuttals and even used preprogrammed humor.

“I have heard you hold the world record in debate competition wins against humans, but I suspect you have never debated a machine,” Project Debater said to Natarajan in her opening remarks. “Welcome to the future.”

IBM wants to use this technology to help people develop more persuasive arguments and make well-informed decisions, said Dan Lehav, computer scientist and a debate expert at the company. Humans are good at presenting information in ways that appeal to other humans, he said. But machines have the ability to analyze unfathomable amounts of data.

“We want to bridge those worlds,” said Lehav. “Humans are going to be able to use this in a complementary way.”

He said the technology has several potential applications in higher education. College debate is an obvious area. Project Debater could be a practice opponent, a research aid or even a judge of the strength of students' arguments -- including those written in essay form.

What makes a “strong” argument is something that IBM scientists have grappled with, said Lehav. Project Debater determines whether there is substantial evidence for or against an argument by analyzing a corpus of more than 300 million newspaper, magazine and journal articles.

AI can “shine a light on distorted facts to provide diverse, well-informed viewpoints -- both the pro and the con,” IBM said on its Project Debater website. “The rise of one-sided and doctored narratives is challenging society and our platforms. Too often, we talk past one another. We need a smarter way.”

Brandon Fleming, founder and CEO of the Harvard Debate Council Diversity Project, said he is excited about the potential applications of the technology in academe. “The technology is simply revolutionary,” he said in an email. “Project Debater has the potential to empower our ability to think critically and communicate effectively.”

Fleming said he wants to use the technology to train college debaters in rhetoric, research and argumentation. “The speed at which Project Debater can generate arguments would provide more content for my students to explore and critique,” he said. This would “inherently accelerate and enhance the amount of academic conditioning that one training session could allot -- producing stronger students in a shorter period of time.”

More generally, he said the technology could be used in classrooms to encourage students to engage in critical examinations of research and argumentation. “In addition to information, Project Debater provides instant access to reason. Information and reason coalesce into ideas and arguments. Argumentation is the vehicle by which ideas are exchanged, challenged and perfected.”

However, not everyone is enthusiastic about the potential of AI in college debate. Jeffrey Jarman, director of the Elliott School of Communication and former director of debate at Wichita State University, said he could see a role for AI in helping students improve their debating skills, but with "lots of caveats and cautions."

"I"m not sure how much the AI understands the full process of argumentation," said Jarman in an email. Debating is about more than just finding strong arguments, he said. "We have to shape and alter our arguments based on the particular audience we are speaking to. It isn't clear that the AI is sophisticated enough to do that yet."

Scott Varda, associate professor of communication and associate director of debate at Baylor University, said the IBM technology was “impressive” but had limitations. In the live debate, Varda felt Project Debater struggled to grasp nuance and lacked creativity. “Debate is both an art and a science,” he said.

While Project Debater could be used to train champion debaters, Varda doesn’t think this would be a scalable or particularly useful application of the technology. A better use might be tapping AI’s analytical power to help academics mine scholarly literature, he said, perhaps to identify new drug targets in medical research.

Sarah Partlow-Lefèvre, a professor of communications and director of debate at Idaho State University, agreed that Project Debater’s debating skills were limited. She likened the AI’s performance in the live debate to that of a novice debater. “It was able to compile all the information but didn’t know how to use it."

The AI started with an advantage over its human opponent because it had access to a treasure trove of raw data and facts, said Partlow-Lefèvre. But it fell down on connecting with the audience through humor, eye contact, delivery and subtlety of argument. “It didn’t feel able to maintain a consistent policy position and didn’t quite understand the nuance of all the arguments,” she said.

Training regimes for college debaters are usually low-tech affairs, said Partlow-Lefèvre. Pen drills, where students speak while biting down on a pen, are used to help students improve their enunciation. Speed reading helps students to process information faster. Students also are encouraged to film themselves and review the video to see where they can improve their delivery.

AI could be a useful sparring partner for college debaters in training, or help them to conduct research, said Partlow-Lefèvre. But she predicts that use of this technology in a competition setting would prove contentious. In competitions where students are given just minutes to prepare their arguments, even allowing access to the internet is unusual and deemed “very controversial.”

Partlow-Lefèvre worries that access to the technology might only be available to students at the wealthiest institutions -- possibly giving them an unfair advantage over their opponents. She also questions whether the use of AI would be deemed intellectually honest by college debate judges.

The world of college debate "is slow to change," she said.