AI tennis commentary is coming to the US Open

Can AI mimic the expressive yet lulling real-time monologues of a sportscaster? Tennis fans are about to find out.

Those tuning in to highlight videos of the US Open tournament may encounter a prolific new color commentator for the first time this year: IBM AI platform watsonx.

The tech company, which has long supplied services to the annual event, is expanding its partnership with the US Tennis Association (USTA) to include “audio commentary and text captions on video highlight reels of every men’s and women’s singles match,” it noted in a blog post. The videos will live on the US Open’s website as well as within the tournament’s official app.

IBM’s goal is not to phase out the well-known announcers who call the primetime action on TV, but to expand the number of matches and languages the league is able to cover with content, according to Monica Ellingson, sports and entertainment practice lead for IBM Consulting.

“We’re not here to replace all of the iconic commentators. Of course not,” Ellingson said last week on a press tour of IBM’s operations at the Billie Jean King National Tennis Center in Queens, New York, which houses 17 courts. “But there are only so many of them. So it’s really just a matter of making the action here at the US Open accessible.”

IBM first rolled out AI-generated spoken commentary for golf’s Masters Tournament earlier this year, tapping into its new suite of enterprise-focused foundational models. In this case, IBM fine-tuned the models on tennis terminology and the effusive language of sports commentary—training data included adjectives like “brilliant,” “dominant,” and “impressive,” for example. The dataset also included content mined from sports blogs and articles.

The model is generally trained to put a positive spin on the players’ performances, according to Tyler Sidell, technical program director for sports and entertainment partnerships at IBM.

“We do need to be in a positive light, but there is an expressive voice. There’s also emotion that’s added into it,” Sidell said. “The models do drift. But we always have humans in the loop, too.”

When asked if IBM has thought about how the program might eventually be expanded to include voices personalized and trained to sound like certain tennis stars or other celebrities, Ellingson responded, “We’re definitely looking at that. Because certainly celebrity voices would be very, very attractive. And then we’re also looking into what are the unique branded voices that we might want to bring to the table that really represent the US Open or Wimbledon or our other partners.”

Tennis star Maria Sharapova, who also attended the press tour, seemed open to the idea of being able to add commentary “from my couch” and said IBM had explored the idea of using her voice for Wimbledon this year.

IBM has been the technology partner of the USTA for three decades, and in recent years, the company has supplied other forms of AI to give fans more access to stats and data on the matchups. In 2021, for instance, IBM rolled out a power rankings feature that uses natural language processing to create leaderboards for players.

Now, a new wave of generative AI experimentation has hit almost every business sector this year—IBM’s enterprise arm included. It was only a matter of time, then, before sports leagues and media companies began to experiment with how AI can fill the role of sports analyst.

When IBM deployed the AI commentary at the Masters Tournament and Wimbledon this year, reports of mispronunciations and odd intonations abounded.

But Ellingson said the team was surprised that Masters fans tended to engage with it in the background while doing other things.

“We were really unsure how it would be taken and how people would use it,” Ellingson said. “But I think the story is that we were surprised that people really used it a lot like when they were at work.”

Ellingson said she imagines eventually using generative AI and computer vision to create a real-time broadcast of matches on all 17 courts in the Billie Jean King National Tennis Center.

“The vision that I’ve always had is that we have 17 courts here…if we were able to actually create a radio program—not at the same level as Jim Nantz or some of our other famous commentators out there for golf and tennis, but to be able to just follow along and understand what’s happening without looking at it,” she said.