Stevens Institute of Technology is part of a four-university team

Press releases and anonymous tips or leaks to reporters are inevitably part spin, part fact. Journalists work hard to decide what’s really a story and what isn’t, and what (if anything) to write about amid the flood of pitches, tips and press releases they receive daily.

Now Stevens School of Business researcher Jeff Nickerson, who studies how AI interacts with journalism and other professions, has co-developed, evaluated and unveiled a new tool that can help reporters find more interesting angles and relevant context in the information they receive.

The tool, known as AngleKindling, was unveiled at an April computing conference in Germany by a four-university team. It’s partly powered by several of the large-language AI models currently receiving wide attention for their realistic responses to queries.

The system was tested with Open AI’s GPT-3 large-language model, says Nickerson, and the team is also working with OpenAI’s ChatGPT and GPT-4 as well as additional language models from Alphabet (a subsidiary of Google). Early evaluations indicate the system could be more useful for journalists than state-of-the-art brainstorming tools are.

“The tools of AI can be turned to positive or negative uses, and brainstorming is one of the truly positive use cases,” explains Nickerson, an expert in human-computer interactions. “While AI doesn’t generate better articles than skilled human writers, it can help journalists with ideation. AngleKindling is a great example of this.”

New tool generates more ideas, more variety

Researchers from Columbia and Syracuse universities collaborated directly with Nickerson in the NSF-funded project to understand how AI can be used to extract relevant concepts from press releases, suggest new lines of inquiry, questions and potential hot-button issues, and provide both historical context and further reading.

The research team began by spending three months meeting with a group of four professional journalists, observing their processes for ideating stories from tips, press releases and other sources. The group also reached out to researchers at Northwestern University working on similar tools.

“We wanted to observe journalists working with AI tools. AI tools are different from other kinds of tools, in that it can take time to figure out what they are capable of and how those capabilities can be applied,” says Nickerson.

Four key needs emerged from those meetings with journalists: the imperative to cut through spin and concisely summarize the pertinent facts included in press releases; the generation of novel story angles; identification of potential controversies and conflicts related to the statements in releases; and a rapid summary of relevant sources such as previous news stories written about similar topics.

Armed with this intel, the research team then created and trained its new large-language model, using OpenAi’s GPT-3 and carefully directed prompts designed to achieve those four goals by (for example) extracting the five most useful keywords from each press release it scanned in the study, then searching archives of The New York Times for the most relevant previous content around those five keywords.

The AI that eventually emerged scans press releases and identifies both the core facts and potential negative outcomes in the information. It also collects working links to high-quality news articles for browsing to get up to speed on a topic. And it suggests questions and interesting new lines of inquiry to a reporter.

All in a few seconds.

A followup study by the team asked 12 more professional reporters — none of them involved in the model’s original design — to use AngleKindling as they worked to create story ideas with a sample set of press releases, comparing its suggestions with the ideas generated by INJECT (a leading idea-generation tool for journalists).

The new system proved both more useful and less time-consuming than the legacy technology as scored by the panel of journalists, the group found, as measured by those survey scores.

And the new AI also generated more varied types of story ideas than INJECT, in the expert opinion of the dozen reporters surveyed.

“We found that AngleKindling was perceived to be significantly more helpful for coming up with ideas, with significantly less mental demand,” wrote the team, summarizing its new model for conference participants, “[by] helping journalists recognize angles they had not considered… providing angles that were useful for multiple types of stories… helping journalists quickly and deeply engage with the press release, and… providing [related content and] context.”

Savvas Petridis, Lydia Chilton and Mark Hansen at Columbia, Nicholas Diakopoulos at Northwestern and Kevin Crowston, Keren Henderson and Stan Jastrzebski at Syracuse collaborated in the research, which was reported in the Association for Computing Machinery’s Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23).