Cornell University Redesigns Videoconferencing for People Who Stutter

As a former tech lead at Meta, Shaomei Wu, Ph.D. ’12, found she had to work extra hard to get her points across in large online meetings. She stutters, which combined with her gender and racial identity, made it challenging to claim and hold the floor. She began doing research with the stuttering community to find out how fellow stutterers felt about dominant communication technologies and the new barriers they introduced.

That experience led her to found AImpower.org, a nonprofit that seeks to co-create more empowering technologies with and for individuals from marginalized communities.

For her newest project, she partnered with Gilly Leshed, Ph.D. ’09, senior lecturer in the Department of Information Science in the Cornell Ann S. Bowers College of Computing and Information Science, and Jingjin Li, Ph.D. ’23, a research fellow at AImpower.org. They worked with people who stutter to find ways to make these platforms less stressful for individuals with speech diversities, while improving the experience for everyone.

“If you look at technology designed for those who stutter, most of them are trying to help you conceal stuttering or make you sound fluent,” Wu said. Instead, she hopes to normalize stuttering and help everyone to be better conversation partners in video calls. “We are not trying to fix you,” Wu said. “Stuttering is just a way of talking and we want to support you to have a stronger presence in meetings and in the workplace.”

Leshed presented “Re-envisioning Remote Meetings: Co-designing Inclusive and Empowering Videoconferencing with People Who Stutter,” on July 4 at the Association for Computing Machinery (ACM) Conference on Designing Interactive Systems (DIS ’24) in Copenhagen, Denmark. The work received an honorable mention for the best paper award.

People who stutter – meaning they sometimes repeat or elongate parts of words, or temporarily can’t speak due to a block – frequently experience discrimination and carry negative feelings as a result of their stuttering.

It takes extra effort for people who stutter to communicate, and this burden is amplified by the design of videoconferencing platforms that prioritize verbal communication. In video calls, people who stutter have a much harder time using nonverbal communication strategies, like eye contact or leaning forward when they wish to speak. They may feel more anxious watching themselves on screen and struggle to enter the conversation. Additionally, they are frequently interrupted, and if they are in a block, others may assume their connection is frozen and talk over them.

In the new work, Wu collaborated with Leshed and Li to conduct co-design research – an approach in which people with disabilities actively offer ideas and guidance during the design process.

“We see the stuttering group not only as people who we should design for, but also as designers who deal with communication challenges every day,” said Li, who led the process.

Li held a series of individual and group videoconferencing sessions with eight people who stutter. They generated suggestions and collectively reflected on each other’s ideas for ways to make the technology more inclusive and empowering.

One popular suggestion was a badge that self-identifies an individual as a person who stutters. Participants thought the badge would reduce stress, allow people to be more authentic in their communications, show vulnerability to increase human connection and help other participants to better understand their needs.

Participants also liked the idea of an “I’m still speaking” button that could be pressed when a person is in a block or struggling with a word, to prevent others from taking over their turn. They noted that the button could also help non-stutterers reclaim the floor after being interrupted.

Some participants proposed more radical ideas, like a “cooling off period” – essentially, muting people who frequently interrupt and dominate the conversation. Others proposed adding audio technology that would subtly smooth out stuttered speech to improve the conversation flow, similar to how users can touch up their appearance on Zoom. This could be useful in situations where people who stutter are often discriminated against, such as job interviews.

Another participant argued that stuttered speech shouldn’t be concealed, and that it should be accurately reflected in automated meeting transcripts, which often smooth out the text. Additionally, participants proposed that platforms provide educational materials about stuttering and link to recordings of people with different speech patterns.

“Instead of trying to homogenize speech, actually celebrating the diversity of different kinds of speech is a way to better connect people,” Leshed said.

Using some of these suggestions, AImpower.org is about to release the beta version of a videoconferencing companion app. So far, almost 80% of the code was written by engineers and interns at the nonprofit who stutter, Wu said. She hopes that by involving people who stutter, “the community can develop real ownership and agency of the technical innovations that benefit them, while empowering more authentic and inclusive interactions for everyone.

“We can reinvent video calls to tap into the kinds of connection we all seek and are missing in those calls,” Wu said, “and to make this technology much better for all.”