Tsinghua and Sogou release World’s First Smart Keyboard for the Visually Impaired
The Department of Computer Science and Technology of Tsinghua University and Sogou Inc. (NYSE: SOGO) released the world’s first smart keyboard for visually impaired users on May 21, the ninth annual Global Accessibility Awareness Day (GAAD), which focuses on raising awareness of digital access and inclusion for people with disabilities and impairments. The smart keyboard, VIPBoard, aims at improving the underlying keyboard algorithm without altering current input interaction.
Entering text on their smartphone remains hugely challenging for visually impaired users. While modern touchscreen keyboards are all powered by the word-level auto-correction ability to handle input errors, visually impaired users cannot enjoy this benefit. They rely on a screen-reader keyboard on their smartphones, which only offers character-level input that has no predictive text correction ability. When users are texting on their smartphones, they often miss the targeted key when touching without visual feedback.
To tackle this problem, Professor Shi Yuanchun and Associate Researcher Yu Chun of the Department of Computer Science and Technology of Tsinghua University have led their research team to successfully design a character-level auto-correction method, achieving a 63% reduction in the character error rate (CER). This means that during typing, the algorithm can help reduce the missed clicks from an average of 3 to 1, for example.
On each tap, the VIPBoard predicts the probability of each key considering both users’ keyboard touch location and language model, and reads the most likely key, which saves calibration time when the user misses the target key. With a one-millisecond response time, VIPBoard allows users to correct a variety of text input errors and supports multi-language input.
The research team published their paper in the 2019 ACM CHI Conference “On Human Factors in Computing Systems” held in Glasgow, United Kingdom, the top academic conference in the field of human-computer interaction. The paper won Honorable Mention Award at the conference.
The VIPBoard, a character-level auto-correction method for the visually impaired.
In partnership with Sogou Inc., Tsinghua University has established the Tiangong Institute for Intelligent Computing in 2016, which includes a joint laboratory that provides faculty members and students in Tsinghua with the opportunity to utilize their theoretical research to adapt to real-world business challenges.
After the research paper was published, Tsinghua and Sogou Inc. started to collaborate immediately on product development. After a series of processes of product evaluation, joint development, and testing, the lab’s research result turned into an actual product within a year.
VIPBoard is one of the many products that are built on the lab’s research results. “Some of these research results are already put into mobile applications and are serving millions of users”, Associate Researcher Yu Chun said. “We are also working on developing other technologies that are aimed at improving digital access for people with disabilities and impairments, and we expect to have another research result published later this year.”