Enhancing Learners’ Reading Skills Through Speech Technology, Say Researchers

The latest Progress in International Reading Literacy Study (PIRLS) report revealed that 81% of Grade 4 learners in South Africa cannot read for meaning in any of our official languages.

“At the end of the foundation phase (Grade 1–3), many learners, especially those who are taught through African languages such as isiZulu and Sesotho sa Leboa as the language of learning and teaching, cannot read for comprehension in their home language. Since early literacy acquisition can be life-changing, we must improve our learners’ literacy skills,” say a group of researchers from Stellenbosch University (SU), the University of Johannesburg (UJ) and the Council for Scientific and Industrial Research (CSIR).

​The Voice Computing Research Group at the CSIR, in collaboration with Laurette Pretorius (SU) and Lionel Posthumus (UJ), started the Ngiyaqonda! (“I understand!”) project to support literacy development and language learning for foundation phase learners. Pretorius and Posthumus, together with Laurette Marais (CSIR) and Ilana Wilken (CSIR), elaborated on the project in an article published recently in the Proceedings of the 5th International Conference on Conversational User Interfaces.

A key aspect of the project is the use of speech technology to help address the challenge of early literacy acquisition.

“Since foundation phase learners have already developed a level of oral competency in their home language long before they can read and write, language and speech technology can be leveraged to support the development of written competency ̶ first in their home language, and eventually also in the language in which they will be taught from Grade 4 onward.”

Emphasising the importance of the project, the researchers add that “when learners enter the intermediate phase in Grade 4, the language of learning and teaching shifts to English, which means that children who were taught to read and write in, for example, isiZulu and Sesotho sa Leboa, must be able to read and write English in almost all the subjects they are taught.”

According to them, the project can help facilitate the shift to English that many learners must make in the intermediate phase (Grade 4–7).

Mobile app

Sponsored by the Department of Sport, Arts and Culture, the Ngiyaqonda! project aims to develop a mobile application that employs grammar-based natural language generation and speech technology to create a multimodal, multilingual learning environment in which Grade 3 learners can practise reading and sentence composition. For the current pilot project, the learners are provided with tablets but eventually any Android device will be supported.

Initially, the application focuses on the learners’ home language, before gradually introducing them to the target language, which would typically be the language of learning and teaching in the intermediate phase, say the researchers.

Containing lesson and task selection screens, the application provides pupils with a progressive learning journey based on their selected home language and target language.

“Initially, the user is only presented with monolingual tasks in their home language. Eventually, the target language (English) is introduced with the requirement that the user exhibits passive knowledge (learner understands what is being said in the target language, but doesn’t have to generate grammatically correct English utterances) of the target language to make progress. Finally, the multilingual tasks require active knowledge (learner produces a grammatically correct utterance in the target language by himself or herself) of the target language.”

Tasks

The researchers point out that there are three kinds of tasks ̶ story tasks, write tasks and game tasks, each presented as a separate activity screen.

They say the story task introduces learners to the story on which the content of the application is based, as well as the drag-and-drop feature to navigate between specific areas on the screen as they construct sentences.

“The write task provides learners with the opportunity to practice reading and sentence composition in a freestyle way, while familiarising them with the growing complexity of utterances. Multiple words are presented in the selection area, from which the user can drag words to the authoring area to compose sentences.”

Using a multilingual story published by the VulaBula reading project, the researchers developed a set of grammars in isiZulu and English (ones in Afrikaans and Sesotho sa Leboa will follow soon) that ensures only grammatically and semantically correct words are presented.

The game task exploits the existing oral competency in the learners’ home language, and developing oral competency in their target language, to gamify reading and sentence composition.

“In these tasks, several random prompts from the grammar are generated and presented to the learners via a listening button. They then listen to the current prompt, and must construct the corresponding sentence by selecting, word for word, the correct token from the selection area. Once the entire prompt sentence has been composed, the sentence is moved to the authored area automatically as audio-enabled text, and the game continues with the next prompt.”

The application is being piloted at schools where isiZulu is the language of learning and teaching for Grades 1 to 3 and English from Grade 4 onward.