University of Florida: UF researchers seek out if AI tutors are as effective as human ones

Artificial Intelligence is now being used to help tutor students at all educational levels. AI tutors can enhance learning outcomes such as facilitating various teaching-learning practices within and outside the classroom, supporting students 24/7, and providing students from economically disadvantaged areas access to all kinds of learning materials without being in the same geographical location.

Yet, despite the now somewhat ubiquitous nature of AI tutoring, many students, teachers, and parents are skeptical of their efficacy. The belief is that human tutors are more effective, in particular, at helping students with language learning, such as creating writing and language conversation.

University of Florida College of Journalism and Communications Associate Professor Yu-Hao Lee and doctoral students Mo Chen and Fanjue Liu, wanted to better understand whether or not AI tutors are as effective as human tutors, specifically in language and writing education.

Their study examined high versus low student involvement learning, and compared AI versus human tutor types and how that affected students’ perception of tutor writing skills.

Findings show that when learning involvement is low, the AI tutor was perceived to have a higher writing quality than a human tutor, and when involvement was high, tutor type did not matter. The study also found that human tutors are preferred overall, with a higher level of credibility than the AI tutor.

Study participants perceived the AI tutor as having less control over writing quality than the human tutor, leading to more credibility for and willingness to work with the human tutor in the future. They also found that people are more likely to blame humans than AI because humans have higher control over their service, leading students to feel that humans are more responsible for their level of service.

This research serves as a foundation for instructors and product designers to better understand how students perceive both kinds of tutors and design future tutoring services. Perceived control is a key factor in student trust of their tutors; emphasizing that the tutoring service’s overall quality is assessed by humans could enhance perceived credibility.

Limitations include the length of assessment, and limited age range of participants. Their perceptions may shift if they have more direct and ongoing experiences with the writing tutor in a more realistic setting. A longer study would also be helpful to see if participants’ perceptions change over time. Surveying a greater age range of participants would be useful to understand if students of different age groups would be more open to AI tutors versus human tutors.