New Research Challenges the Accuracy and Validity of Student Satisfaction Ratings
A single personality trait alone – innate happiness – accounts for 24% of student-satisfaction scores in higher education, according to a new international study in the peer-reviewed journal Assessment & Evaluation in Higher Education.
The study by a team from the University of Reading, Hawaii Pacific University, and the University of Bath suggests over half of student satisfaction is attributable to unalterable individual-level personality traits, such as neuroticism, extraversion and others, rather than to actual quality of received education.
Its findings, based on data from 409 students studying at 63 universities across 20 countries, cast doubt on the credibility and usefulness to students, their parents, universities and governments of simple student satisfaction scores that fail to control systematically for trait happiness and other inherited personality characteristics.
“Two deeply flawed assumptions underlie student satisfaction assessment at all levels,” said Dr Florence Phua of Reading University, who led the research.
“It’s assumed, first, that students’ reported satisfaction directly reflects the quality of the education they get and, second, that their satisfaction can be readily increased by changing aspects of that education, such as, for example, the extent, nature and speed of assignment feedback – something students’ unions keenly promote.
“But these two erroneous assumptions directly contradict extensive satisfaction research in job, consumption and other domains that consistently finds levels of satisfaction with most things largely reflects inherited and unalterable personality traits, especially innate happiness. Consequently, satisfaction levels are found substantially unsusceptible to alteration by any objective change in, say, a job or consumed product,” Dr Phua said.
“We tested if genetically-determined innate happiness explains student satisfaction levels and found that it does, about a quarter. Twin studies of genetically-determined personality more broadly suggest inborn character traits combined likely account for over 50% of reported student satisfaction,” continued Dr Phua.
Co-researcher Dr Gerard Dericks from Hawaii Pacific University cautioned: “A worrying implication of our study is that universities keen to game student satisfaction rankings might be tempted to use trait happiness as an admission criterion.
“Accepting only innately happy students while rejecting the miserably unacceptable might be unethical, but unscrupulous university heads determined to manipulate satisfaction rankings could see it as a quick and easy alternative to the much harder pedagogical professionalism needed to ensure genuinely excellent education.”
Professor Edmund Thompson, a co-researcher from University of Bath, noted: “Student selection by trait happiness would likely be counterproductive to the core purpose of universities: the production and dissemination of useful knowledge.
“Had universities historically selected students based on trait happiness, Cambridge might not have admitted notorious curmudgeon Isaac Newton, and infamous melancholic Søren Kierkegaard could have been rejected by Copenhagen University.
“Even cursory analyses of data from respected university rankings like Complete University Guide and Guardian University Guide reveal that student satisfaction has no correlation whatsoever with either graduate prospects or degree value-added, both logical proxies for worthwhile education,” continued Professor Thompson.
“These data also show that scores on learning feedback – a favourite metric of student unions – is strongly, but in fact negatively correlated with both continuation rates and post-graduation career success, each objectively more accurate indicators of higher education effectiveness and value-for-money than personality-driven student satisfaction scores.”
The study’s overall conclusions raise serious questions about the accuracy and value of simplistic student satisfaction assessment.
Dr Phua concluded: “That so much assessed student satisfaction stems not from extrinsic educational experience but from intrinsic personality traits like innate happiness that cannot be altered by mighty education ministers or even august university administrators – never mind paltry professors and lowly lecturers – begs a question that governments, universities and scholars alike must answer for fee-paying students, their parents and tax-payers alike.
“Why, and to what quantifiably beneficial individual student or wider societal avail, is so much attention, time, effort and expense lavished on student satisfaction assessment when it demonstrably captures immutable and educationally irrelevant personality traits, while much less evidently reflecting the true student or societal value of professional, rigorous, worthwhile and objectively excellent higher education?”