University of Alberta: Public health students help community groups improve programs through evaluation

It’s become a truism in business that you can’t manage what you don’t measure. That’s why students in the University of Alberta’s School of Public Health are keen to take a unique course that gives them hands-on experience helping community groups evaluate their programs. And it’s why those community groups are lining up to get the help.

UEval: Evaluation in the Community Context brings together students and community groups for a week-long intensive course to build ready-to-implement evaluation plans to make programs stronger.

“We all evaluate every day when we look at the weather out the window, make decisions about what to wear, whether to walk to work or take public transit because it might rain,” said Rebecca Gokiert, professor and member of the leadership team of the Community-University Partnership for the Study of Children, Youth, and Families.

“When we evaluate community programs, we want to know whether they are making a difference. Are they providing the right activities, reaching the right participants, offering services at the right times, so the desired impact is actually happening?”

Measuring effectiveness, conveying results
Funke Smith, founder and CEO of Skillcity Institute, which serves 600 youth from racialized and marginalized communities each year, came away from the June 2021 course with an evaluation plan for the STEM Heroes Mentoring Program, which connects young people with professionals working in science, technology, engineering and mathematics careers.

“I can now go to funders more confidently because I know how to gauge the effectiveness of our program and how to convey the results to different stakeholders,” said Smith.

I can now go to funders more confidently because I know how to gauge the effectiveness of our program and how to convey the results to different stakeholders.

Master’s student Kristen Deschamps, who is specializing in health promotion, helped Smith build the plan. Deschamps hopes to use her new evaluation skills in a future career in workplace health.

“Now I know what questions to ask, where to start, what to look for—all important evaluation skills that are transferable between settings,” said Deschamps.

Gokiert, who is also director of the Canada-wide Evaluation Capacity Network and a member of the Women and Children’s Health Research Institute, launched the course in 2019 thanks to a Teaching and Learning Enhancement Fund grant. She recently shared results in a special edition of the Canadian Journal of Program Evaluation. Nineteen community groups and more than a hundred graduate, undergraduate, professional and continuing education students have participated to date.

“Evaluation takes technical skills, interpersonal skills, managerial skills and situational skills,” she said. “In this course, the students get to practise the competencies that they learn in the morning alongside a community-based organization in the afternoon.”

Evaluation tools range from questionnaires to interviews and even art projects, which systematically gather quantitative and qualitative feedback from program participants and stakeholders.

“The goal is to gather really good evidence about impacts, to understand how to better meet the needs of those you are trying to reach, and use it to make shifts to the organization’s practices, programs and policies,” explained Gokiert.

“Data is king”
Thorough program evaluation is important for groups like Skillcity to prove the merit of their work, showcase successes and pitch for potential growth, Deschamps said.

“They need to demonstrate to people who aren’t immediately involved that it is a fantastic program that is offering long-term support, positive outcomes and changes to the youths’ lives that wouldn’t have been there without it,” Deschamps said.

“We help youth find their path,” said Smith. “The bedrock of that is to build a relationship of trust between them and their mentor, sharing stories and creating a safe space with the young people to feel empowered and that they matter.”

Smith said the data gathered through the new evaluation plan will help Skillcity attract funders, sponsors, mentors and community partners.

“For any organization that is going somewhere, they should consider including evaluation in their whole process,” Smith said. “They say data is king—if you don’t have data to confirm your impact, then you have nothing.”