ChatGPT Offers Opportunities for Revitalizing Traditional Assessment Methods

In a recent article for the Daily Maverick, Prof Michael le Cordeur, Vice-Dean: Teaching and Learning in the Faculty of Education, wrote that artificial intelligence programmes like ChatGPT hold many challenges, but they also offer opportunities to renew the traditional methods of assessment.

Michael le Cordeur

Much has been written and said about artificial intelligence (AI) and ChatGPT (generative pre-trained transformers) in the search for a suitable approach to this new technology. To express an opinion is risky; ChatGPT renews itself continually. I write in a language that everyone can understand and from the perspective of higher education, but it is also applicable to basic education.

What is ChatGPT?

ChatGPT is a user-friendly AI chatbot. It can read, write essays and poems, solve maths problems and even point out errors in calculations. To test it, my Afrikaans class and I asked ChatGPT to write a poem about load shedding. On the screen, we could see the Chatbot write the following:

Beurtkrag

Kragonderbrekings, ons land se pyn

Gedwonge donkerheid, dis nie fyn

Die kerse brand, die batterye dood

Ons wag vir krag, ons wag vir nood

 

Daar’s geen krag, geen lig, geen hoop

Die lug is stil, ons roep en roep

Maar niks gebeur, die donkerheid bly

En ons sit hier, en wag vir die dag wat sal bly

 

Ons bid vir krag, ons vra vir hulp

Ons vra vir antwoorde, ons vra vir skulp

Maar niks gebeur, die nag is lank

Ons staan alleen, in donkerheid gevang

 

Kragonderbrekings, ons vloek en kla

Ons wonder net, wanneer dit ophou dra

Die krag is weg, ons land staan stil

Ons wonder net, hoe lank dit nog wil

For the sake of this article, I asked ChatGPT to translate the poem for me into English (for my English readers) and this is what the chatbot came up with:

Loadshedding

Power outages, our country’s pain

Forced darkness, it’s not fine

The candles are burning, the batteries dead

We wait for strength, this is insane

 

There’s no power, no light, no hope

The sky is dark, we moan and moan

But nothing happens, the darkness remains

And we sit here, waiting for the night to end

 

We ask for help, for strength we pray

We ask for answers, we ask for shelter

But nothing happens, the night is here to stay

We stand alone, caught in darkness

 

Power outages, we curse and complain

The power is gone, our country stands still

And we wonder, when this nightmare will

Stop – as we pray for the night to end.

In response to the efforts of the chatbot, a fascinating class discussion followed, especially about the challenges that the chatbot holds for teachers, learners and assessments.

On the face of it, the poem looks like a good attempt by a talented learner. On close inspection, the poem however shows a number of language and poetic errors. There are examples of forced rhyme, bothersome repetition and strange Afrikaans words and phrases like “donkerheid” instead of “donker” or “duisternis”. This raises the suspicion that this may be someone else’s work.

ChatGPT cannot think or argue (at least not yet); it receives information from data supplied by its human creator. The only information we supplied was that load shedding causes pain and keeps us in the dark. Now take note how it repeats “pain”, “night” and “dark”.

Incorrect data leads to incorrect responses which are incorrectly accepted as facts. The chatbot often responds with incorrect statements presented as fact. The chatbot only supplies answers according to the assignment received. The first lesson for students is that ChatGPT often supplies incorrect information as fact. As it does not store its responses, one cannot trace the sources with search instruments.

Nothing new

Artificial intelligence is nothing new; it has been part of our lives for a long time. For learners a school day without a calculator is unthinkable. You pay accounts with an app on your smartphone instead of a cheque. We receive an SMS from the bank after hours when there is no one there. With the help of AI you can do online shopping, find your way to an unknown destination and correct your spelling errors in an essay.

Similar AI technologies form the basis of various educational tools which are already in use, including SUNLearn (the platform on which my university’s module frameworks are loaded) and Turnitin (which measures the percentage of quotations from other sources).

What to do about assessment

AI and ChatGPT hold many challenges but also exciting opportunities for Assessment of Learning and Teaching (TLA). The impact of assessment will be groundbreaking, especially non-invigilated assessment such as assignments done at home and critical feedback about texts. At first, the (predictable) reaction to ChatGPT was (1) to return to all assessment taking place with invigilation; and (2) the development of new, better software which can trace ChatGPT. This reaction would however only address the symptoms.

At the speed at which AI develops, option 1 is not attainable. Although option 2 is technically possible, it will lead to a continuous battle to keep up with AI, a battle we will never win. What is needed is that human behaviour should change.

• Other challenges for teaching and learning are:
• Students will become too dependent on chatbots, preventing critical thinking;
• Chatbots can supply answers to questions, but students will not always understand the thinking behind the answers; and
• Chatbots can store students’ personal information in their memory, which can lead to violation of privacy if it is not properly managed.

What can we do?

In order to protect the academic integrity of students and lecturers, the most responsible thing to do is to adapt TLA practices to AI. I share a few proposals supplied by various academics and thought leaders:

• We have the ability to do research, interpret facts and to reason about it critically and to apply this knowledge in our TLA practices;
• Emphasise academic integrity and ensure that students know where to get guidance and support about healthy research practices;
• Communicate: Discuss AI tools with students and ensure that they understand what the correct and incorrect usage entails;​​​
• Utilise contact time for active learning (instead of soul-destroying and boring lectures) and get students involved in the class;
• Adapt the learning outcomes by aligning the modules with the critical skills above so that students are equipped with these skills;
• Renew assessment methods: assess less; everything does not have to be scored. Use less high-risk assessment and focus more on process-driven assessment;
• Avoid tasks which tempt students to use ChatGPT by focusing on creative alternatives such as oral assessment, presentations, class and individual discussions, critical reading and writing editorials; and;
• Use the flipped classroom method by moving ​classes outdoors.

Creative and innovative

I would like to share a creative alternative which I have tested with my class. Instead of the normal lecture which discusses one poem, the classroom is moved outdoors. Three groups could each draw one of the poems below (to avoid selection of known poems): Vroegherfs (NP van Wyk Louw), Delft (Nathan Trantraal) and Oppie parara (Adam Small).

Groups had to make a cellphone video to illustrate the poem and assign tasks to team members, such as photographer, copywriter, lighting and sound. The video about Vroegherfs was filmed on an autumn day in Stellenbosch, and the video about Delft was filmed in this crime-ridden area of the Cape Flats. For the video of Oppie parara the Parade in Cape Town was visited. They also had to set a paper and a memorandum.

Each group had to present to the class on their poem and video. It served as an oral assessment, while simultaneously showing whether the student could teach. The enthusiasm with which the class tackled the project, and the competition to win the prize for the best team, motivated everyone to give their best. Groups assessed each other and I was pleasantly surprised by how strict they were. On the reflection form filled in afterwards, students asked for more such opportunities.

Paradigm shift

A crisis can lead to a unique opportunity. It requires teachers/lecturers to be innovative in their thinking about classrooms.

The American futurist Alvin Toffler said that all we can be sure of is change. ChatGPT has shaken academics out of their academic nap. There is no longer a place for handing out notes which students must memorise.

A paradigm shift is required similar to the mind shift made with the arrival of the internet. To keep up with the rapid development of AI and remain relevant, education is forced to adjust their approach to TLA.

Here to stay

Educational institutions will thus have to prepare for the world of AI. There are concerns. There will always be. But it would not be wise to discourage students from making use of it. Students will use it, just like they use smartphones, calculators and Google. Rather we must equip students with the skills to use ChatGPT critically and responsibly, in which case it can follow the same route as previous AI models.

ChatGPT is here to stay but will never replace humans. It facilitates calculations but still requires human intervention to find the right answers.