Submissions with an unacceptable level of AI text were reduced by 57%
"At the IMM Graduate School, maintaining academic integrity is at the core of our values, especially in the evolving landscape of AI-driven education. The collaboration with Norvalid has empowered us to ensure fairness in assessments by providing a multi-layered approach to authorship validation. This innovative solution not only enhances trust between educators and students but also supports our mission to foster a culture of ethical learning while embracing the potential of AI in education."
Riana Prins
Head of Assessments and Learning Management System
Who is IMM Graduate School?
The IMM Graduate School is a private higher education institution who offer degrees, diplomas and certificates in marketing management and supply chain management. Operating for more than 60 years in South Africa, they specialise in outcomes-based education curricula that supports flexible learning for studying online, on-campus and combinations of both. Students from more than 20 different countries around the world are enrolled in their programmes due to the flexibility of study options offered by the IMM Graduate School.
Challenge
Like many higher educational institutions across the globe, the IMM Graduate School noticed an impact on student submissions following the introduction of Generative Artificial Intelligence (Gen AI) tool since November 2022.
They had been using plagiarism detection tools to combat copy-and-paste practices. However, these tools were not able to accurately detect the misuse of Gen AI, which presented new academic integrity challenges. Because AI can write new text, this capability of AI enables students to prompt Gen AI to complete their assessments and submit the work as if it were their own.
Solution
The IMM Graduate School team was introduced to the Norvalid team at a conference, and following some introductions, it was decided to test the solution. This testing involved approximately 360 assessments from 90+ post graduate students over multiple submissions. The Norvalid tool was integrated into the IMM Graduate School’s Moodle Learning Management System used to support learning. To enable Norvalid to show the verification results, a student writing sample of approximately 500 words was collected from each student participating in the testing.
Students in the test group submitted their work through the Learning Management System as usual, with one key addition, each student had to answer personalised questions about their submission. Once the deadline passed, instructors could access detailed verification reports directly, offering a comprehensive, multi-layered view of student authorship including:
Four types of Evidence:- Verification result: A linguist comparison of the students’ submission against their own writing samples.
- Perplexity results: Measures the likelihood of human authorship based on text complexity (a contrast to traditional AI detection methods).
- Free Text Questions: Measures students' understanding of what they submitted for the purpose of authorship.
- Gap Match Questions: Asks students to complete missing parts of their own text, further verifying authorship.
The results of all four of these tests are calculated into a Student Authenticity Score. This is a summative measurement of the amount of original student writing in a submitted text, reflecting the strength of the “student voice.”
Key Insights & Results
By analysing these multiple layers of evidence, IMM Graduate School was able to see more than half of students changed behaviours to a status where their original writing was within an acceptable level of AI writing assistance.
The percentage of change over four submissions:
- More than 40% AI text (perplexity test) reduced by 57%
- Less than 80% student text (linguistic test) reduced by 27%
- The overall Student Authenticity Score below 4 out of 5 (all 4 tests combined) was reduced by 47%
Identify Cases for Intervention: Students with low “Student Authenticity Score” were flagged for further review, allowing educators to focus their time effectively.
Reduce False Positives: The AI detection tools flagged submissions as AI-generated; however, Norvalid with its multi-layers of evidence provided a holistic view validating the students' work. This ensured that students using AI for editing and refinement—rather than fully writing the submission—were not unfairly penalized.
Enhance Trust with Students and Improving Academic Integrity: Using the Norvalid tool to have informed conversations with students on how to improve their writing style. Thus supporting a culture of academic integrity in the age of AI.
This enables IMM Graduate School to move towards embracing AI within assessment in a more secure way. As a result, IMM Graduate School continues to collaborate with Norvalid to refine and expand the use of authorship validation, ensuring students are assessed with fairness, accuracy, and integrity.