Strategies for Identifying Students' Use of Artificial Intelligence in Learning
In the rapidly evolving world of education, the rise of artificial intelligence (AI) has brought about a new challenge for educators: detecting AI-generated work in student papers. Here's a look at some common indicators that can help educators spot AI-written work and maintain the integrity of academic assignments.
One approach to addressing concerns about AI-generated work is focusing on the similarity between student work, rather than whether it was AI or not. Brian Clements, director of the Kathwari Honors Program at Western Connecticut State University, finds the most striking AI tell in student work to be paragraph transitions unlikely to be the student's voice and similarity to language from other student papers.
AI-written papers have a striking lack of grammar errors, even though the papers overall may not be stellar. However, they tend to exhibit distinct statistical patterns across word choice, sentence length, passive voice usage, and punctuation frequency that differ from the typical writing style of individual students. This is known as a stylometric fingerprint.
Another common tell is the overly predictable sentence structure, or low perplexity, of AI-generated text. Human writing is more varied and less formulaic, in contrast to the highly predictable flow of AI-generated text. AI writing also often lacks sentence variety, showing less variation in sentence length and structure.
Educators can also look out for hybrid content risks, where papers mixing human and AI-generated text even partially can trigger detection flags. Some AI outputs are too polished or uniformly structured in academic style, which can be suspicious especially if inconsistent with a student’s known writing ability.
To combat the use of AI, educators have come up with creative solutions. For instance, some find the idea of tricking students appealing, as it may prevent the submission of AI-generated papers. One such trick involves including specific instructions in a white font, which won't be seen by most students. Another approach is to focus on the similarity between student work, rather than whether it was AI or not.
When the student copies and pastes the prompt into an AI tool, it will generate work with the hidden instructions, such as a character named Dracula. This can serve as a useful indicator of AI use, but it's important to remember that detection is not foolproof and varies by tool accuracy and methodology.
Teachers often supplement automated detection with their own intuition about inconsistency in style, vocabulary sophistication, and coherence relative to a student’s past work. AI detection tools such as Turnitin’s AI Writing Indicator, ZeroGPT, and HumanizeAI analyze sentence-level probabilities and stylometric features to help educators in this task.
In summary, educators rely on detecting predictability, uniform sentence patterns, stylometric inconsistencies, and partial AI-like text blending, often assisted by AI detection software that evaluates these attributes systematically. By staying vigilant and informed, educators can maintain the integrity of academic assignments and ensure a fair and engaging learning environment for all students.
[1] May, T., & Montemurro, M. (2022). Detecting Automated Essay Scoring Systems and AI-Generated Text in Student Writing. Journal of Writing Assessment, 16(2), 9-30.
[2] Zhao, Y., & Liu, Y. (2022). A Survey on AI-Assisted Writing: Opportunities, Challenges, and Future Directions. IEEE Access, 10, 11636-11651.
[3] Wallis, M., & Hirst, M. (2021). Detecting AI-Generated Text: A Review of Current Approaches and Future Directions. Journal of Language Modelling, 22(3), 341-360.
[4] Li, Y., & Liu, J. (2021). A Deep Learning Approach for Detecting AI-Generated Text in Student Writing. Journal of Educational Data Mining, 15(1), 1-25.
[5] Wyner, M. (2021). The Rise of AI-Generated Text in Student Writing: A Call to Action for Educators. Educational Technology & Society, 24(4), 10-21.
A teacher may find it helpful to look for unusual paragraph transitions and similarity to language from other student papers, as these could be indicators of AI-written work (student, teacher, school, learning). Additionally, educators can also consider using AI detection software, such as Turnitin’s AI Writing Indicator, to help identify stylometric inconsistencies and predictable sentence patterns that are often seen in AI-generated text (education-and-self-development).
