Automatic Detection of AI Writing
As automated AI detection remains unreliable, Montclair has not approved or licensed any AI detection tools for evaluating student work at this time. Because these systems frequently both miss AI-generated content and incorrectly identify human-written work as AI-generated, their results should not be used as evidence of academic integrity violations.
Furthermore, AI-detecting software in its present state often raises equity and privacy concerns. Research suggests that AI detectors are more likely to flag content that was created by non-native English writers. Studies are underway to see if there is a similar bias against neurodivergent writers. This makes automatic AI detection potential liability for educators and institutions.
AI Detection Tools at Montclair
Turnitin's AI detection functionality, previously available to Canvas users at Montclair, was officially discontinued in 2023 (Provost's Memorandum on Turnitin AI Detection Functionality from 11/14/2023) due to concerns of reliability and bias. There is no plan to license another tool specifically trained for AI detection, as these concerns are unlikely to be fully eliminated.
Those faculty who choose to use a digital platform to assess student work for possible presence of AI-generated content should follow these best practices:
- use tools that are currently institutionally licensed to protect student data and student work. Montclair users currently have institutional access to Google's Gemini and MS Copilot, with protection from training on user data.
- maintain FERPA compliance at all times.
- disclose and explain to your students why you believe the use of automated detection of AI-generated content is necessary in each situation.
What Else You Can Do
If you suspect that one of your students’ submissions is AI-generated, we recommend the following:
-
Check manually for the red flags that are typical for AI writing
major factual errors in combination with uncharacteristically competent and confident writing; disciplinary knowledge far beyond expected course level and/or the specific student's typical performance; excessive wordiness and inefficiency of written expression (the writing is polished, but demonstrates little to no knowledge or learning tied to the course or assignment goals.)
-
Consider the big picture
Each of these red flags individually may occasionally occur in human work and writing. However, if you see multiple red flags in a submission, that's more informative, especially if coming from a student whose work in your course raises other concerns.
-
Talk to the Student
In AI-rich environments, products of student work may no longer demonstrate learning reliably and efficiently. However, a conversation with a student about the process that led to their submission can be informative and insightful. Here are a few tips to help facilitate a successful conversation.
- real-time conversation (in-person or on zoom) will work better than email;
- enter this conversation from a non-assuming standpoint: most of the time, it is not possible to say with full confidence that AI has been used to generate a submission.
- ask highly specific questions about the submission, especially the process that led to it and the parts that alarmed you.
-
When Resolution Isn't Possible
If the student denies the unauthorized use of AI, faculty have two options.
- grade the submission bypassing the issue of AI use, factoring in any errors or flaws and grading them fairly, as they are the student's responsibility. For a well-designed assignment, AI bots will often fail to generate work that satisfies even basic passing requirements.
- Start the Academic Dishonesty process for that student. This process can be educational for many students, especially if there is a fundamental misunderstanding of the principles and standards of academic honesty on the student's side.