What do we do with AI Similarity Reports?

We use Turnitin, which also offers AI similarity reports. There is nearly universal acceptance of Turnitin similarity reports in higher education, but AI similarity reports are a different case. I know a few students whom I trust had high similarity rates. This week, we discussed this with my colleagues. Some colleagues are very aggressive and tend to accept the Turnitin results. However, the majority is aware of the false positive issue. For now, we will check the reports, and if there is a high rate, we decide to talk to the student. If not persuaded, we ask them to write again more carefully. I feel like this will continue to be debated.

In the meantime, I asked Perplexity to summarize the issue globally:

Higher education institutions have been grappling with the challenges of AI similarity reports, particularly in terms of false positive issues. These reports are generated by tools like Turnitin, which use algorithms to detect similarities between submitted work and existing sources. However, the increasing use of generative AI has led to a rise in false positives, where students are incorrectly flagged for academic dishonesty[2].

### False Positive Issues and Consequences

False positives can have severe consequences for students, including damage to their academic records and reputations. Students who are falsely accused of AI cheating may face penalties such as failing grades, suspension, or even expulsion. These consequences can be devastating and may lead to long-term harm to a student’s academic and professional prospects.

### Student Strategies for Resolving False Positive Issues

Students who are falsely accused of AI cheating can take several steps to resolve the issue:

1. **Initiate a respectful dialogue**: Approach the instructor with a polite and conciliatory tone, explaining that you did not use AI for cheating or only used it within permitted course guidelines[2].
2. **Provide evidence**: Offer evidence to support your claim, such as documentation of your work process or peer reviews that confirm your originality.
3. **Seek support**: Consult with academic advisors, student services, or ombudsmen who can help mediate the situation and advocate for your rights.
4. **Escalate if necessary**: If the issue is not resolved, students can escalate the matter to higher authorities, such as department chairs or deans.

### Institutional Responses

Higher education institutions are also taking steps to address the issue of false positives:

1. **Improved training**: Institutions are providing training for instructors on how to properly use AI detection tools and how to identify false positives.
2. **Algorithmic updates**: Tool providers like Turnitin are continually updating their algorithms to improve accuracy and reduce false positives.
3. **Student education**: Institutions are educating students on the proper use of AI tools and the importance of academic integrity.
4. **Policy revisions**: Some institutions are revising their policies to better address the challenges posed by AI similarity reports and false positives.

### Future Directions

The increasing use of generative AI in higher education necessitates a proactive approach to addressing false positive issues. Institutions must prioritize the development of robust assessment methods that do not rely solely on AI detection tools. Instead, they should focus on cultivating a culture of academic integrity, where students are encouraged to use AI as a tool for learning and innovation, rather than a means to cheat[1][2].

### Conclusion

Higher education institutions must be proactive in addressing the challenges posed by AI similarity reports and false positive issues. By providing students with the necessary support and resources, and by continually improving the accuracy of AI detection tools, institutions can ensure that academic integrity is maintained while also embracing the benefits of AI in education.

Citations:
[1] https://arxiv.org/ftp/arxiv/papers/2312/2312.05241.pdf
[2] https://www.washingtonpost.com/technology/2023/08/14/prove-false-positive-ai-detection-turnitin-gptzero/
[3] https://surferseo.com/blog/avoid-ai-detection/
[4] https://www.turnitin.com/blog/ai-plagiarism-changers-how-administrators-can-prepare-their-institutions
[5] https://staff.acu.edu.au/our_university/news/2023/may/turnitin-ai-detection-false-positive-risks
[6] https://originality.ai/blog/ai-content-detector-false-positives
[7] https://www.teqsa.gov.au/sites/default/files/2023-05/AAIN-Institutional-Responses-Generative-Artificial-Intelligence.pdf
[8] https://www.turnitin.com/blog/new-research-turnitin-s-ai-detector-shows-no-statistically-significant-bias-against-english-language-learners
[9] https://contentwriters.com/blog/how-to-avoid-an-ai-detection-false-positive/
[10] https://blogs.lse.ac.uk/impactofsocialsciences/2023/07/21/generative-ai-should-mark-the-end-of-a-failed-war-on-student-academic-misconduct/
[11] https://www.turnitin.com/blog/understanding-false-positives-within-our-ai-writing-detection-capabilities
[12] https://www.eastnets.com/thought-leadership-articles/reducing-false-positives-using-contextual-ai
[13] https://www.insidehighered.com/news/tech-innovation/artificial-intelligence/2024/02/09/professors-proceed-caution-using-ai
[14] https://www.diplomacy.edu/updates/universities-stop-using-ai-detection-tool-such-as-turnitin/
[15] https://www.researchgate.net/publication/370224159_Artificial_intelligence_in_higher_education_the_state_of_the_field
[16] https://www.vanderbilt.edu/brightspace/2023/08/16/guidance-on-ai-detection-and-why-were-disabling-turnitins-ai-detector/
[17] https://www.icms.edu.au/news/academic/icms-response-academic-integrity-ai/
[18] https://www.itnews.com.au/news/australian-universities-hits-and-misses-detecting-chatgpt-in-assignments-599141
[19] https://www.economicsnetwork.ac.uk/handbook/plagiarism_2023
[20] https://www.researchgate.net/publication/378200791_AI_Detection%27s_High_False_Positive_Rates_and_the_Psychological_and_Material_Impacts_on_Students