Generative Artificial Intelligence (GenAI) in Higher Education

Research has found high rates of both false positives (that is, actual human work accused of being AI-written) and false negatives (that is, known AI composition that was judged to be human-written) (Edwards, 2023; Fowler, 2023). OpenAI, the company that developed ChatGPT, also developed an AI detection tool. They eventually decommissioned it due to inaccuracy (Nelson, 2023). Consider that: Even the developers of this popular chatbot could not develop a tool to recognize that chatbot's use with sufficient accuracy. Universities like Vanderbilt and the University of Pittsburgh at first adopted detection tools but eventually shut them off as well because their efficacy was not supported by the data (Watkins, 2023).

In cases of traditional plagiarism, it is generally a simple matter to produce a student's paper and the plagiarized source side-by-side, thus proving the plagiarism. However, in cases of suspected generative AI use, there is no such evidence. The instructor cannot unequivocally prove that the student used AI. And likewise, the student has little way to defend themselves and unequivocally prove that their writing is original or that only standard computerized writing tools were used, such as spelling and grammar checking.

Instructors should not risk making serious accusations about student integrity based only on highly flawed detection tools.

Instead, MIT recommends:

• Include clear policies in the course syllabus about whether and how generative AI use is permitted. When possible, try to include examples that help students distinguish between permitted uses (e.g., reviewing grammar or exploring ideas before formulating a research question) and prohibited uses (e.g., writing whole blocks of content for an assignment).
• Openly discuss generative AI with your students. Educate them on how it actually works, its strengths and weaknesses, and your reasons for permitting or prohibiting it in given contexts (why are certain uses okay and others aren't; what is the impact to their learning?).
• Be cautious about converting all coursework to in-person formats to eliminate risk of generative AI use. Not all students can write by hand quickly and easily; they may be disadvantaged by assessments such as in-class handwritten essays. At the same time, all students would lack time for revision to show their best work. Similarly, non-native English speakers and students with social anxiety or neurodivergence may be disproportionately challenged by oral presentation assessments. Despite feeling concern about generative AI use, strive to ensure inclusive teaching.

References:

Edwards, B. (2023, July 14). Why AI detectors think the US Constitution was written by AI. Ars Technica. https://arstechnica.com/information-technology/2023/07/why-ai-detectors-think-the-us-constitution-was-written-by-ai 

Fowler, G. A. (2023, April 14). We tested a new ChatGPT-detector for teachers. It flagged an innocent student. The Washington Post. https://www.washingtonpost.com/technology/2023/04/01/chatgpt-cheating-detection-turnitin

MIT Sloan Teaching & Learning Technologies. AI detectors don't work: Here's what to do instead. https://mitsloanedtech.mit.edu/ai/teach/ai-detectors-dont-work/

Nelson, J. (2023, July 24). OpenAI quietly shuts down its AI detection tool. Decrypt. https://decrypt.co/149826/openai-quietly-shutters-its-ai-detection-tool

Watkins, Marc. (2023, September 3). Beyond ineffective: How unreliable AI detection actively harms students. https://marcwatkins.substack.com/p/beyond-ineffective-how-unreliable