Princeton student builds GPTZero to check for ChatGPT-based plagiarism

Princeton University student Edward Tian did not spend New Year’s eve the way most people do. Instead, Tian was hard at work building a tool that could arguably benefit almost anyone today. On January 3, Tian tweeted about his work, unveiling the tool — called GPTZero — which can detect whether a piece of text has been written by a human or generated by artificial intelligence (AI) research firm OpenAI’s ChatGPT platform.

GPTZero is an app that can detect whether an essay is handwritten or written by ChatGPT. The hype around OpenAI’s ChatGPT began in late November 2022 when the company launched a beta version for everyone to use. Multiple concerns have been raised about the possible misuse of such a platform since.

For instance, earlier this month, New York City’s Department of Education announced a city-wide ban of access to ChatGPT among all public schools. Jenna Lyle, a department spokesperson, said the decision stems from “concerns about negative impacts on student learning, and concerns regarding the safety and accuracy of contents”.

Tian, whose tweet has now gone viral with over 7 million views, is a computer science major minoring in journalism, and said he was motivated to build the tool because of “increasing AI plagiarism”. He argued that high school teachers wouldn’t want students to use ChatGPT to write history essays — a question that many critics have asked since ChatGPT went viral on the Internet, writing everything from poems to computer code.

“In short, there's so much ChatGPT hype going around. Is this and that written by AI? We as humans deserve to know!” he wrote on Twitter. He posted demonstrations of how the application works, using an article called Frame of Reference written for the New Yorker by John McPhee and the text of a Linkedin Post by Nandini Jammi, the co-founder of adtech watchdog Check My Ads.

The application determines that the text is handwritten or AI-generated based on two properties: Perplexity and Burstiness. Perplexity refers to the randomness of a text to a model and Burstiness determines the perplexity which remains constant in machine generated text and varies for human written text or rather the complexity.

To be sure, Tian’s tool may not be perfect yet. According to a report by Futurism, that the accuracy of GPTZero while high, is not always precise. The publication fed 16 pieces of text, each at least 300 words in length, half of which were generated by ChatGPT, and Tian’s platform failed to identify the AI’s work in one out of the eight artificially generated pieces.

Tian’s app is freely available for everyone right now, and can be accessed from this link. Like ChatGPT itself, GPTZero too is in a beta phase right now.