Can you tell if this article was written by artificial intelligence?

The explosion in popularity of large language models like ChatGPT prompted AI researcher Dr. Maiga Chang to shift his focus towards the question of authorship

With the proliferation of AI applications like ChatGPT that can write at a level as a sophisticated as many humans, the question of “did an AI write this?” has become far more important. 

This has prompted Athabasca University AI researcher Dr. Maiga Chang, a professor in the School of Computing and Information Systems in the Faculty of Science and Technology, to shift his research focus to this topic, called authorship forensics. This includes work developing AI that can differentiate between AI writing and human writing—or even identify a specific human author of a text. 

“We do think that people when writing will prefer to use different kinds of nouns, noun combinations, pronouns; some people use letters over numbers, that sort of thing,” he said.  
“People have different behaviours.” 

There are many contexts in which this work could be applied. Being able to determine if AI wrote something, rather than a student, has clear impacts in an educational context, where having someone else do your homework is not allowed. But being able to differentiate between human authors could have important implications for law enforcement. 

We do think that people when writing will prefer to use different kinds of nouns, noun combinations, pronouns; some people use letters over numbers, that sort of thing.

Dr. Maiga Chang, associate dean of research and innovation, Faculty of Science and Technology

Chang said he hopes to see this work evolve to the point where an AI could reliably identify the author of a text. This could have impacts not only in helping teachers to understand whether students are doing their own homework, but also in helping law enforcement agencies identify who wrote a threatening letter or a ransom note. 

“That’s the optimal goal, but of course we’re not there yet,” he said. 

Chang’s background in AI goes back to the 1990s, and he has worked on many different aspects of AI since then. Mostly this focused on evolutionary computing with artificial neural networks—designing a computer program that can adapt and evolve to new data and events and provide better services.

Improving reliability and efficiency in authorship forensics

With the explosion in popularity of large language models like ChatGPT, Chang shifted his focus to improving authorship forensics, as what’s currently available is not particularly reliable. 

He explained that even the organization that created ChatGPT, OpenAI, released a classifier, meant to detect if a text is written by an AI, in January 2023 but then had removed it by July 2023 because of its low reliability. 

“Their classifier is not currently reliable,” he said, noting with a true positive rate of about one in four and a false positive rate of about one in ten, the precision of this classifier was not high enough to be reliable. 

One of Chang’s goals is to improve that level of precision and reliability. He explained his approach has been to try to identify specific features for the AI to look at, similar to the way facial recognition works. With facial recognition, the AI may only analyze specific identified features: the corners of the eye or mouth, placement of the nose, etc. It can then determine if the person trying to unlock a smartphone, for example, is authorized to do so. 

In the case of an AI analyzing a text, rather than a face, these features could include things like word choice, sentence length, or grammatical structure. 

Chang’s work in this area, with the help of research assistants and students, has had positive results. While it’s time consuming for an AI to perform the analysis, results have definitely improved. A three-class model, meaning one that compares text from three different sources, can be trained on a virtual machine in just 15 minutes. 

“The trained model can tell you whether or not this is written by a human, ChatGPT 3.5, or ChatGPT 4, within one minute, with a 98% precision,” he said. 

Chang emphasized that 100% precision is probably impossible. Since there’s no way for a human to be able to do this, there would be no way for humans to be able to provide the rules for an AI to provide 100% efficiency, either. 

Contributing to the authorship forensics project has made me feel as though I am potentially able to become a piece of the counterweight to the imbalance that AI is currently placing on society. I couldn't be prouder of the progress we have made.

Rob Schmidt, Bachelor of Science in Computing and Information Systems student

Opportunities for students to do AI research

Despite some limitations, Chang’s AI models are improving—and quickly—thanks in part to the support from students and other researchers. 

“We have many open projects for students,” he said. 

There are students already working on projects related to this authorship forensics research, whether they’ve applied for research assistantships, are doing undergraduate projects, or are working on related projects through the essay, project, or thesis options in the Master of Science in Information Systems (MScIS) degree. 

Rob Schmidt, who expects to finish the Bachelor of Science in Computing and Information Systems program in early 2025, said working as part of Chang’s research group provided several years' worth of practical real-world experience before he even graduated. 

“Contributing to the authorship forensics project has made me feel as though I am potentially able to become a piece of the counterweight to the imbalance that AI is currently placing on society,” he said. “I couldn't be prouder of the progress we have made.” 

Kevin Haghighat, an MScIS student Chang supervised and mentored, likewise speaks highly of his experience. 

“Working on this research project at AU has sharpened my analytical, research, and writing skills—key for teaching. It has also enhanced my ability to assess student work and support diverse learners in writing and comprehension,” he said. 

Even more opportunities are available now and will be in the future. On top of opportunities working to help improve the AI model itself, there is a lot of work to be done on developing a browser plug-in and a mobile app that could produce results for users. 

“There are many project opportunities that we’re opening right now with this authorship forensics research,” Chang said. 

Research assistant opportunities at AU are published to the Research Office website.