Did a Human Write That? Detect AI Writing With These Tips

CNET

5 hours ago

AI has exploded in popularity over the last few years, thanks in no small part to large learning models like ChatGPT. As AI has evolved, it's become an easy way to whip up emails, and for students to write their assignments. The days of paying someone else to write their essay are over, when there are AI writing tools that can do the deed in mere moments.
As a professor of strategic communications, I encounter students using AI tools like ChatGPT, Grammarly and EssayGenius on a regular basis. It's usually easy to tell when a student has used one of these tools to draft their entire work. The telltale signs include ambiguous language and a super annoying tendency for AI to spit out text with the assignment prompt featured broadly.
Don't miss any of CNET's unbiased tech content and lab-based reviews. Add us as a preferred Google source on Chrome.
How to tell if it was written by AI
Some of the most common ways to tell if something was written using AI are:
Key terms from your assignment prompt are used repeatedly.
Inaccurate facts are included, thanks to the AI chatbot hallucinating.
Sentences don't sound natural.
Explanations are generic and repetitive, rather than actually leading anywhere.
The tone doesn't sound like their usual writing style.
For example, a student might use ChatGPT -- an AI chatbot that uses large language model learning and a conversational question and answer format to provide query results -- to write a short essay response to a prompt by simply copying and pasting the essay question into the tool.
Take this prompt: In 300 words or fewer, explain how this SWAT and brand audit will inform your final pitch.
This is ChatGPT's result:
Screenshot by Rachel Kane/CNET
I have received responses like this, or those very close to it, a few times in my tenure as a teacher, and one of the most recognizable red flags is the amount of instances in which key terms from the prompt are used in the final product.
Students don't usually repeat key terms from the prompt in their work in this way, and the results read closer to old-school SEO-driven copy meant to define these terms rather than a unique essay meant to demonstrate an understanding of the subject matter.
But can teachers use AI tools to catch students using AI tools? I devised some ways to be smarter in spotting artificial intelligence in papers.
Catch AI cheaters
Here's how to use AI tools to catch cheaters in your class.
Understand AI capabilities
There are AI tools on the market that can scan an assignment and its grading criteria to provide a fully written, cited and complete piece of work in a matter of moments. Some of these tools include GPTZero and Smodin. Familiarizing yourself with tools like these is the first step in the war against AI-driven integrity violations.
Do as the cheaters do
Before the semester begins, copy and paste all your assignments into a tool like ChatGPT and ask it to do the work for you. When you have an example of the type of results it provides specifically in response to your assignments, you'll be better equipped to catch AI-written answers. You could also use a tool designed specifically to spot AI writing in papers.
Get a real sample of writing
At the beginning of the semester, require your students to submit a simple, fun and personal piece of writing to you. The prompt should be something like "200 words on what your favorite toy was as a child," or "Tell me a story about the most fun you ever had." Once you have a sample of the student's real writing style in hand, you can use it later to have an AI tool review that sample against what you suspect might be AI-written work.
Ask for a rewrite
If you suspect a student of using AI to cheat on their assignment, take the submitted work and ask an AI tool to rewrite the work for you. In most cases I've encountered, an AI tool will rewrite its own work in the laziest manner possible, substituting synonyms instead of changing any material elements of the "original" work.
Here's an example:
Screenshot by Rachel Kane/CNET
Screenshot by Rachel Kane/CNET
Now, let's take something an actual human (me) wrote, my CNET bio:
Screenshot by Rachel Kane/CNET
The phrasing is changed, extracting much of the soul in the writing and replacing it with sentences that are arguably more clear and straightforward. There are also more additions to the writing, presumably for further clarity.
Can you always tell if something was written using AI?
The most important part about catching cheaters who use AI to do their work is having a reasonable amount of evidence to show the student and the administration at your school if it comes to that. Maintaining a skeptical mind when grading is vital, and your ability to demonstrate ease of use and understanding with these tools will make your case that much stronger.
Good luck out there in the new AI frontier, fellow teachers, and try not to be offended when a student turns in work written by a robot collaborator. It's up to us to make the prospect of learning more alluring than the temptation to cheat.