What happens when AI does the grading?

If you’re a physics educator who has tried generative AI, your mind has probably wandered through one, or all, of a few thoughts.

“Maybe this could save me time,” you think, wincing at the stack of ungraded exams on your desk. So you ask ChatGPT a question, and it gives an unnervingly good answer. "Oh my," you think. "Will this replace me?" Then it makes a couple of ridiculous mistakes, and you think, "This is useless."

Each one might have some truth to it. But after spending months testing the abilities and limits of GPTs, I now worry less about whether or when AI will replace human educators, and I think more about other important questions: What is the unique value of humans in education, and how can we work with AI to amplify those values?

When I began studying generative AI grading, I too rode a roller coaster of predictions. I started with a simple goal: get ChatGPT to grade student written responses. My first few trials were so good that I thought graders would soon be out of a job. Then it screwed up spectacularly, and I wondered if AI grading was a disaster waiting to happen.

But I’m like a lot of physicists: If you say there’s a skunk under my porch, I’ll run toward it for a closer look.*

I decided to test AI’s grading ability thoroughly, starting by replicating what others had done. Nearly a dozen studies have shown that models like GPT-4 could score students’ written responses at accuracy levels comparable to human raters.

I then upped the ante. I collected around 75 student responses to several calculation-heavy physics problems for which students had typed answers using a mixture of words and their own creative math notation, from "½ k x^2" to "0.5k x2." Instead of asking the AI for a single numeric score, I gave it a multi-component rubric and tried to get AI to do three things: Evaluate each rubric component individually; produce some type of “confidence” score for its own grading; and, for each answer, write feedback for a student.

Some of the results were expected. After months of prompt engineering, GPT-4o's grading agreed with human graders as much as, or more than, human graders agreed among themselves. Such high-quality grading takes GPT-4o about 2 hours for every 100 responses — not very different from a human, but at $5 to $10 for the batch, far cheaper than a human.

But two results surprised me. The first was how transparent the entire grading process had become. To get accurate results, we instructed the AI to write down its reasoning for every point it assigned or deducted. This prevented the AI from just making up scores that looked right. As a byproduct, the AI meticulously explained every single grading decision, which I would never expect from a human grader.

The second was how easy it became to give students detailed, personalized feedback. After just one morning of light coding and prompt engineering, AI gained the ability to write a personalized explanation for each student of what they got right or wrong — something I'd never seen any colleague who teaches more than 20 students do regularly.

Finding the human role

Since I started this project, I’ve wondered: Am I just finding a way to eliminate human graders? But the investigation actually helped me identify what humans can uniquely contribute.

There are at least three roles that still need a human, at least for now. First, a human has to explain the rubric to the AI. The biggest jump in accuracy came when I started telling the AI what an intro physics instructor would normally count as valid evidence for each rubric component. When a student writes a sloppy answer like “mv^2 - gh = 0,” do I want to give them credit for thinking about energy? Without that context, the AI acted like either an overly strict or an overly generous teacher. In contrast, human graders who had taken the course themselves graded just fine without the explanation. This seems to come from the shared experience between instructors and students — something that apparently can't be learned from massive training data.

Second, humans need to be the supervisors. To achieve high grading accuracy, I always had to let the AI grade a small batch first, have a human review, tweak the prompts, and then let AI grade the rest. Human review is needed not only at the beginning, but also the end. There were always a handful of responses where the AI couldn't make up its mind — it would offer three or four different outcomes when asked to grade five times. Those needed human eyes, too.

Finally, all that added transparency creates a new challenge. When students can see detailed reasoning for every point they missed, many will disagree with the AI's decisions. These students will want to argue with a human instructor, in much the same way that I yell, “talk to a human representative!” when I sense a chatbot on the phone.

Should our next grader be an AI?

Yes, I think so, but not because it's cheaper. Rather, it’s because an AI grader provides unparalleled transparency in its grading process, and it can be continuously and iteratively improved.

I know many will disagree, and for good reason. We haven’t yet mentioned the environmental cost of running the large language models. Also, AI graders can make mistakes, show bias against certain language patterns, and miss unconventional or creative answers. Students may try to fool AI graders, too.

But are human graders immune to these problems? Probably not. And we usually have no idea how often our human graders make mistakes or show bias. How can you ask a grader to justify every point they assign when you've just handed them 100 responses?

With AI, we can spot most problems, measure them, and fix them. If we work on improving the prompts and process, the next batch of students gets more accurate and fairer grading than the previous batch. Will my next human grader be better than my last one? Who knows?

The real advantage isn't efficiency. It's transparency and the ability to continuously improve.

Will humans become obsolete?

If AI graders are so good, shall we stop hiring human graders altogether? I think it depends on what we mean by “graders.”

If a grader’s only job is to assign scores to student responses, then yes, that person will probably become obsolete. But grading can be so much more than that. As I found, the AI grading process relies on humans to serve in three roles: explainers, supervisors, and final arbitrators.

Will those roles disappear as more powerful models become available? I don’t see it happening anytime soon. This is because I see grading not as simply comparing student answers to a standard solution, but as a way to communicate our expectations to students — what we want from them at each stage in their learning. Humans must also monitor whether AI is giving us what we want, and make corrections when AI deviates from what we want. These are likely to remain human tasks.

There's another, bigger reason I don't think humans will become obsolete in education. Yes, research shows that AI can be more efficient, transparent, and even accurate than many human instructors in daily tasks. But those results assume we keep doing the same regular, repetitive tasks in education — assigning the same textbook questions, giving the same lectures, administering the same exams. We've become so used to these routine parts of education that we forget they were designed to accommodate resource constraints, not to optimize learning.

Now that generative AI can help us overcome many of these constraints, there's no better time to rethink the basic building blocks of education itself.

If we can evaluate any answer without worrying about grading time, what types of questions will we choose to ask students? If students can access any resource they need, in any format they prefer, do we still want them to spend time in a classroom designed for centralized dissemination of knowledge? If they still need to be in one room, what should they be doing? If we can create and grade new exams at any time, do we still need midterm and final exams? Do we even need "terms"?

These questions, and the tasks they could spur, are uniquely human because they require innovation — testing new ideas that haven't been documented yet. As OpenAI co-founder Andrej Kaparthy puts it, LLMs are essentially high-resolution compressions of the internet. But innovation, by definition, means trying things that haven’t been tested or documented. And deciding what's worth trying? That's entirely about "what we want."

This opportunity comes with urgency. Today's students will live in a world deeply integrated with AI, but our educational system isn't preparing them for that reality. If we keep lecturing and grading multiple-choice questions, we'll become irrelevant soon enough.

Being uniquely human

As I wrote this article about whether AI will replace humans in education, I found limited use for AI. The experience is uniquely mine, and the ideas are relatively new, so they're still scarce in the training data. Sure, I asked ChatGPT to find the right word or check a reference now and then, but that's about it.

Meanwhile, my daughters are in kindergarten, happily learning from a teacher who somehow gets a room full of six-year-olds to sit still and learn to read and count. If there's one type of teacher that won't be replaced by AI anytime soon, my bet is on them, not us college professors. It’s hard to imagine a machine doing a job like that.

Most discussions about AI in education focus on skill mastery and knowledge delivery. But there's a uniquely human component that is arguably more important: I learned to think like a scientist by watching my mentors think, then mimicking and internalizing their approaches. Can future scientists develop that kind of thinking just by talking to an AI? I doubt it.

The real promise isn't that AI will replace educators. It’s that AI might free us from routine tasks, so we can focus on what’s more important. Perhaps when we no longer spend hours grading the same types of answers, we can think, discuss, and innovate more with our students — and, in the process, discover the parts of education that are uniquely human.

*Dave Pritchard, my mentor at MIT, gave me the “where’s the skunk?” story and mentality.