The Rubric I Almost Used Without Reading It Twice

Last term I generated a rubric for a persuasive writing assignment using an AI rubric generator, glanced at it, thought it looked reasonable, and nearly printed thirty copies before something made me stop and read it a second time.

The problem was not obvious on a first read. The four performance levels were labeled clearly. Each row addressed a different criterion. It looked like a rubric. What it did not do, once I actually compared the language across the levels, was describe four genuinely different kinds of student writing. It described one kind of writing with four different levels of enthusiasm attached to it.

Take the row for evidence use. The top level said the student uses strong evidence that clearly supports the claim. The level below said the student uses evidence that supports the claim. The level below that said the student uses some evidence that somewhat supports the claim. The bottom level said the student uses little evidence.

Read that again slowly. Those four sentences are not describing four different qualities of thinking. They are the same sentence with the adjectives quietly removed one at a time. A student reading that rubric before writing their essay would have no idea what actually distinguishes an excellent argument from an adequate one, because the rubric never tells them. It just tells them adequate is less than excellent, which they already knew.

That is the specific failure mode of most AI generated rubrics, and it is worth understanding clearly before you build a workflow around any AI rubric generator.

Why This Happens and Why It Matters More Than It Seems

Good rubrics use language that is qualitatively different across performance levels, not just quantitatively different. The distinction sounds academic until you see it in practice.

A qualitatively distinct rubric row for evidence use might describe the top level as using evidence that connects directly to the claim through explicit reasoning, naming exactly how the evidence proves the point rather than simply presenting it. The level below might describe evidence that is relevant but left to speak for itself, without the writer explaining the connection. The level below that might describe evidence that is present but does not clearly relate to the specific claim being made.

Those three descriptions are teaching a student something different at every level. A student reading them before writing knows exactly what separates strong analytical writing from writing that just includes quotes without explaining them. That is what a rubric is actually for. It is not primarily a grading tool. It is a description of what quality looks like, written specifically so a student can read it before they start and understand what they are aiming for.

Most AI rubric generators, run with a simple prompt describing the assignment and asking for a four level rubric, default to the quantitative pattern. Strong, adequate, developing, weak. Same sentence, four intensities. This happens because that pattern is extremely common in the training data these models learned from, since a huge number of rubrics that exist online and in teacher resource libraries are built this exact way. The AI is reflecting a common but weak convention, not inventing a new problem.

The Specific Prompt Change That Fixes This

The fix is not a different tool. It is one additional instruction in your prompt, and it changes the output meaningfully across every AI tool I have used for this task.

Instead of asking for a rubric on a topic, ask specifically for a rubric where each performance level describes a genuinely different kind of thinking or writing, not simply more or less of the same quality. Ask the tool to make sure a student could tell the difference between two levels even without seeing a score attached, based purely on what the description says the writing does.

Here is a version of that instruction you can copy directly into any AI tool.

Generate a rubric for this assignment with four performance levels. Do not use quantitative language across levels, meaning avoid descriptors that differ only by words like strong, adequate, some, or little describing the same underlying quality. Each level should describe a genuinely different kind of thinking or writing that a student could recognize even without a score attached. For each criterion, explain what specifically changes about the student's approach between each level, not just how much of the same thing is present.

That last sentence, asking the tool to explain what specifically changes rather than how much is present, is the part that produces the biggest improvement. It forces the model to think about the actual difference in cognitive demand between levels rather than just varying the intensity of praise.

What Good Output Actually Looks Like

After adding that instruction, the evidence use row for the same persuasive writing assignment came back looking like this across the levels.

Top level: the student selects evidence specifically because it proves the claim, and explains the logical connection between the evidence and the argument in a way that would convince a skeptical reader.

Second level: the student selects relevant evidence and states that it supports the claim, but does not fully explain the reasoning connecting the two, leaving some of the logical work for the reader to do.

Third level: the student includes evidence related to the general topic, but the connection to the specific claim is unclear or requires the reader to infer the intended purpose of the evidence.

Bottom level: the student includes little evidence, or the evidence included is unrelated to the claim being argued.

Every level there describes something a teacher could actually recognize while reading a specific student's paragraph. A student reading this rubric before drafting understands that the goal is not just including a quote, but explaining why the quote proves their point. That is teaching through the rubric itself, before a single essay has been written.

Where AI Rubric Generators Still Fall Short Even With a Good Prompt

Even with the improved prompt, review every generated rubric for two specific things before using it.

First, check whether the levels are actually achievable in the order they are presented. Occasionally a generated rubric places a genuinely harder skill at a lower level than an easier one, simply because the model associated certain vocabulary with higher performance without checking the actual cognitive demand. Read through the levels in order and confirm that they represent an honest progression of difficulty.

Second, check for criteria that quietly test something other than what the assignment is meant to assess. A rubric for a persuasive essay occasionally includes a heavily weighted grammar and mechanics criterion that ends up functioning as the deciding factor in a student's overall grade, even though the assignment's actual purpose is argumentative reasoning. If mechanics matter, weight it appropriately rather than letting it accidentally dominate.

Both of these checks take about three minutes on a typical four criteria rubric. They are worth doing every time, regardless of how good the prompt was, because the output still requires a human reading it with the specific assignment and specific students in mind.

The Tools Worth Using for This Specific Task

MagicSchool AI produces the most reliable qualitative distinction across levels when you use its rubric generator with the specific instructions above added to your input. The platform was built for educational use specifically, and its default behavior leans slightly closer to qualitative language even without the extra prompting, though adding the instruction still improves it further.

Claude produces strong results when given the full prompt described above, particularly for rubrics assessing more nuanced or creative work, such as creative writing or open ended project based assessment, where the criteria are less standardized than something like a lab report format.

General purpose tools without education specific training, including the free tier of most general assistants, need the full explicit instruction every time. Without it, the default output reliably falls back into the quantitative pattern described at the start of this piece.

What I Actually Do Now

I do not generate a rubric and use it. I generate a rubric, read every level of every criterion out loud to myself, and ask whether I could describe the actual difference between two adjacent levels to a student without using the words in the rubric itself. If I cannot do that, the rubric has not actually described anything. It has just assigned different amounts of enthusiasm to the same sentence.

That check takes about four minutes for a typical four by four rubric. It caught the persuasive writing rubric that almost went out unedited last term. I have not skipped it since.