Home > Uncategorized > Robots Cannot Replace Humans When it Comes to Grading Essays— But No Matter! They are Cheap, Fast, and Unbiased.

Robots Cannot Replace Humans When it Comes to Grading Essays— But No Matter! They are Cheap, Fast, and Unbiased.

July 17, 2018

NPR recently ran a story by Tovia Smith on the use of robots (or more precisely computers) to grade student essays and found to no educators’ surprise, that they were not not up to the task. The story opens with this quiz:

Multiple-choice tests are useful because:

A: They’re cheap to score.

B: They can be scored quickly.

C: They score without human bias.

D: All of the above.

It would take a computer about a nano-second to mark “D” as the correct answer. That’s easy.

But now, machines are also grading students’ essays. Computers are scoring long form answers on anything from the fall of the Roman Empire, to the pros and cons of government regulations.

From this point forward in the story, Ms. Smith provides an account of the expansion of the use of computer-graded essays, work done by MIT research affiliate, Les Perelman, who has developed algorithms that generate nonsense responses to machine-graded essays that yield high scores, a rebuttal to Mr. Perelman’s work by ETS, who argue, in effect, that if someone is smart enough to game to the essays they deserve the high grade, and the “cat-and-mouse” game underway to catch students who use algorithms in states that have adopted computerized grading.

At the end of the report, Ms. Smith hits on the real problem with computerized grading: it compels teachers to teach students formulaic writing.

Indeed, being a good writer is not the same thing as being a “higher-scoring GRE essay writer,” says Orion Taraban, executive director of Stellar GRE, a tutoring company in San Francisco.

“Students really need to appreciate that they’re writing for a machine … [and when students] agonize over crafting beautiful, wonderfully logically coherent and empirically validated paragraphs, it’s like pearls before swine. The computer can’t appreciate what this person has done and they don’t get the score that they deserve.”

Instead, Taraban tutors students to give the computer what it wants. “I train them in fabricating evidence and fabricating fake studies, which is a lot of fun,” he says, quickly adding, “but I also tell them not to do this in real life.”

For example, when writing a persuasive essay, Taraban advises students to use a basic formula and get creative. It goes something like this:

A [pick any year] study by Professor [fill in any old name] at the [insert your favorite university] in which the authors analyze [summarize the crux of the debate here], researchers discovered that [insert compelling data here] … and that [offer more invented, persuasive evidence here.] This demonstrates that [go to town boosting your thesis here!]”

It results in a kind of mad-lib writing that is anything but artful, thoughtful, or pleasing to read. But it is cheap, easy, unbiased, and unbiased…. and the ultimate triumph of efficiency over excellence.

 

Advertisements
%d bloggers like this: