Home > Uncategorized > Should NYC’s Test Get a Failing Grade? Probably Not… Should It’s Use as the Sole Metric Get a Failing Grade? Absolutely YES!

Should NYC’s Test Get a Failing Grade? Probably Not… Should It’s Use as the Sole Metric Get a Failing Grade? Absolutely YES!

August 24, 2018

The NYTimes has run several articles on the ongoing debate about the use of the SHSAT as the metric for determining which 8th grade students in the city qualify for entry into the city’s competitive high schools. Today’s paper features an article by James B. Stewart titled “Should These Tests Get a Failing Grade?“, an article that dwells on one clearly ambiguous question that appeared in an online version of the test Mr. Stewart took to see if he would qualify for a competitive school. The fact that one of the questions on the test was flawed was completely unsurprising to me. As noted in earlier posts on standardized testing, the first course I took in graduate school was on testing and the first assignment in that course was to read chapter one of the text and then find five flawed questions in the standardized test whose aggregate scores were used to “measure” the effectiveness of individual schools in the city. While that seemed like a daunting task, it turned out that 20% of the questions on the test had some kind of flaw. It seems that the SHSAT is somewhat better than the test used to measure Philadelphia schools in the early 1970s, but it still has some poorly conceived questions. But, as Daniel Koretz, a testing expert, explains in the article, that’s to be expected:

Daniel Koretz, a professor at the Harvard Graduate School of Education, the author of “Measuring Up” and “The Testing Charade,” and one of the country’s foremost experts on standardized tests, agreed that the question is, at best, ambiguous. “Problematic items do sometimes occur even in good tests, and that is one more reason it is never acceptable to make a consequential decision based on a single test score,” he said.

Which led Mr. Stewart to seek out an explanation from Pearson, the company that develops the test. Here’s what he got:

A spokesman for Pearson, Scott Overland, said the company’s assessment team was examining the revised question in light of the issues I raised. He added that the test undergoes a “rigorous, multi-step development process” and that “the New York City Department of Education is involved in all aspects of the test development process and provides final approval before students take the SHSAT.”

Alrighty then… In the end, Mr. Stewart found himself on the same side of this issue as the mayor:

Mayor Bill de Blasio has caused an uproar by proposing to abolish the SHSAT, and admit students to elite schools based solely on grades and class rank. He said it was “insane” to rely on one taking of a single test. Some elite colleges are now making standardized tests optional.

My experience suggests they may have a point. Test scores, at least, shouldn’t be the sole basis for admission.

Standardized tests DO play a role in assessing an individual’s ability and the overall performance of a school. But they should never be the sole determinant for no matter how rigorous the review process is and how many steps it has there will always be at least one question that is flawed… and when one question can make or break the entry to an elite school or the determination of “success” or “failure” for a school the use of a test as a sole metric is wrong.

Categories: Uncategorized Tags: ,
%d bloggers like this: