In a victory for common sense over spreadsheets, Justice Roger D. McDonough of State Supreme Court in Albany vacated the value-added calculations that were the basis of NY teacher Sheri Lederman’s unfavorable rating “…in part because of the difficulty in measuring growth for students who already perform above grade level on state tests.” The NYTimes article explaining the court decision described the basis for Ms. Lederman’s “unfavorable” rating as follows:
For the 2012-13 school year, Sheri G. Lederman, a longtime teacher in the high-performing Great Neck public school district, on Long Island, received what was known as a growth score of 14 points out of a possible 20; the score was meant to calculate student progress over time. Her students scored substantially higher than the state average on annual standardized tests in English and math, and her score put her in the “effective” range.
The next year, her students scored a bit better on the math test than they had the year before, and slightly lower on the English exam. But her growth score plummeted to one out of 20, or “ineffective.”
This phenomenon results when a standardized test based on a bell curve is used since it is mathematically impossible for high performing students to “grow” on such a test due to the lack of “head room”. If one teacher’s class gets 48 out of 50 correct on a test in the baseline year and another teacher’s class gets 25 out of fifty correct that same year, in the subsequent year it is impossible for the high performing students to get 3 or more questions correct and, thus, impossible for them to show as much “growth” as the low performing students. Despite this inherent flaw, NYS, goaded on by Race-to-the-Top, decided to use “growth” as the primary metric for determining teacher performance. After all, test scores are an “objective” and “quantifiable” means of measuring the effectiveness of teachers. Fortunately for Ms. Lederman, the judge who heard her case— unlike the Board of Regents who adopted the evaluation system— understood basic statistics ad saw the flaw in the methodology.
While the judge limited his decision to only Ms. Lederman, as Carol Burris notes in Valerie Strauss’ Washington Post blog the decision was based on the evidence presented by academics who pointed out flaws in the system, evidence that could undermine the basis for all evaluations based on VAM. Here’s hoping other teachers in other States will follow Ms. Lederman’s lead and challenge their unfavorable ratings based on VAM and help drive a stake through the “testocracy” that drives public education today.
An article by Elizabeth Harris earlier this week drove another nail into the Value Added coffin. The article uses lots of obfuscatory verbiage to paper over the blunt headline, “Over 200 Educators in New York Receive Erroneous Scores Linked to Student Scores”. Using language from a letter sent by the NYSED, Harris writes that the errors in calculations effected “less than 1 percent of the more than 40,000 educators who received such feedback” and to further diminish the impact quoted Dennis Tompkins, a spokesman for the Education Department, who noted that “…that while about 250 principals and teachers received incorrect scores, the error was large enough only to change the growth ratings for 30 educators, all of whom were principals.” The NYSED insinuation seems to be that just because “only” 30 principals got bad scores the system is just fine…. but their actions speak louder than their words:
Nonetheless, (Tompkins) said scores for the more than 40,000 educators would be recalculated at the contractor’s expense; the higher score would be the one that counts.
Sorry, reformers, the recalculation will not restore credibility to VAM….
My favorite blogger, Yves Smith, wrote a post today describing how the oligarchs use their power to indoctrinate the public and advance their own agendas. Using a New York Review of Books article and a blog post by Inside Philanthropy blogger Mike Massey as a springboard for her post, Smith offers several examples of the “cognitive capture initiatives” launched by the philanthro-capitalists. Because she omitted one of the hugest “cognitive capture initiatives” I offered the following comment to her post:
No public enterprise has been “rescued” by seemingly well-intentioned philanthropists more than public education. The philanthropists’ have successfully convinced the public that public schools would be better off if they were subjected to market forces, run like businesses who are answerable to shareholders, and measured by standardized achievement tests that assume the one-size-fits-all industrial model of schooling established in the 1920s is inviolable. Philanthropists have underwritten studies and pilot programs that use the cold analytics of data analysis combined with test scores to impose “value added” measures to reward good teachers. And, as we’ve just witnessed for 7 years, this “run schools like a business” mental model has captured the imaginations of both parties. When you child cannot experience art, music or PE because they need to boost their test scores, send your thank you notes to the philanthropists.
On Wednesday of this week the NYTimes ran an op ed essay by Shael Polakow-Suransky, currently the President of Bank Street School who formerly held the position of chief accountability officer of the New York City Department of Education— the second highest ranking position in the NYC Education Department— under the Bloomberg administration.
I was astonished and pleased to see that Mr. Polakow-Suransky has been disabused of the notion that test scores should play a role in determining teacher effectiveness because in his earlier life as second in command in NYC he sung a different song. In an article profiling him in 2010, the Times wrote:
…if he has his way, there will be better tests, and more of them.
“Until we start seeing assessments that ask kids to write research papers, ask them to solve unfamiliar problems, ask them to defend their ideas, ask them to engage with both fiction and nonfiction texts; until those kinds of assessments are our state assessments, all we’re measuring are basic skills,” Mr. Polakow-Suransky said in an interview.
In his evolution from an idealist teacher to a data-mining administrator, Mr. Polakow-Suransky, 38, personified the seismic changes in education that were beginning to take shape just as he was drawing up his first lesson plans. He came of age as the school system was moving to replace large high schools with small ones, and making testing both a means and an end. He jumped aboard both movements, mentored along much of the way by an educator who, next month, will be working under him.
But Mr. Polakow-Suransky has changed his tune! Now he decries the NYS evaluation system because it “…relies on tests designed for one task (measuring student learning) and uses them for another (measuring each teacher’s impact). Good data is important but we have to use it for what it can actually tell us, not for what we wish it could tell us.”
Now… if he can just persuade the Governor that his evaluation system is relying on flawed data he might fully atone for his advocacy. After all, Diane Ravitch once worked for Lamar Alexander and promoted many of the bad ideas promoted by the business wing of he Republican party before she saw the flaws in the ideas of “running schools like a business”. It is that concept that led to the testing regimen that strangles the creativity of teachers and makes schools more like factories. Maybe Mr. Polakow-Suransky can join her efforts to eliminate the reliance on the seemingly exact standardized test scores as the primary means of evaluating teachers.
Kate Taylor of the NYTimes reported today that the Board of Regents voted to decouple test scores from teacher evaluations, the ultimate repudiation of Duncan/Cuomo’s efforts to lay the blame for poor performance on tests on the doorsteps of teachers. As the Times reported without judgment,
The vote completed a sharp reversal of the state’s policy earlier this year, when the Legislature voted to increase the weight of test scores in evaluations.
The Regents were following a recommendation made last week by a task force created by Gov. Andrew M. Cuomo. It called for revising the state’s standards on what students should know and eliminating state tests when evaluating teachers, at least through the 2018-19 school year.
The task force’s report, which came with Mr. Cuomo’s implicit approval, represented an about face by the governor, a Democrat, who in January had called for test scores to account for half of some teachers’ evaluations.
At that time, he declared that the tests needed to be used for evaluations since students were “failing” them at record highs while 96% of the teachers received solid evaluations. At the time I wrote a post indicating that the 96% figure seemed right to me based on my 29 years experience as an administrator and the notion of using tests to evaluate teacher, especially tests whose cut scores could be manipulated for political purposes, seemed preposterous.
While it is heartening to see a 180 degree turnabout by the governor and the Regents, it is disheartening to see that the Times made no mention of the fact that statisticians have challenged the use of value added measures and gave no indication that that fact played a role in the Regent’s determination to stop using the tests to “measure” teacher effectiveness. Until State Boards and the mainstream media make the public aware of the invalidity of VAM advocates will continue promoting it as the best means of measuring teacher performance…. because it is inexpensive, objective, and seemingly precise.
I am using this blog to release my frustration… I’m frustrated that ESSA was endorsed by education organizations across the board based primarily on the notion that “it could have been worse” and a parallel belief that a 1,000+ document would not include any surprises. I’m frustrated that Senators and legislators voted for it without any pushback on the insistence that standardized testing continue and opting out be forbidden. As noted previously in this blog I’m especially frustrated because the passage of ESSA means that public education will not be a national campaign issue and it also means that high stakes testing will continue until all the teachers who remember a time when it didn’t exist are retired and at least two generations of students will only know schooling that measured their worth based on test scores.
A column by David Kirp in today’s NYTimes failed to make any of those points, instead focusing on the optimistic notion that shifting the responsibility for education policy back to the states will result in a change to the way schools are measured. Kirp notes that the new law eliminated AYP, that states have to provide help to the lowest performing schools, and the RTTT mandate that tests count for a high percentage of school and teacher performance is gone and that states must include “at least one other measure of academic improvement, like graduation rates and, for nonnative speakers, proficiency in English” in their measurement of school performance.
As I’ve written repeatedly, I am not so sanguine about leaving education policy decisions to the states. The original intent of ESEA in 1964 was to provide all children in the nation with an equal opportunity to a good education. Yet since that time 42 states have been sued because their funding formulas are inequitable and very few of them have passed laws to remedy the problem. 22 states refused to accept Medicare, which clearly has an adverse impact on the neediest children. Virtually all the states responded to the 2008 fiscal crisis by cutting funds for public schools.
The notion that the curriculum is being determined by the States gives me no comfort. Some states did not support the Common Core because they were fearful it might require students to learn about evolution. Other state legislatures passed laws forbidding the mention of climate change in their chambers. One State, TX, adopted a text book that referred to slaves as “workers”.
And finally, the idea that States will loosen the test-and-punish regimen is hard to imagine. RTTT did not invent the flawed idea of value added measurement: it came out of TN in the 1990s and took root as a result of NCLB. It’s hard to believe that governors like Walker, Abbott, Brownstein, Rauner, and Katich will abandon high stakes testing any time soon.
Kirp concludes his article with this:
Hope springs eternal in school reform, only to be followed by disappointment. (Announcing his education bill, Lyndon B. Johnson declared his education plan the “passport from poverty.” Clearly, that didn’t work.) Rewriting the standards of evaluation and giving states freer rein in bailing out weak schools, as this law does, is a good day’s work inside the Beltway, but it’s no guarantee that the quality of teaching and learning will change. Making those improvements will take hard work on the part of committed educators and parents. Stay tuned.
I’ll stay tuned… but I fear the programming isn’t going to change… except MAYBE in VT and NH.