I hate to sound like a broken record when it comes to calling out the NYTimes… but I intend to take every chance I get to make the point that VAM is flawed and the Times is complicit in the public’s misunderstanding of that mathematical and statistical fact. Sunday’s column by Nick Kristof, “Are You Smarter than an Eighth Grader” gives me such a chance. The column offers three questions from a recent international test of eighth graders and used them as examples of how poorly our students fared as compared to students in other countries. This led me to offer the following comment:
This paper contributes to the public’s misunderstanding of mathematics and statistics by supporting flawed ideas like “value added” measures as a basis for measuring individual teacher performance despite the rebuke of the methodology by the American Statistical Association and by publishing test data on individual schools without explaining their statistical significance.
I could have made the response more political by noting that the Times reports clearly incorrect and/or incomplete mathematical information when it comes to budget proposals, giving column inches to budget balancing ideas that lack specifics or, in some cases, don’t add up at all. When the “newspaper of record” supports statistical measures that are rebuked by professionals in the field and fails to provide its readers with mathematically accurate facts it is failing the public far more than its schools who need to defend themselves against baseless and inaccurate charges of “failure”.
And then this morning I read a letter to the editor to the Lubbock Avalanche Journal from George McFarland, a local superintendent, pointing out how their media have jumped onto the “failing schools” meme without looking at the facts, which are:
For example, news media like to grab onto quotes that public schools are clearly failing because there are 146,000 students trapped in almost 300 failing public schools. However, considering that 146,000 students is 2.8 percent of the 5,151,925 Texas students, simple math can identify more than 97 percent of Texas public school students are not enrolled in “failing” schools.
Likewise, 300 schools represent 3.5 percent of the 8,574 public school campuses in Texas, meaning 96.5 percent of campuses are not “failing.” These numbers might suggest there are areas where public education can improve but certainly don’t necessitate the need to completely trash an entire system which is serving so many successfully.
Thankfully the newspaper published the article… but if they were doing their job every time a politician said schools were failing they would note that 96.5% are NOT failing… but that FACT undercuts the narrative that is stuck in the minds of readers and voters.
In what could have national ramifications, a post in Politico suggests that the STATE may have the power to withhold FEDERAL funds from districts who fail to participate in the testing.
Here’s the context for this story:
Last year, parents across NYS launched a campaign to opt out of the state tests because they feel that the emphasis on test results is undermining the curriculum in their districts, placing inordinate and inappropriate pressure on their children, and providing them with no information whatsoever about their child’s mastery of the information tested. Teachers unions tacitly supported this movement for the same reasons, emphasizing the flaws in using value added measures for evaluating them and the lack of useful information made available following the testing.
Sensing the growing opposition to the testing regimen, Governor Cuomo included a provision that 50% of the teacher evaluation be used on test results in his budget proposal. When this proviso didn’t fly in the legislature, he accepted a compromise that would allow the Regents to determine the extent to which testing would be the basis for evaluations. As noted in earlier posts, this effectively gave Cuomo a green lift to proceed with VAM since the majority of the Regents and the Regents chair are supporters of the testing regimen Cuomo wants to put in place.
Last year the opt-out campaign was marginal… but this year nearly 200,000 parents have opted out of the testing program, teachers unions have explicitly supported the opt-out movement, and some school boards and superintendents have formally and publicly endorsed the movement. As it became evident that parents were not in support of the movement, Tisch and Cuomo both made claims that they their hands were tied win it came to withholding of federal funds… but as Politico notes that may not be the case:
State officials had previously suggested that the matter was out of their hands. Representatives for the U.S. Department of Education and the state Education Department have said the federal government could withhold Title I funds—grants for schools that serve low-income students—if fewer than 95 percent of students in an individual school or district take the tests, and Governor Andrew Cuomo on Tuesday also said the federal government holds the power to decide whether to withhold funding.
But public statements and regulatory guidance from both the U.S. and state education departments suggest the decision is not totally up to the feds.
“They [federal officials] seem to indicate—I’m hearing that we have discretion, but we will find out how much discretion we have,” state Board of Regents chancellor Merryl Tisch told Capital on Tuesday. “If we do have discretion, we intend to use it.”
Duncan has put Tisch and Cuomo in a bind! Here’s why. Affluent districts with high opt-out percentages and low Title One allocations have less to lose than districts serving children raised in poverty that have high opt-out percentages and high Title One allocations. Thus, if Tisch and Cuomo use the withholding of Federal Title One funds as a penalty they will be hurting children raised in poverty more than those in affluent districts.
And Tisch may have put herself in a bind with her assertion that the Regents intend to use any discretion they have because if they DO assert themselves by withholding the marginal funds from affluent districts they will unleash a massive protest. IF the Regents intend to withhold funds they need to do so quickly because local budgets will be adopted in May and presumably Boards will be advised of their State funding in advance of those budget votes.
Finally, Duncan’s position in NYS may have created a problem for himself: If NY State can withhold federal funds as a penalty, why couldn’t ANY state do the same? And does this ability to withhold funds mean that States have the authority to re-allocate the federal dollars they receive?
It seems to me that a can or worms has just been opened in NYS and conceivably across the country. We may see some interesting fireworks in the coming weeks!
I re-blogged Diane Ravitch’s post on the NYTimes article by Kate Taylor and Mokoto Rich because it perpetuated the myth that unions and not parents were the primary force behind the opt-out movement. This post will provide a blow-by-blow listing of everything that is wrong about their article… and bear with me because the list is a long one!
- Taylor and Rich write: “In Florida, the teachers’ union has lobbied to limit the use of standardized tests, and the governor last week signed a bill thatlimits the number of hours students can spend taking them”, linking the union lobbying to the passage of the law when anyone familiar with FL politics knows that it was the conservative right who compelled Governor Scott to abandon the testing regimen first put in place by presumptive presidential aspirant Jeb Bush.
- They write: “Lawmakers are considering a bill that removes the most punitive consequences for schools and makes clear that states do not have to use test scores to evaluate teachers” as if this will put an end to testing… which, as previous posts have noted is NOT the case. The new bill will require that states develop their own accountability measures and those measures may (and probably will) include annual tests.
- Taylor and Rich quote testing advocates such as Joshua Edelman who is quoted as follows: “It’s right at the point when we finally actually have the kind of improved tests that so many folks petitioned for and advocated for for years,” and while “Mr. Edelman said that the organization supports legislation to reduce unnecessary testing (he felt that) “encouraging parents to opt out is not an effort to reduce over testing… “It’s an effort to undermine accountability”. This is wrong on two scores: first the tests being used now are NOT the kind of “improved tests so many folks petitioned for” because their results cannot be used by teachers to help individual students improve their knowledge and understanding nor can they prescribe the kind of instruction each child needs. And parents and unions are not opposed to “accountability”, they are opposed to the kind of accountability model imposed on schools by politicians who want to privatize public education.
- The article gives several inches of space to those who oppose the campaign against testing, and Taylor and Rich then write: “The union argued that it was not fair to make test scores so big a part of a teacher’s rating because many factors outside the classroom can influence scores.” That statement is accurate but incomplete: it omits the fact that the unions argue the use of test scores is unfair because there is no statistical validity to their use for evaluating teachers and even the use of the tests to evaluate schools is debatable.
In the article there is no mention whatsoever of the statistical science community’s virtually unanimous agreement that “value added” measures are impossible… which is akin to having an article denying climate change published based on the findings of a scientist on the payroll of the petroleum lobby. By turning a blind eye to the statistical worthlessness of value added measures the Times is effectively telling the public that test scores can be a valid measure teacher performance… which is clearly untrue. The Times is doing a disservice to students, parents, taxpayers, and– yes— teachers.
Today’s blog post title is a play on yesterday’s NYTimes featured an op ed article by Will Miller titled “Want Reform? Principals Matter, Too”. In the article, Miller, who is president of the Wallace Foundation, breathlessly reports that the Principal of a school plays a key role in school improvement… a fact that true school reformers like Ron Edmunds knew decades ago. Miller’s op ed piece recounts all of the reasons this is the fact, touches on some of the research that demonstrates this, and offers some recommendations on how this can be addressed.
One point Mr. Miller overlooked was the impact of VAM on school administrators, especially in New York. The latest thinking on “reform” in NY insists that test scores take greater precedence than principal evaluations. Indeed, Governor Cuomo has so little regard for Principals’ ability to evaluate that he wants to institute a system that requires independent third party evaluations. Why? Because the failure rate for teachers is way too low! To paraphrase Mr. Miller, it’s hard to think of another profession where so little attention is paid to leaders. Organizations like the military, corporations and universities listen to and respect their leaders. If we’re going to do this in public education, a lot has to change… beginning with abandoning the notion that test results can replace direct observation in the classroom as a means of judging teacher and administrator performance.
The fundamental principle that test scores cannot measure the human interactions between a teacher and a student and a leader and a subordinate needs to be brought to the forefront…. because THAT principle matters A LOT more than any Principal.
The NYTimes, reputed to be the national newspaper of record, seems to be missing something in it’s reporting of the opt-out movement. Based on it’s article on Monday, “Some Parents Oppose Standardized Testing on Principle, But Not n Practice”, a reader would come away with the impression that the opt out movement is thin on results. contrast this with the New York Daily News’ coverage of the same phenomenon titled “Fed-Up Parents Revolt Against State’s Standardized Tests“. There IS a another difference between the two articles: the New York Daily News’ article has actual statistics that support it’s contention. Statistics like these:
More than half the pupils at several Long Island and upstate school districts joined in — at some schools in New York City boycott percentages neared 40%.
At the Patchogue-Medford School District in Suffolk County, 65% of 3,400 students in grades three to eight abstained from the test, District Superintendent Michael Hynes told the Daily News.
At West Seneca District near Buffalo, nearly 70% of some 2,976 students refused testing. Likewise, at tiny Southold School District on Long Island’s North Fork, 60% of the 400 students opted out; so did 60% of Rockville Centre’s 1,600 pupils. And in the Westchester town of Ossining, nearly 20% of 2,100 students boycotted.
Here in the city, a Department of Education spokeswoman claimed the number of opt-outs won’t be known for weeks. But there’s little doubt the boycott totals in city schools will dwarf last year’s numbers, when fewer than 2,000 pupils abstained…
At Central Park East 1, a K-to-5 school in East Harlem, 59 of 76 children refused the test, according to Toni Smith-Thompson, co-president of the Parents Association and a leader of the boycott…
At Public School 29 in Cobble Hill, Brooklyn, some 20% refused. And at Public School 321 in Park Slope, Brooklyn, 36% boycotted, local parent leaders said.
Hm-m-m-m…. could it be the NYTimes is avoiding coverage of the unravelling of the reform movement it has championed on its editorial pages? It will be interesting to see their coverage of this in the days ahead.
I was heartened to read two articles describing a pilot program underway in four New Hampshire School districts. This paragraph from Julia Freedland’s article in the Clayton Christensen Institute for Disruptive Learning weekly newsletter provides a synopsis of the program:
New Hampshire’s Performance Assessment for Competency Education (PACE) pilot will allow locally managed assessments to count toward federal accountability requirements. New Hampshire’s PACE project began in 2012 as an opt-in effort for districts to coordinate local approaches to performance assessment. Starting this year, the four PACE implementing districts—Sanborn Regional, Rochester, Epping, and Souhegan—will administer the Smarter Balanced assessment once in elementary school, once in middle school, and once in high school (in three grades instead of seven). In all other years when students aren’t taking Smarter Balanced assessments, the PACE districts will administer carefully designed common and locally managed “performance assessments” that were developed by the districts themselves and validated at the state level.
Some background: in 2005 the New Hampshire State Board of Education adopted a policy that eliminated seat time as the primary basis for awarding high school credit. This opened the door for school districts to put competency based programs in place at the high school level, a door that few districts walked through during the five years that followed… and a door that many school boards were reluctant to open at all. In the high achieving New Hampshire School District district I led from 2004-11 we already had independent study courses in place that awarded competency-based credit. These courses were available to students who excelled in certain areas and wanted to pursue independent research projects in that area and, on a limited basis, to students who needed a different format for learning. Expanding this concept to all courses was difficult for two reasons: it required more marginally higher staffing than the traditional model and it required a change from the traditional method of grading and grouping of students— a change that could result in lower SAT scores and/or confusion in admissions offices of elite colleges.
I believe the cost differentials could be minimized if not eliminated altogether if a school system adopted competency-based learning across the board and could prepare spreadsheets to illustrate this. But I am not so certain that politicians, school boards, parents, and traditional colleges will be easily persuaded that the abandonment of the Carnegie Unit is feasible. New Hampshire’s experience illustrates how difficult a change like this is. Former State Superintendent Nick Donohue succeeded in getting the State Board to adopt his proposal to abandon “seat time” and a decade later four of the 90+ districts in the State are experimenting with an alternative to standardized testing that is based on the assumptions implicit in that action… and doing so on an experimental basis. How much longer will it take for other districts to join in? This paragraph from Stacy Khadaroo’s Christian Science Monitor article suggests it will take some time:
New Hampshire hopes to slowly scale up the experiment. There are other districts waiting in the wings to switch to the performance-based assessments as early as next year, (Sanborn HS Principal) Mr. Marion says. He’s also aware of a handful of states eager to move in this direction, though he wonders if they might get “skittish” when they find out how much work it’s taken to develop and vet the new assessments.
The combination of gridlocked politicians, sluggish school boards, tradition-bound teachers and parents, and timid leaders will make change a daunting a challenge. The only hope is that the weariness over standardized testing will accelerate the changes students need.
Frank Bruni’s column, “Best, Brightest… and Saddest“, describes a rash of teen suicides in Palo Alto, CA where the pressure to succeed in school is creating mental health problems. Having led a school district with the same demographics as Palo Alto, I am all too familiar with the kinds of problems Mr. Bruni describes in his column. But having read Mr. Bruni’s critiques of public schooling, I feel that he overlooks the root cause of the competitive environment in schools, which is the factory model.
As long as we measure student success and school success based on test scores we will impose stress on students. How can we claim to value the well-being of each child when we promote a competitive grading system that sorts and selects based on comparisons with age cohorts within a school. We have the ability to provide self-paced individualized learning to each student yet we insist on continuing the practice of grouping children in age cohorts and ranking them based on how quickly they learn as compared to their peer group. Our outmoded method of schooling and measurement is creating the pressure that is neither productive for our economy or healthy for our children.