Bravo to the Lower Hudson Council of School Superintendents for laying bare NY state’s sham evaluation system. (Full disclosure: I was chairman of this group in the late 1990s, so my support for their bravery might be biased) In a two-page position paper that is more measured than its title, “APPR Creates an Illusion of Teacher Accountability and Must Be Replaced”, the authors recount the findings of an independent study they commissioned. Those findings are summarized in one phrase: “...there is no ability to compare ratings between or among teachers or districts“. The position paper summarized three key findings from their study:
• Teachers whose students did not have to take Common Core exams typically received higher evaluation scores than teachers whose students did take the exams. The result? A double standard for teacher evaluation, and one that is ripe for legal challenge that will be costly to local districts.
• The State Education Department claims that individual local districts are responsible for 80% of the scoring under APPR, a claim that is wildly inaccurate. The Education Analytics study found that because of the APPR formula design, the local impact of scoring is closer to 35% of the total.
• The researchers identified that the required local assessments known as Student Learning Objectives (SLOs) – typically take 5‐10 years of data gathering, development and training before scores can be reliably used as an evaluative measure – are being used for APPR. They were critical of this effort, noting that the absence of training resources and rushed implementation have resulted in an inaccurate evaluation system.
Those findings notwithstanding, the Regents, the State Superintendent, and Governor Cuomo are all standing behind the tests, though Cuomo seemed disappointed that the tests found that 94% of the teacher were effective or highly effective, thereby undercutting his “reform” message that TEACHERS were the cause of low performance on the tests.
The Lower Hudson Council’s speaking out against the testing regimen is especially heartening because many of the districts in that group are among the most affluent in the state. Their children will likely “succeed” on measures like standardized tests. For the most part, these district superintendents have nothing to gain from taking this position except controversy among community members who buy into the notion that testing “proves” schools are bad and that VAM is a viable means of measuring teacher effectiveness… a meme that the media has promoted.
Will politicians and political appointees like the NY Regents— or Arnie Duncan for that matter— listen to school district leaders? Will other regional superintendent groups follow the LHCSS’s lead? Stay tuned!
“Washington State Should Not Lose it’s NCLB Waiver”, one of yesterday’s blog posts by Diane Ravitch, included a petition a group of Washington public education leaders intend to send to Arne Duncan appealing for him to rescind his decision to withdraw WA’s waiver because the State legislature failed to pass a law requiring VAM as part of teacher evaluations. The petition will include a link to a web site that will provide indicators that WA’s schools are not “failing” and includes several salient points on the way the tests are designed.
The petition misses one key point, however: NCLB and especially RTTT are expropriating policy areas that rightfully belong to State and local boards. I am no constitutional lawyer, but I seem to recall that the constitution gives states the authority to determine the kind of schooling their children need. Brown vs. Board of Education, which some might construe as an invasion of “states rights” wasn’t an EDUCATIONAL issue: it was a civil rights issue. NCLB and RTTT are different. When NCLB passed the State and local boards effectively ceded key decisions to the federal government. The Federal government would establish categories for the classification of schools and dictate the use of standardized tests as a key means of classification. RTTT took this a step further and, in doing so, flagged the overreach on the part of USDOE.
I think the The letter writers should have used their petition as an opportunity to explain to Duncan (AND Obama AND Congress) that STATE and LOCAL Boards set policy, not the Federal Government. Hopefully RTTT’s overreach will become evident to voters in ALL states and public education can get the testing genie back in the bottle.
One of Diane Ravitch’s posts yesterday incorporated a lengthy commentary from Laura Chapman titled “Laura Chapman on Churning the Workforce, VAM, and Magical Thinking“. The post describes the latest work of a team of economists from the Brookings Institute on a means of stack-ranking teachers based on an algorithm that includes test scores, evaluations, and other factors that can be assigned a mathematical value. After reading the post and the comments, I offered the following reaction:
The premise for all of this quantification and standardization is that “bad teachers” are the problem and the causes and conditions that students bring to school are immaterial. Economists are notorious for creating mathematical models are precise and exacting but do not measure what is important.
All of this mathematical manipulation of data brought to mind Dr. Gomberg, a professor I had at Penn who worked as a labor mediator before becoming a teacher. Whenever someone in class would offer a lengthy theoretical discourse he would cut them short by characterizing their comments as “mental masturbation”. I’m sure if he were alive to witness “school reform” he would be offering the same feedback to the economists who dream up these VAM models!
I’ve read several articles and blog posts about the NYS tests… and all of them underscore the flaws of using standardized tests to measure the effectiveness of schools.
The “Lace To The Top” blog provides the clearest example of the flaws with standardized test scores: the cut score can be modified from year to year to alter the pass rates. As anyone familiar with standardized tests realizes, the scores are not “standard” in a pure mathematical sense. As the blog points out:
Results of the Math tests are up 4.6%, but the cut score was lowered by 3% (3rd grade). In 2013, students needed to receive 44 out of a possible 60 points in order to achieve a passing score of 3. In 2014, students needed to only receive 42 out of a possible 60 points in order to receive a passing grade of 3.
Results of the ELA tests are up 0.1%, but the cut score was lowered by 2% (3rd grade). In 2013, students needed to receive 35 out of 55 possible points to achieve a passing score of 3. In 2014, students needed to only receive 30 out of a possible 49 points to receive a passing grade of 3.
So while the perception of the public is that these “standards” are objective and mathematically invariant the reality is that they are subjective and fungible… as they must be since the questions vary from year to year. Thus, as any student who passed a course in education statistics knows, standardized tests are NOT precise. The public, however, has a different perception, a perception that is reinforced when schools are rated against each other based on numeric scales generated by the test developers and promulgated without any caveats by State Departments of Education.
The overall lack of transparency in the NYS tests was the topic of a City Limits blog post by Fred Smith last Monday. The post describes the change in the way test results were shared with teachers and the public. Before Pearson took over the testing in 2011,
SED made complete copies of the annual statewide exams available on its web site shortly after they were given, along with the answer keys. People could ponder the test content and thought processes required of students to answer the questions. Every year, no later than December, a technical report was issued giving specific information about each item.
Now, only 50% of the questions are shared and the technical report is non-existent or released too late for teachers to use the results to inform instruction. This is especially problematic given that the tests will be used to rate and rank teacher performance. The practice underscores the reality that these tests are NOT being used to inform instruction in the classroom or help students but rather being used as an evaluation tool… or more accurately being MIS-used as an evaluation tool. And though Smith does not mention it in his analysis of the test results, I believe Pearson is the culprit here: since they are writing the test they probably are using some kind of “proprietary right” argument to avoid releasing the kinds of broad data SED formerly issued– another adverse by-product of privately contracting for the test development.
Anthony Cody’s Living in Dialogue blog post recounts the sordid history of standardized testing, describing how the tests were originally designed to sort employees into manual labor assignments or office jobs. It emphasizes the explicit link between eugenics and standardized testing but overlooks the more subtle and, in my judgment, more damaging link between Taylorism and testing. The use of tests reinforces the notion that the “output” of education can be objectively measured in the same way that a manufactured product can be measured using, say, a micrometer. As noted previous posts, the standardization paradigm is necessary for those who wish to privatize education and leads to a narrowing of the curriculum because teaching to the test is more efficient than giving children the time to learn a concept deeply.
Alas, the testing shenanigans will continue until the public wakes up to the realization that standardized testing ISN’T exacting and IS undermining their children’s joy of learning.
A Diane Ravitch blog post yesterday and a New Yorker article she linked to earlier this week describe examples of the kind of dishonesty that results when “competition” is introduced into schools.
One of her blog posts yesterday describes the ongoing saga of Terrence Carter, who was announced as the incoming superintendent of schools in New London CT and was subsequently found to have lied about his credentials and plagiarized his letter of application. It seems that one of the reasons the New London School Board might have overlooked these details is that MR. Carter (not DR. Carter as he presented himself) had letters of reference from Arne Duncan, CT Governor Malloy, and CT Commissioner Stefan Pryor. Unfortunately for MR. Carter, several activist parents, Jonathan Lender of the Hartford Courant, and a professor from Connecticut College did some legwork. Here’s two excerpts from the Courant’s article:
This is how the pro-privatization, big-philanthropy-funded networks and organizations tend to work. They pass their own people along and up, greasing rails and plumping resumes as they go. And the main criteria for ‘success’ often seems not to be real leadership characteristics, so much as willingness to be a good soldier when it comes to pushing forward a particular reform agenda, said Lauren Anderson, an assistant professor of education at Connecticut College in New London.
…(Carter) filed for bankruptcy twice; his application essay included long passages identical with other educators’ writings on the Internet; a national research organization released a copy of a bio that it says Carter submitted in 2011 with the claim that he had a Ph.D. from Stanford University, which Stanford says he does not; and he got a Ph.D. in 1996 from “Lexington University” — which doesn’t have a campus and had a website offering degrees for several hundred dollars with the motto “Order Now, Graduate Today!”
Diane Ravitch’s concluding paragraph is a chillingly accurate synopsis of the privatization trend in public schools:
The Carter story is not about one man, but about the bipartisan movement to disregard credentials, to close schools, to hire ill-prepared TFA, and to favor privately managed schools over community public schools. To favor democratically elected school boards over management by hedge fund millionaires.
The New Yorker article describes how competition played out in Atlanta where the cheating scandals in the mid 2000s brought down the Superintendent and 178 staff members and resulted in students believing they had achieved academic success when they had, in fact, not improved at all. The most telling paragraph from the entire article is this:
John Ewing, who served as the executive director of the American Mathematical Society for fifteen years, told me that he is perplexed by educators’ ”infatuation with data,” their faith that it is more authoritative than using their own judgment. He explains the problem in terms of Campbell’s law,a principle that describes the risks of using a single indicator to measure complex social phenomena: the greater the value placed on a quantitative measure, like test scores, the more likely it is that the people using it and the process it measures will be corrupted. “The end goal of education isn’t to get students to answer the right number of questions,” he said. “The goal is to have curious and creative students who can function in life.” In a 2011 paper inNotices of the American Mathematical Society, he warned that policymakers were using mathematics “to intimidate—to preëmpt debate about the goals of education and measures of success.”
Mr. Ewing needs to know that educators are not infatuated with use of data to measure school or teacher performance: politicians, economists, and those who want to “run schools like a business” are the ones who believe a “…single indicator can be used to measure complex social phenomenon“… and they believe this because it is a cheap, easy, and fast way to make changes to school.
Both of these articles illustrate what can happen when competition occurs in an unregulated or lightly regulated atmosphere… people cut corners to get ahead. We’ve seen it in professional sports, in banking, and in politics. Competition isn’t a bad thing if the game isn’t rigged and/or the rules are clearly understood and impartially regulated. Unregulated competition will always result in cheating.