Posts Tagged ‘value added’

Lower Hudson Superintendents Expose VAM Sham

October 27, 2014 Leave a comment

Bravo to the Lower Hudson Council of School Superintendents for laying bare NY state’s sham evaluation system. (Full disclosure: I was chairman of this group in the late 1990s, so my support for their bravery might be biased) In a two-page position paper that is more measured than its title, “APPR Creates an Illusion of Teacher Accountability and Must Be Replaced”, the authors recount the findings of an independent study they commissioned. Those findings are summarized in one phrase: “...there is no ability to compare ratings between or among teachers or districts“. The position paper summarized three key findings from their study:

• Teachers whose students did not have to take Common Core exams typically received higher evaluation scores than teachers whose students did take the exams. The result? A double standard for teacher evaluation, and one that is ripe for legal challenge that will be costly to local districts.

• The State Education Department claims that individual local districts are responsible for 80% of the scoring under APPR, a claim that is wildly inaccurate. The Education Analytics study found that because of the APPR formula design, the local impact of scoring is closer to 35% of the total.

• The researchers identified that the required local assessments known as Student Learning Objectives (SLOs) – typically take 5‐10 years of data gathering, development and training before scores can be reliably used as an evaluative measure – are being used for APPR. They were critical of this effort, noting that the absence of training resources and rushed implementation have resulted in an inaccurate evaluation system.

Those findings notwithstanding, the Regents, the State Superintendent, and Governor Cuomo are all standing behind the tests, though Cuomo seemed disappointed that the tests found that 94% of the teacher were effective or highly effective, thereby undercutting his “reform” message that TEACHERS were the cause of low performance on the tests.

The Lower Hudson Council’s speaking out against the testing regimen is especially heartening because many of the districts in that group are among the most affluent in the state. Their children will likely “succeed” on measures like standardized tests. For the most part, these district superintendents have nothing to gain from taking this position except controversy among community members who buy into the notion that testing “proves” schools are bad and that VAM is a viable means of measuring teacher effectiveness… a meme that the media has promoted.

Will politicians and political appointees like the NY Regents— or Arnie Duncan for that matter— listen to school district leaders? Will other regional superintendent groups follow the LHCSS’s lead? Stay tuned!

Obama Could End Testing Today

October 18, 2014 Leave a comment

I was frustrated to read a Christian Science Monitor article titled “As Overtesting Outcry Grows Education Leaders Pull Back on Standardized Tests” for several reasons. The comments and/or questions following quotes from the article will provide some insights into my frustration:

As the outcry against the overtesting of American children has grown, state and local education leaders – in a move endorsed by President Barack Obama– have announced a new focus on dialing back the volume of standardized testing and dialing up the quality.

“I have directed [Education Secretary Arne] Duncan to support states and school districts in the effort to improve assessment of student learning so that parents and teachers have the information they need, that classroom time is used wisely, and assessments are one part of fair evaluation of teachers and accountability for schools,” Mr. Obama said in a statement Wednesday night.

Wait a minute! He’s supporting an improved assessment of student learning that is linked to a “fair evaluation of teachers and accountability for schoolsAs long as teacher and school evaluations are linked with STUDENT test results districts will have a de facto incentive to test students early and often. And hasn’t the President read ANY of the research on VAM? There IS no valid means of linking test scores to teacher performance!

Whether a student faces a large number of tests is not solely determined by federal or state testing mandates, but is largely the product of local district decisions, concludes a report released Thursday by the Center for American Progress.

Wait a minute! As noted frequently in the blog, Race to the Top was a de facto mandate that States adopt the Common Core and also adopt standardized tests that had to be used, to quote the President, “as one part of fan evaluation of teachers and accountability for schools”. While the number of standardized tests administered throughout the year IS a local decision, the administration of a minimum number of high stakes tests is not… and the consequences of administering such tests is described above.

“As states and districts work to clear out unhelpful, unnecessary tests, it would be a grave mistake to stop annual statewide standardized assessments,” noted the Education Trust, a nonprofit working to close achievement gaps for disadvantaged students. “Parents deserve to know how their students are performing … when compared to their peers.”

We know how the comparisons will play out right now: well funded schools serving affluent students will outperform underfunded schools serving children raised in poverty. This isn’t a mystery. It’s been true for at least fifty years. How will MORE assessments help us unless we provide ALL students with the same level of programming and opportunity as the students in the most affluent schools receive?

More than 30 state and urban school leaders endorsed the new statement of principles, which supports Common Core aligned state testing. Among them was John King Jr., the education commissioner in New York. The state recently received a federal waiver to avoid double-testing 8th grade math students, and has offered grants to districts to help reduce nonessential testing.

When John King pushes back on the legislature’s unwillingness to provide equitable funding to public schools and pushes back on Governor Cuomo’s decided favoritism toward deregulated for profit charter schools and disavows VAM, he can be singled out as a school chief pushing back against testing.

Two paragraphs DID hit the nail on the head:

“Hollow pledges to ‘review the entire array of assessments’ are insufficient. In the short run, we need … an elimination of test-based consequences for students, teachers and schools,” said a statement from FairTest, the National Center for Fair & Open Testing.

Randi Weingarten, president of the American Federation of Teachers, applauded the state and local leaders’ effort to reduce testing and ensure high quality, but said in a statement that it “addresses the symptoms, not the root cause, of test fixation…. It’s unconscionable that everything about our schools, our kids and our teachers is reduced to one math and one English high-stakes standardized test per year” under the federal No Child Left Behind law.

Finally, and most importantly, President Obama could end this madness. He ignited the over testing with Race to the Top: he can end it abruptly by eliminating all standardized tests except NAEP, which is minimally disruptive to schools and provides the most statistically significant findings. I hope that at least one candidate running for President will make a pledge to do that… otherwise the factory model will persist and we will continue to sort and select students based on their parents education and income and the wealth disparity will increase.


Columbus Media Discover Poverty’s Impact

September 22, 2014 Leave a comment

Today’s Columbus Dispatch featured an education article by Jim Siegal titled “Data Link Poverty, School Performance in Ohio” that stated what educators have known for decades:

No matter what measure is used — performance index, proficiency scores, ACT scores — the latest results are clear: Poverty rates continue to have a direct, negative link to Ohio student achievement.

To illustrate this reality in graphic form, the paper printed these three charts:


Here’s a question for the politicians in Ohio and every state in the union who claim that we need to use data to inform our decisions: what does this data tell you? Does it suggest that “bad teachers” are the problem? Does it suggest that if teachers were paid for performance of children in poverty would improve? Does it suggest that Teacher tenure is the problem? Does it suggest that teachers unions are the problem? Fortunately some legislators in Ohio are starting to see the light:

“If we are going to address poverty in the state of Ohio, the first thing we need to do is figure out how to start educating these kids,” said Sen. Peggy Lehner, R-Kettering, chairwoman of the Senate Education Committee.

There is just no way around it that we’re going to need to invest money in different ways than what we’ve been doing, because what we’ve been doing isn’t working.

So… how has Ohio been “investing money?” They are one of a handful of states that have supported on-line for-profit charter schools, advocated the removal of tenure for teachers, tried to link teacher pay to test scores, and issued report cards that provide parents with information about the performance of their child’s school. Has any of this worked? And since it hasn’t worked, what might work better?

It’s going to take a significant investment, but I can’t think of a better way to jump-start the economy,” she said. “If we have this many kids coming out of our schools incapable of doing jobs in the 21st century, you need to start there rather than jobs programs. Many of the interventions we look at to start the economy are coming too late.”

The data make an “extremely strong case” for more early-childhood education, Lehner said. Last year, she pushed for $100 million for early education and eventually got $48 million.

ASSUMING Ms. Lehner is serious about making a “significant investment”, early interventions are a far more productive avenue for spending than merit pay based on test scores and a far more productive use of political capital than arguing with unions and teachers about tenure.

For one Ohioan, this was not news at all:

Howard Fleeter, who analyzed the data on behalf of Ohio’s major public-education organizations, knows the correlation between poverty and performance isn’t exactly a breakthrough. The issue has been discussed nationally since at least the 1960s, and his Ohio report-card data back to 2007 show similar correlations each year.

The fact that we’re still looking at a graph in 2014 that shows this pattern is disturbing,” Fleeter said.

Actually, the “fact that we’re still looking at this graph” is evidence that the likelihood of political action is unlikely. It’s too easy to blame teachers for the failure that results from the effects of poverty… and it’s too hard to advocate for higher taxes, especially if those taxes go to the “takers” who are finding it hard to find work and whose children are suffering as a result. Here’s hoping this set of charts, probably the 50th set of such charts, will make a difference.

RTTT’s Overreach

September 22, 2014 Leave a comment

“Washington State Should Not Lose it’s NCLB Waiver”, one of yesterday’s blog posts by Diane Ravitch, included a petition a group of Washington public education leaders intend to send to Arne Duncan appealing for him to  rescind his decision to withdraw WA’s waiver because the State legislature failed to pass a law requiring VAM as part of teacher evaluations. The petition will include a link to a web site that will provide indicators that WA’s schools are not “failing” and includes several salient points on the way the tests are designed.

The petition misses one key point, however: NCLB and especially RTTT are expropriating policy areas that rightfully belong to State and local boards. I am no constitutional lawyer, but I seem to recall that the constitution gives states the authority to determine the kind of schooling their children need. Brown vs. Board of Education, which some might construe as an invasion of “states rights” wasn’t an EDUCATIONAL issue: it was a civil rights issue. NCLB and RTTT are different. When NCLB passed the State and local boards effectively ceded key decisions to the federal government. The Federal government would establish categories for the classification of schools and dictate the use of standardized tests as a key means of classification. RTTT took this a step further and, in doing so, flagged the overreach on the part of USDOE.

I think the The letter writers should have used their petition as an opportunity to explain to Duncan (AND Obama AND Congress) that STATE and LOCAL Boards set policy, not the Federal Government. Hopefully RTTT’s overreach will become evident to voters in ALL states and public education can get the testing genie back in the bottle.

Fun With Mathematics

September 7, 2014 Leave a comment

One of Diane Ravitch’s posts yesterday incorporated a lengthy commentary from Laura Chapman titled “Laura Chapman on Churning the Workforce, VAM, and Magical Thinking“. The post describes the latest work of a team of economists from the Brookings Institute on a means of stack-ranking teachers based on an algorithm that includes test scores, evaluations, and other factors that can be assigned a mathematical value. After reading the post and the comments, I offered the following reaction:

The premise for all of this quantification and standardization is that “bad teachers” are the problem and the causes and conditions that students bring to school are immaterial. Economists are notorious for creating mathematical models are precise and exacting but do not measure what is important.

All of this mathematical manipulation of data brought to mind Dr. Gomberg, a professor I had at Penn who worked as a labor mediator before becoming a teacher. Whenever someone in class would offer a lengthy theoretical discourse he would cut them short by characterizing their comments as “mental masturbation”. I’m sure if he were alive to witness “school reform” he would be offering the same feedback to the economists who dream up these VAM models!

NYS Testing Shenanigans

August 16, 2014 Leave a comment

I’ve read several articles and blog posts about the NYS tests… and all of them underscore the flaws of using standardized tests to measure the effectiveness of schools.

The “Lace To The Top” blog provides the clearest example of the flaws with standardized test scores: the cut score can be modified from year to year to alter the pass rates. As anyone familiar with standardized tests realizes, the scores are not “standard” in a pure mathematical sense. As the blog points out:

Results of the Math tests are up 4.6%, but the cut score was lowered by 3% (3rd grade). In 2013, students needed to receive 44 out of a possible 60 points in order to achieve a passing score of 3. In 2014, students needed to only receive 42 out of a possible 60 points in order to receive a passing grade of 3.

Results of the ELA tests are up 0.1%, but the cut score was lowered by 2% (3rd grade).  In 2013, students needed to receive 35 out of 55 possible points to achieve a passing score of 3. In 2014, students needed to only receive 30 out of a possible 49 points to receive a passing grade of 3.

So while the perception of the public is that these “standards” are objective and mathematically invariant the reality is that they are subjective and fungible… as they must be since the questions vary from year to year. Thus, as any student who passed a course in education statistics knows, standardized tests are NOT precise. The public, however, has a different perception,  a perception that is reinforced when schools are rated against each other based on numeric scales generated by the test developers and promulgated without any caveats by State Departments of Education.

The overall lack of transparency in the NYS tests was the topic of a City Limits blog post by Fred Smith last Monday. The post describes the change in the way test results were shared with teachers and the public. Before Pearson took over the testing in 2011,

SED made complete copies of the annual statewide exams available on its web site shortly after they were given, along with the answer keys. People could ponder the test content and thought processes required of students to answer the questions. Every year, no later than December, a technical report was issued giving specific information about each item.

Now, only 50% of the questions are shared and the technical report is non-existent or released too late for teachers to use the results to inform instruction. This is especially problematic given that the tests will be used to rate and rank teacher performance. The practice underscores the reality that these tests are NOT being used to inform instruction in the classroom or help students but rather being used as an evaluation tool… or more accurately being MIS-used as an evaluation tool. And though Smith does not mention it in his analysis of the test results, I believe Pearson is the culprit here: since they are writing the test they probably are using some kind of “proprietary right” argument to avoid releasing the kinds of broad data SED formerly issued– another adverse by-product of privately contracting for the test development.

Anthony Cody’s Living in Dialogue blog post recounts the sordid history of standardized testing, describing how the tests were originally designed to sort employees into manual labor assignments or office jobs. It emphasizes the explicit link between eugenics and standardized testing but overlooks the more subtle and, in my judgment, more damaging link between Taylorism and testing. The use of tests reinforces the notion that the “output” of education can be objectively measured in the same way that a manufactured product can be measured using, say, a micrometer. As noted previous posts, the standardization paradigm is necessary for those who wish to privatize education and leads to a narrowing of the curriculum because teaching to the test is more efficient than giving children the time to learn a concept deeply.

Alas, the testing shenanigans will continue until the public wakes up to the realization that standardized testing ISN’T exacting and IS undermining their children’s joy of learning.

Categories: Uncategorized Tags: ,

Unregulated Competition’s Progeny

August 3, 2014 Leave a comment

A Diane Ravitch blog post yesterday and a New Yorker article she linked to earlier this week describe examples of the kind of dishonesty that results when “competition” is introduced into schools.

One of her blog posts yesterday describes the ongoing saga of Terrence Carter, who was announced as the incoming superintendent of schools in New London CT and was subsequently found to have lied about his credentials and plagiarized his letter of application. It seems that one of the reasons the New London School Board might have overlooked these details is that MR. Carter (not DR. Carter as he presented himself) had letters of reference from Arne Duncan, CT Governor Malloy, and CT Commissioner Stefan Pryor. Unfortunately for MR. Carter, several activist parents, Jonathan Lender of the Hartford Courant, and a professor from Connecticut College did some legwork. Here’s two excerpts from the Courant’s article:

This is how the pro-privatization, big-philanthropy-funded networks and organizations tend to work. They pass their own people along and up, greasing rails and plumping resumes as they go. And the main criteria for ‘success’ often seems not to be real leadership characteristics, so much as willingness to be a good soldier when it comes to pushing forward a particular reform agenda, said Lauren Anderson, an assistant professor of education at Connecticut College in New London.

…(Carter) filed for bankruptcy twice; his application essay included long passages identical with other educators’ writings on the Internet; a national research organization released a copy of a bio that it says Carter submitted in 2011 with the claim that he had a Ph.D. from Stanford University, which Stanford says he does not; and he got a Ph.D. in 1996 from “Lexington University” — which doesn’t have a campus and had a website offering degrees for several hundred dollars with the motto “Order Now, Graduate Today!”

Diane Ravitch’s concluding paragraph is a chillingly accurate synopsis of the privatization trend in public schools:

The Carter story is not about one man, but about the bipartisan movement to disregard credentials, to close schools, to hire ill-prepared TFA, and to favor privately managed schools over community public schools. To favor democratically elected school boards over management by hedge fund millionaires.

The New Yorker article describes how competition played out in Atlanta where the cheating scandals in the mid 2000s brought down the Superintendent and 178 staff members and resulted in students believing they had achieved academic success when they had, in fact, not improved at all. The most telling paragraph from the entire article is this:

John Ewing, who served as the executive director of the American Mathematical Society for fifteen years, told me that he is perplexed by educators’ ”infatuation with data,” their faith that it is more authoritative than using their own judgment. He explains the problem in terms of Campbell’s law,a principle that describes the risks of using a single indicator to measure complex social phenomena: the greater the value placed on a quantitative measure, like test scores, the more likely it is that the people using it and the process it measures will be corrupted. “The end goal of education isn’t to get students to answer the right number of questions,” he said. “The goal is to have curious and creative students who can function in life.” In a 2011 paper inNotices of the American Mathematical Society, he warned that policymakers were using mathematics “to intimidate—to preëmpt debate about the goals of education and measures of success.”

Mr. Ewing needs to know that educators are not infatuated with use of data to measure school or teacher performance: politicians, economists, and those who want to “run schools like a business” are the ones who believe a “…single indicator can be used to measure complex social phenomenon“… and they believe this because it is a cheap, easy, and fast way to make changes to school.

Both of these articles illustrate what can happen when competition occurs in an unregulated or lightly regulated atmosphere… people cut corners to get ahead. We’ve seen it in professional sports, in banking, and in politics. Competition isn’t a bad thing if the game isn’t rigged and/or the rules are clearly understood and impartially regulated. Unregulated competition will always result in cheating.