Archive

Posts Tagged ‘value added’

Reformers’ Worries About Standardized Tests Too Little and Too Late

November 17, 2018 Comments off

Two days ago Chalkbeat’s Matt Barnum posted an article describing the epiphany of many reformers regarding standardized tests titled “In a shift, more education reformers say they’re worried about schools’ focus on testing“.

The epiphany is summarized in the opening paragraphs:

“If there is one office in every state I would want to get rid of, it’s the accountability office,” said Andre Perry, a fellow at the Brookings Institution who previously led a charter school in New Orleans. “I would replace that office with some kind of statewide coordination around personalized learning.” No one on the panel with him disagreed.

I think too much time, attention, and resources have been devoted to accountability systems that don’t produce outcomes for students that historically struggled,” Lewis Ferebee, the head of Indianapolis Public Schools, said later.

“The way we’re doing [assessment] now — that is so time-, age-, grade-based — is really constraining for those innovators that are developing models that will support all kids,” said Susan Patrick of iNACOL, an organization that promotes technology-based personalized learning in schools.

To no educators surprise, once No Child Left Behind mandated the use of standardized tests to determine whether a school was succeeding or failing and then tied those test results to the compensation of teachers and made the continued operation of the schools contingent on performing well on the tests, MOST of the teachers time, attention and resources were devoted to passing those tests. And to no educators’ surprise the students who struggled the most to pass those tests were children who came from homes where education was not as important as, say, figuring out where the family would sleep or where their next meal would come from.

And once standardized tests became the basis for judging schools, it became evident to reformers and politicians who were claiming these tests would “prepare students to enter the workforce” that it was necessary to ensure that the tests in every state were based on the same set of skills… which opened the door to the Common Core.

By the time the Obama administration had every state engaged in a Race to the Top, standardized tests were entrenched in the DNA of every school system in the nation and their importance was magnified.

So, nearly two decades later, the “reformers” who wanted a cheap, simple, and fast way to measure “school effectiveness” and “student success” have come to the conclusion that standardized tests, while cheap, simple, and fast, do NOT measure the effectiveness of schools or do an adequate job of measuring individual student learning.

But the tests cannot be abandoned as quickly as they were imposed… because there is not a quick, cheap and easy metric to take their place when it comes to measuring schools or individual student performance… and if it is impossible to do so how can a school be given a low grade and recommended for takeover by a private for profit charter chain? And how can a parent make an informed choice about the school they want to attend.

Mr. Barnum’s article concludes with this offer from a group that has long opposed standardized testing:

“I’m happy to hear that these groups are in fact grappling with and realizing some of the same problems we are,” said Andre Green, the executive director of FairTest, a group that pushes for a smaller role for testing. “Come talk to us.”

I doubt that FiarTest’s phone is ringing off the hook… and that’s too bad because they might have some insights based on what makes sense to teachers and administrators.

Advertisements

State Mandated Rating Systems Reinforce Commodification… Multiple Metrics Determined by Local Boards the Best Accountability

November 3, 2018 Comments off

Diane Ravitch’s posted blogger Laura Chapman’s critique of Ohio’s rating system for schools that, in response to ESSA mandates, now incorporates “multiple measures”. But the results of changing to multiple measures made no difference in terms of what the report cards actually measure: family income and education.

To my surprise, neither Diane Ravitch nor Laura Chapman nor any of the commenters noted the interest flaw of rating systems: the fact that they reinforce the commodification of public schools. Ratings were introduced by “reformers” based on the premise that parents can shop for schools the way they shop for cars or groceries. Until affluent school districts with space in their classrooms open their doors to children raised in poverty in neighboring districts these ratings should be ignored.

BTW, one of the most bogus “selling points” of ESSA was the notion that STATES would be in complete control of the mandated rating system, a system that would ultimately need to be approved by the USDOE. If you think any state will abandon standardized testing as the primary metric there is a bridge I can sell you…. And worse yet, in the case of Ohio, one of the”multiple measures” is the introduction of a value added metric… despite the fact that VAM has been repeatedly discredited.

NYTimes Reporting on Tests Gets a “D”… the Lowest Passing Grade Possible

September 28, 2018 Comments off

NYTimes reporter Eliza Shapiro’s article on the release of last year’s test results gets a passing grade because it is factually accurate, but it gets a “D” because of what it fails to report. It factually recounts the reasons why last year’s results cannot be compared to previous year’s tests. It also factually reports on the opt out movement data and accurately contrasts the Mayor Bloomberg’s overemphasis on test results as compared to his successor, Bill de Blasio. But here are the points Ms. Shapiro missed:

  • Ms. Shapiro writes: “Just five years ago, it seemed the state was poised to have some of the toughest teacher evaluations in the nation, based on some of the most intensive exams.” The “tough evaluations” Mr. Bloomberg championed were based on VAM, a statistically invalid and discredited method of evaluation– not discredited by teachers unions, but by statisticians!
  •  Ms. Shapiro writes: “This year’s scores are the latest confusing data points in a long history of zigzagging test results in New York, but what they do tell us is how much the political pendulum has swung on standardized testing.” One VERY possible reason for the “zig-zag” in test scores, as any education writer should realize, is the fact that the definition of a “passing score” is not based on how students perform compared to a fixed set of standards: it is based on a cut score that can be set arbitrarily by those who designed the test… and, surprise… cut scores have proved to be a function of political variances and not a function of teaching variances. That is, political forces swing the pendulum on test scores and parents, teachers, and many voters are beginning to realize that reality. Ms. Shapiro’s article makes no mention of this, leading an innocent reader to believe that performance, not the cut score, zig-zagged. This only adds to the public’s confusion about the data points.
  • Ms. Shapiro oversells the differences in test scores… even after she wrote in one of the opening paragraphs that “…the (State education) department cautioned that the exams cannot be measured against previous tests and should be considered a new baseline.” What Ms. Shapiro fails to realize is that if they can’t be measured against a previous baseline, they can’t be analyzed by comparing how one group of schools did as opposed to another set. Despite this reality, Ms. Shapiro compares the performance of charter schools to regular public schools and regular charter schools to “Renewal schools” and draws a long series of faulty conclusions as a result.
  • Ms. Shapiro downplays the bottom line: all of the “reforms” in NY have made no difference whatsoever. She writes: “Although state exam results have fluctuated wildly in the past 10 years, the National Assessment of Educational Progress, considered the gold-standard measure of academic growth, found that New York City’s students did not make progress in English or math between 2015 and 2017.” 
  • And last, but not least, Ms. Shapiro concludes her article with this: “This year’s results will no doubt influence fresh questions about education policy in New York, including the state Board of Regents’ decision about whether to use exam results in teacher evaluations.

So Ms. Shapiro got the facts right… but she got the conclusions all wrong… and the public, as a result, is getting the message that reforms advocated by the billionaires are doing well (i.e. charters “outperform” public schools) but the reforms advocated by progressives like the Mayor are not making a difference (i.e. the renewal school scores are flat) and that VAM is still a viable means of evaluating teachers. Ugh!

Arne Duncan Still True Believer in VAM, “Failure” of Public Schools, Standardized Testing

August 9, 2018 Comments off

Arne Duncan has written a new book, How Schools Work: An Inside Account of Failure and Success From One of the Nation’s Longest-Serving Secretaries of Education, and he is getting lots of publicity as he tours the country selling his book and the tired ideas in it. Here’s the opening paragraph from a review of his book by Atlantic reporter Alia Wong:

Arne Duncan, the former education secretary under President Barack Obama, has always been more candid than others who’ve served in that role. He’s often used his platform to talk about what he sees as the persistent socioeconomic and racial disparities in access to quality schools. His new book, How Schools Work: An Inside Account of Failure and Success From One of the Nation’s Longest-Serving Secretaries of Education, further cements that reputation. How Schools Work’s first chapter is titled “Lies, Lies Everywhere.” The first sentence: “Education runs on lies.” If one were to create a word cloud of the book, lies would probably pop out as one of the most frequently used words. Duncan writes that even the countless fantastic schools across the country “haven’t managed to defeat the lies that undermine our system so much as they’ve been able to circumvent them.” These lies, according to Duncan, include a culture of setting low expectations for high schoolers who later discover they’re not prepared for the real world, and poorly designed accountability systems that allow teachers to fudge their students’ test-score results.

This paragraph itself is full of canards about public education that only someone who never set foot in a public school could believe. I worked in an urban middle school, a blue collar suburban high school, and a rural high school that served many poor families. The teachers in these schools, even the weakest, had high expectations for their children.

As for the “accountability systems that allow teachers to fudge their test-score results”, I presume he must be referring to the grading systems that allow students to pass a course with a “C” or a “D”, grades that typically require a student to get grades that do not require mastery of ALL the information presented. And the norm-referenced tests that were the backbone of the RTTT “accountability systems” Mr. Duncan imposed on schools that were presumably designed to avoid the “fudging” did nothing to help students. They only reinforced the notion that students were poorly prepared because teachers were lazy and incompetent and did so by providing a sheen of precision.

In the interview with Ms. Wong that accompanied this overview of his book Mr. Duncan DID reveal an understanding of the root cause of “failing” schools… and it isn’t the teachers… it’s parents who are disengaged from the lives of their students, parent’s whose disengagement is often the result of working multiple jobs or, in the worst case, drug and alcohol abuse. Here’s Mr. Duncan’s take:

It’s the parents who aren’t present whose kids you have to worry about even more because those parents just have too much going on in their own lives to be engaged in their children’s education. Those kids are the ones I actually worry about the most.

But, as written frequently in this blog, actions speak louder than words. IF Mr. Duncan believed this as the head of public education in Chicago and then the nation, why did he not take action to provide support for the children of disengaged parents, the children whose performance pulls down the test scores he values so highly and whose ultimate withdrawal from schools increases the drop out rates he blames on “the system”?

Mr. Duncan’s perspective on gun violence was also on point. But like his views on the problems presented by disengaged parents, it’s a perspective he failed to share when he led the nation’s schools:

I talk a lot about gun violence—it’s what I’m dealing with in Chicago all the time; it unfortunately shaped me as a kid; we saw it in the Sandy Hook massacre, which happened when I was education secretary. There’s no political leader who says they don’t value kids, but the truth is: we value guns more than we value the lives of our children .And that is irrefutable if you look at the rates of gun deaths in the U.S. compared to other nations that make other policy choices.

Mr. Duncan purports to be one who perceives education as a great equalizer and one who attempts to use data to help him see what works and what doesn’t work. I wish that as Secretary of Education emeritus he would take a dispassion look at the true impact of RTTT and acknowledge that it was a doubling down on NCLB, a program he viewed as “horribly constructed.” I wish he would acknowledge that the standardized tests he advocated were not constructed to perform the VAM he mandated and resulted in the discrediting of the teaching profession. I wish that he would trumpet the need for programs to support parents who “…just have too much going on in their own lives to be engaged in their children’s education” and speak out against the politicians who value guns more than we value the lives of our children. Finally, I wish he would acknowledge that the programs he advocates, the expansion of choice and charters, reward those parents who are engaged in the lives of their children, sidestep the need for a larger investment in the safety net, and divert needed funds away from public schools.

Tennessee’s Faith in Testing is Based on the Flawed Premise that More Money is Unnecessary

August 2, 2018 Comments off

Diane Ravitch wrote a post yesterday decrying the “colossal failure” of the school district comprised of the “failing schools” taken over by the state, a district that received millions of dollars from Race to the Top to replace “failing” public schools with charters. There’s only one problem, as noted in the Chalkbeat article that was the basis for Ms. Ravitch’s post: low-performing schools operated by Shelby County Schools, where most of the “failing schools” taken over by the state are located, have outpaced progress of those run by the state! 

The agreeable fantasy that a “state takeover” of “failing schools” or the outsourcing of those same schools to deregulated charter schools would lead to their “improvement” underpins virtually all federal and state legislation. It also underpins the reform movement and leads to other agreeable fantasies promoted by reformers. These agreeable fantasies enable politicians to dodge the need for legislation that would either require them to raise more money for public education or divert the money already allocated to less affluent districts. It also enabled them to adopt other ideas based on magical thinking, ideas that don’t require more money but result in “improvement”. Ideas like: firing “bad teachers” would improve “failing schools”; or, “adopting a uniform curriculum” would improve the teaching and learning in “failing schools”; or “eliminating frills” would direct more resources “to the classroom”; or implementing merit pay plans that would reward the “best and brightest teachers” and withhold raises from “weak teachers” thereby “improving failing schools”; or implementing programs that “increase the grit” of students raised in poverty to help them overcome the adversity they face. None of these ideas require more money and all of them directly or indirectly scapegoat the teachers who work tirelessly to improve their “failing” schools.

But the biggest agreeable fantasy is that statewide standardized norm referenced tests are the best means of measuring the “quality of education”… and, as we are witnessing, the state that gave us value added testing has tremendous faith in that fantasy… and. alas, so do most voters across the country despite the accumulating evidence to the contrary.

Atlantic Article Advises Against Aggregating Nation’s Schools

July 31, 2018 Comments off

Earlier this month, Atlantic writer Jack Schneider wrote an article urging our nation to stop thinking of “America’s Schools” as a monolith, arguing that in doing so we are doing more harm than good. At the outset of his essay. Mr. Schneider identified A Nation at Risk as the the point when we established “…a new way of talking about public education in the United States, a master narrative that has endured — and even subtly changed American education policy for the worse — over the past several decades.” That “master narrative” is that public education is a monolith and that it is “failing”. But Mr. Schneider asserts that this is not the case at all:

The abstraction of “America’s schools” may be convenient for rousing the collective conscience, but it is not particularly useful for the purpose of understanding (or improving) American education. Consider the issue of funding. On average, federal money accounts for less than 10 percent of education budgets across the country, and the rest of the financial responsibility falls to states and local schools. If local schools are unable to raise what they need, the state is usually well positioned to make up the difference, but states differ dramatically in their approaches. On average, states spend roughly $13,000 per student on public education — but looking at the average alone is misleading. Only about half of states spend anything close to that figure: A dozen spend 25 percent more than the national average, and 10 states spend 25 percent less. The result is significant disparities, and some striking incongruities. New York’s schools, for instance, spend roughly three times as much per student as Utah’s schools — a huge difference, even after accounting for New York’s higher cost of living.

And once the “collective conscience” of the politicians was roused by A Nation at Risk, both political parties bought into the “failing schools” narrative and began imposing one-size-fits-all solutions to the monolith, ignoring the reality that funding was hugely disparate and the policies governing schools resided in state and local governments. But Mr. Schneider does see a value in looking at public education as a national issue.

This is not to say that taking the national perspective can’t be valuable. Troubling patterns do exist across the U.S., and discussions about them can play an important role in shaping both public understanding and education policy. Achievement gaps across race and class, for instance, are an important reminder of broader social and economic inequalities, and advocates have used evidence about those patterns to make the case for universal early-childhood education. Similarly, a national dialogue about the disproportionate punishment of black and brown children in schools has drawn attention to an issue that might otherwise have gone unnoticed. These kinds of broad conversations can generate both political will and policy responses.

But, alas, while these kinds of broad conversations CAN generate political will and policy responses, they have not done so thus far. Instead, they’ve generated policy responses like NCLB, Race to the Top, and ESSA, all of which use standardized testing to reinforce the notion that public education is a monolith and it is failing. Mr. Schneider concludes his essay with this question:

The authors of “A Nation at Risk” concluded their report with a simple claim: “Education should be at the top of the Nation’s agenda.” And in creating a new kind of school-reform rhetoric, they seem to have achieved their aim. The question is, has it done more harm than good?

The answer is clear: it has done far more harm than good.

Categories: Uncategorized Tags: , , , ,

Bill Gates Millions Get “Lackluster Results”… And Christiansen Institute Sees Innovation as the Answer

July 14, 2018 Comments off

Christiansen Institute (CI) fellow Thomas Arnett writes thoughtful blog posts that buttress the CIs belief that technology-based disruption is the way forward if we hope to improve public schools. In his recent post in the CI’s weekly digest, Mr. Arnett offers his perspective on the recently released study of Bill Gates’ initiatives, describing their findings as “lackluster” and poses two questions based on these findings:

Following a multi-year, multi-million-dollar initiative by the Gates Foundation to boost school performance through better teacher evaluation systems, RAND’s researchers found that the initiative failed to produce significant gains in student outcomes. When a promising education reform like this one misses the mark, where should education leaders turn next? If we step back and look at the big picture, what options do we really have for dramatically improving schools?

In his examination of “the big picture” Mr. Arnett astutely observes that teachers have a limited time frame. He uses a principle of economics called the “production possibilities frontier” to illustrate this reality.

Here’s a simplified example to illustrate the concept: Suppose a hypothetical non-profit called Basic Aid makes two goods—bread and shirts—for people in need. If the non-profit has its employees spend all their time producing bread, it can churn out nine thousand loaves a month. Alternatively, Basic Aid could schedule its employees to spend all their time making shirts and produce seven thousand shirts a month. Most months, however, the non-profit has its employees split their time between these two forms of production.

The graph below represents Basic Aid’s production options. At point A, Basic Aid only makes bread. At point B it only makes shirts. Connecting those two points is a curve that shows all the ways Basic Aid could maximize its production of bread and shirts given its available workforce. This means Basic Aid can make tradeoffs to produce any combination of bread or shirts that lies on or below the curve. But it is impossible for Basic Aid to produce combinations of bread and shirts that are beyond the blue curve. Put simply, the curve represents Basic Aid’s production possibilities frontier.

How does this apply to schools? Teachers, like the imaginary non-profit, have a zero-sum time budget. I was heartened to see an acknowledgement that schools have a time budget that is more limited than it’s financial budget, but was disappointed to see that Mr. Arnett had no appreciation for the driving force behind how time is spent in schools operated by “reformers”. You see, in Bill Gates view and the view of most “reformers” the best way to determine a teacher’s impact was by examining changes in test scores. This, in turn, led to teachers squandering their time budgets on test preparation. If the only metric that counts is changes in test scores, it’s no surprise that teachers worked on test preparation… and no surprise that the results of their efforts were “lackluster”. Why? Because children have their own time budgets. They begin their schooling at different places and learn at different rates and tests expect all children to start at the same place and proceed at the same rate year-after-year. Anyone who has raised or taught children realizes that they grown and mature at different rates and that growth rates often have no bearing on where a child will ultimately end up. Standardized tests, though, do not take these individual differences into account. To make matters worse, many of the “reforms” that emerged from this test driven world developed scripts for teachers to follow based on the premise that learning was uniform… in effect that children’s individual “time budgets” would somehow change. This just in Mr. Gates: if time is a constant, learning will be variable no matter how you try to write a script wishing it were otherwise.

If the “reformers” want to see changes in performance, that learning should be constant, they need to accept the fact that time should be a variable.