Archive

Posts Tagged ‘value added’

NYTimes Reporting on Tests Gets a “D”… the Lowest Passing Grade Possible

September 28, 2018 Comments off

NYTimes reporter Eliza Shapiro’s article on the release of last year’s test results gets a passing grade because it is factually accurate, but it gets a “D” because of what it fails to report. It factually recounts the reasons why last year’s results cannot be compared to previous year’s tests. It also factually reports on the opt out movement data and accurately contrasts the Mayor Bloomberg’s overemphasis on test results as compared to his successor, Bill de Blasio. But here are the points Ms. Shapiro missed:

  • Ms. Shapiro writes: “Just five years ago, it seemed the state was poised to have some of the toughest teacher evaluations in the nation, based on some of the most intensive exams.” The “tough evaluations” Mr. Bloomberg championed were based on VAM, a statistically invalid and discredited method of evaluation– not discredited by teachers unions, but by statisticians!
  •  Ms. Shapiro writes: “This year’s scores are the latest confusing data points in a long history of zigzagging test results in New York, but what they do tell us is how much the political pendulum has swung on standardized testing.” One VERY possible reason for the “zig-zag” in test scores, as any education writer should realize, is the fact that the definition of a “passing score” is not based on how students perform compared to a fixed set of standards: it is based on a cut score that can be set arbitrarily by those who designed the test… and, surprise… cut scores have proved to be a function of political variances and not a function of teaching variances. That is, political forces swing the pendulum on test scores and parents, teachers, and many voters are beginning to realize that reality. Ms. Shapiro’s article makes no mention of this, leading an innocent reader to believe that performance, not the cut score, zig-zagged. This only adds to the public’s confusion about the data points.
  • Ms. Shapiro oversells the differences in test scores… even after she wrote in one of the opening paragraphs that “…the (State education) department cautioned that the exams cannot be measured against previous tests and should be considered a new baseline.” What Ms. Shapiro fails to realize is that if they can’t be measured against a previous baseline, they can’t be analyzed by comparing how one group of schools did as opposed to another set. Despite this reality, Ms. Shapiro compares the performance of charter schools to regular public schools and regular charter schools to “Renewal schools” and draws a long series of faulty conclusions as a result.
  • Ms. Shapiro downplays the bottom line: all of the “reforms” in NY have made no difference whatsoever. She writes: “Although state exam results have fluctuated wildly in the past 10 years, the National Assessment of Educational Progress, considered the gold-standard measure of academic growth, found that New York City’s students did not make progress in English or math between 2015 and 2017.” 
  • And last, but not least, Ms. Shapiro concludes her article with this: “This year’s results will no doubt influence fresh questions about education policy in New York, including the state Board of Regents’ decision about whether to use exam results in teacher evaluations.

So Ms. Shapiro got the facts right… but she got the conclusions all wrong… and the public, as a result, is getting the message that reforms advocated by the billionaires are doing well (i.e. charters “outperform” public schools) but the reforms advocated by progressives like the Mayor are not making a difference (i.e. the renewal school scores are flat) and that VAM is still a viable means of evaluating teachers. Ugh!

Advertisements

Arne Duncan Still True Believer in VAM, “Failure” of Public Schools, Standardized Testing

August 9, 2018 Comments off

Arne Duncan has written a new book, How Schools Work: An Inside Account of Failure and Success From One of the Nation’s Longest-Serving Secretaries of Education, and he is getting lots of publicity as he tours the country selling his book and the tired ideas in it. Here’s the opening paragraph from a review of his book by Atlantic reporter Alia Wong:

Arne Duncan, the former education secretary under President Barack Obama, has always been more candid than others who’ve served in that role. He’s often used his platform to talk about what he sees as the persistent socioeconomic and racial disparities in access to quality schools. His new book, How Schools Work: An Inside Account of Failure and Success From One of the Nation’s Longest-Serving Secretaries of Education, further cements that reputation. How Schools Work’s first chapter is titled “Lies, Lies Everywhere.” The first sentence: “Education runs on lies.” If one were to create a word cloud of the book, lies would probably pop out as one of the most frequently used words. Duncan writes that even the countless fantastic schools across the country “haven’t managed to defeat the lies that undermine our system so much as they’ve been able to circumvent them.” These lies, according to Duncan, include a culture of setting low expectations for high schoolers who later discover they’re not prepared for the real world, and poorly designed accountability systems that allow teachers to fudge their students’ test-score results.

This paragraph itself is full of canards about public education that only someone who never set foot in a public school could believe. I worked in an urban middle school, a blue collar suburban high school, and a rural high school that served many poor families. The teachers in these schools, even the weakest, had high expectations for their children.

As for the “accountability systems that allow teachers to fudge their test-score results”, I presume he must be referring to the grading systems that allow students to pass a course with a “C” or a “D”, grades that typically require a student to get grades that do not require mastery of ALL the information presented. And the norm-referenced tests that were the backbone of the RTTT “accountability systems” Mr. Duncan imposed on schools that were presumably designed to avoid the “fudging” did nothing to help students. They only reinforced the notion that students were poorly prepared because teachers were lazy and incompetent and did so by providing a sheen of precision.

In the interview with Ms. Wong that accompanied this overview of his book Mr. Duncan DID reveal an understanding of the root cause of “failing” schools… and it isn’t the teachers… it’s parents who are disengaged from the lives of their students, parent’s whose disengagement is often the result of working multiple jobs or, in the worst case, drug and alcohol abuse. Here’s Mr. Duncan’s take:

It’s the parents who aren’t present whose kids you have to worry about even more because those parents just have too much going on in their own lives to be engaged in their children’s education. Those kids are the ones I actually worry about the most.

But, as written frequently in this blog, actions speak louder than words. IF Mr. Duncan believed this as the head of public education in Chicago and then the nation, why did he not take action to provide support for the children of disengaged parents, the children whose performance pulls down the test scores he values so highly and whose ultimate withdrawal from schools increases the drop out rates he blames on “the system”?

Mr. Duncan’s perspective on gun violence was also on point. But like his views on the problems presented by disengaged parents, it’s a perspective he failed to share when he led the nation’s schools:

I talk a lot about gun violence—it’s what I’m dealing with in Chicago all the time; it unfortunately shaped me as a kid; we saw it in the Sandy Hook massacre, which happened when I was education secretary. There’s no political leader who says they don’t value kids, but the truth is: we value guns more than we value the lives of our children .And that is irrefutable if you look at the rates of gun deaths in the U.S. compared to other nations that make other policy choices.

Mr. Duncan purports to be one who perceives education as a great equalizer and one who attempts to use data to help him see what works and what doesn’t work. I wish that as Secretary of Education emeritus he would take a dispassion look at the true impact of RTTT and acknowledge that it was a doubling down on NCLB, a program he viewed as “horribly constructed.” I wish he would acknowledge that the standardized tests he advocated were not constructed to perform the VAM he mandated and resulted in the discrediting of the teaching profession. I wish that he would trumpet the need for programs to support parents who “…just have too much going on in their own lives to be engaged in their children’s education” and speak out against the politicians who value guns more than we value the lives of our children. Finally, I wish he would acknowledge that the programs he advocates, the expansion of choice and charters, reward those parents who are engaged in the lives of their children, sidestep the need for a larger investment in the safety net, and divert needed funds away from public schools.

Tennessee’s Faith in Testing is Based on the Flawed Premise that More Money is Unnecessary

August 2, 2018 Comments off

Diane Ravitch wrote a post yesterday decrying the “colossal failure” of the school district comprised of the “failing schools” taken over by the state, a district that received millions of dollars from Race to the Top to replace “failing” public schools with charters. There’s only one problem, as noted in the Chalkbeat article that was the basis for Ms. Ravitch’s post: low-performing schools operated by Shelby County Schools, where most of the “failing schools” taken over by the state are located, have outpaced progress of those run by the state! 

The agreeable fantasy that a “state takeover” of “failing schools” or the outsourcing of those same schools to deregulated charter schools would lead to their “improvement” underpins virtually all federal and state legislation. It also underpins the reform movement and leads to other agreeable fantasies promoted by reformers. These agreeable fantasies enable politicians to dodge the need for legislation that would either require them to raise more money for public education or divert the money already allocated to less affluent districts. It also enabled them to adopt other ideas based on magical thinking, ideas that don’t require more money but result in “improvement”. Ideas like: firing “bad teachers” would improve “failing schools”; or, “adopting a uniform curriculum” would improve the teaching and learning in “failing schools”; or “eliminating frills” would direct more resources “to the classroom”; or implementing merit pay plans that would reward the “best and brightest teachers” and withhold raises from “weak teachers” thereby “improving failing schools”; or implementing programs that “increase the grit” of students raised in poverty to help them overcome the adversity they face. None of these ideas require more money and all of them directly or indirectly scapegoat the teachers who work tirelessly to improve their “failing” schools.

But the biggest agreeable fantasy is that statewide standardized norm referenced tests are the best means of measuring the “quality of education”… and, as we are witnessing, the state that gave us value added testing has tremendous faith in that fantasy… and. alas, so do most voters across the country despite the accumulating evidence to the contrary.

Atlantic Article Advises Against Aggregating Nation’s Schools

July 31, 2018 Comments off

Earlier this month, Atlantic writer Jack Schneider wrote an article urging our nation to stop thinking of “America’s Schools” as a monolith, arguing that in doing so we are doing more harm than good. At the outset of his essay. Mr. Schneider identified A Nation at Risk as the the point when we established “…a new way of talking about public education in the United States, a master narrative that has endured — and even subtly changed American education policy for the worse — over the past several decades.” That “master narrative” is that public education is a monolith and that it is “failing”. But Mr. Schneider asserts that this is not the case at all:

The abstraction of “America’s schools” may be convenient for rousing the collective conscience, but it is not particularly useful for the purpose of understanding (or improving) American education. Consider the issue of funding. On average, federal money accounts for less than 10 percent of education budgets across the country, and the rest of the financial responsibility falls to states and local schools. If local schools are unable to raise what they need, the state is usually well positioned to make up the difference, but states differ dramatically in their approaches. On average, states spend roughly $13,000 per student on public education — but looking at the average alone is misleading. Only about half of states spend anything close to that figure: A dozen spend 25 percent more than the national average, and 10 states spend 25 percent less. The result is significant disparities, and some striking incongruities. New York’s schools, for instance, spend roughly three times as much per student as Utah’s schools — a huge difference, even after accounting for New York’s higher cost of living.

And once the “collective conscience” of the politicians was roused by A Nation at Risk, both political parties bought into the “failing schools” narrative and began imposing one-size-fits-all solutions to the monolith, ignoring the reality that funding was hugely disparate and the policies governing schools resided in state and local governments. But Mr. Schneider does see a value in looking at public education as a national issue.

This is not to say that taking the national perspective can’t be valuable. Troubling patterns do exist across the U.S., and discussions about them can play an important role in shaping both public understanding and education policy. Achievement gaps across race and class, for instance, are an important reminder of broader social and economic inequalities, and advocates have used evidence about those patterns to make the case for universal early-childhood education. Similarly, a national dialogue about the disproportionate punishment of black and brown children in schools has drawn attention to an issue that might otherwise have gone unnoticed. These kinds of broad conversations can generate both political will and policy responses.

But, alas, while these kinds of broad conversations CAN generate political will and policy responses, they have not done so thus far. Instead, they’ve generated policy responses like NCLB, Race to the Top, and ESSA, all of which use standardized testing to reinforce the notion that public education is a monolith and it is failing. Mr. Schneider concludes his essay with this question:

The authors of “A Nation at Risk” concluded their report with a simple claim: “Education should be at the top of the Nation’s agenda.” And in creating a new kind of school-reform rhetoric, they seem to have achieved their aim. The question is, has it done more harm than good?

The answer is clear: it has done far more harm than good.

Categories: Uncategorized Tags: , , , ,

Bill Gates Millions Get “Lackluster Results”… And Christiansen Institute Sees Innovation as the Answer

July 14, 2018 Comments off

Christiansen Institute (CI) fellow Thomas Arnett writes thoughtful blog posts that buttress the CIs belief that technology-based disruption is the way forward if we hope to improve public schools. In his recent post in the CI’s weekly digest, Mr. Arnett offers his perspective on the recently released study of Bill Gates’ initiatives, describing their findings as “lackluster” and poses two questions based on these findings:

Following a multi-year, multi-million-dollar initiative by the Gates Foundation to boost school performance through better teacher evaluation systems, RAND’s researchers found that the initiative failed to produce significant gains in student outcomes. When a promising education reform like this one misses the mark, where should education leaders turn next? If we step back and look at the big picture, what options do we really have for dramatically improving schools?

In his examination of “the big picture” Mr. Arnett astutely observes that teachers have a limited time frame. He uses a principle of economics called the “production possibilities frontier” to illustrate this reality.

Here’s a simplified example to illustrate the concept: Suppose a hypothetical non-profit called Basic Aid makes two goods—bread and shirts—for people in need. If the non-profit has its employees spend all their time producing bread, it can churn out nine thousand loaves a month. Alternatively, Basic Aid could schedule its employees to spend all their time making shirts and produce seven thousand shirts a month. Most months, however, the non-profit has its employees split their time between these two forms of production.

The graph below represents Basic Aid’s production options. At point A, Basic Aid only makes bread. At point B it only makes shirts. Connecting those two points is a curve that shows all the ways Basic Aid could maximize its production of bread and shirts given its available workforce. This means Basic Aid can make tradeoffs to produce any combination of bread or shirts that lies on or below the curve. But it is impossible for Basic Aid to produce combinations of bread and shirts that are beyond the blue curve. Put simply, the curve represents Basic Aid’s production possibilities frontier.

How does this apply to schools? Teachers, like the imaginary non-profit, have a zero-sum time budget. I was heartened to see an acknowledgement that schools have a time budget that is more limited than it’s financial budget, but was disappointed to see that Mr. Arnett had no appreciation for the driving force behind how time is spent in schools operated by “reformers”. You see, in Bill Gates view and the view of most “reformers” the best way to determine a teacher’s impact was by examining changes in test scores. This, in turn, led to teachers squandering their time budgets on test preparation. If the only metric that counts is changes in test scores, it’s no surprise that teachers worked on test preparation… and no surprise that the results of their efforts were “lackluster”. Why? Because children have their own time budgets. They begin their schooling at different places and learn at different rates and tests expect all children to start at the same place and proceed at the same rate year-after-year. Anyone who has raised or taught children realizes that they grown and mature at different rates and that growth rates often have no bearing on where a child will ultimately end up. Standardized tests, though, do not take these individual differences into account. To make matters worse, many of the “reforms” that emerged from this test driven world developed scripts for teachers to follow based on the premise that learning was uniform… in effect that children’s individual “time budgets” would somehow change. This just in Mr. Gates: if time is a constant, learning will be variable no matter how you try to write a script wishing it were otherwise.

If the “reformers” want to see changes in performance, that learning should be constant, they need to accept the fact that time should be a variable.

 

DC Miracle Story Evidence of Traction of “Fake News”

June 30, 2018 Comments off

A few days ago AP writer Ashram Kahlil wrote an article titled “DC’S Public Schools Go from Success Story to Cautionary Tale“, a story that was picked up by NPR and some other mainstream news outlets. But alas, Time magazine is unlikely to run a cover story with Michelle Rhee sitting on a dunce stool or holding a broken broom.

In 2008, both Time and Newsweek offered overs depicting then rising star Michelle Rhee, the no-nonsense DC Superintendent who pledged to clean up the public schools in that city by implemented a test-and-punish policy that garnered support among those who thought schools needed to be operated using a no nonsense “business” approach and negative attention from anyone who actually worked in schools and realized that instead of a clean sweep their schools needed new floors, new lighting, and more money.

Since 2008, funding for schools has diminished, in some cases in real dollars and in all cases in terms of actual funding… and the consequences of test-and-punish has not been the improvement of test scores but rather the expansion of corruption in the administration of those high stakes tests. And DC has had its eyes blackened badly. As Mr. Kahlil reports:

As recently as a year ago, the public school system in the nation’s capital was being hailed as a shining example of successful urban education reform and a template for districts across the country.

Now the situation in the District of Columbia could not be more different. After a series of rapid-fire scandals, including one about rigged graduation rates, Washington’s school system has gone from a point of pride to perhaps the largest public embarrassment of Mayor Muriel Bowser’s tenure.

This stunning reversal has left school administrators and city officials scrambling for answers and pledging to regain the public’s trust.

A decade after a restructuring that stripped the decision-making powers of the board of education and placed the system under mayoral control, city schools in 2017 were boasting rising test scores and a record graduation rate for high schools of 73 percent, compared with 53 percent in 2011. Glowing news articles cited examples such as Ballou High School, a campus in a low-income neighborhood where the entire 2017 graduating class applied for college.

Then everything unraveled.

An investigation by WAMU, the local NPR station, revealed that about half of those Ballou graduates had missed more than three months of school and should not have graduated due to chronic truancy. A subsequent inquiry revealed a systemwide culture that pressured teachers to favor graduation rates over all else — with salaries and job security tied to specific metrics.

The internal investigation concluded that more than one-third of the 2017 graduating class should not have received diplomas due to truancy or improper steps taken by teachers or administrators to cover the absences. In one egregious example, investigators found that attendance records at Dunbar High School had been altered 4,000 times to mark absent students as present. The school system is now being investigated by both the FBI and the U.S. Education Department, while the D.C. Council has repeatedly called for answers and accountability.

It takes a long time to inculcate a culture of support, but a culture of fear can be implemented rapidly… and once that culture is in place it is hard to change. And that culture is especially hard to change when “salaries and job security tied to specific metrics” and those metrics can be manipulated by those who will be damaged the most: the administrators and politicians who based their careers and campaigns on their ability “…to improve public education.”

And who implemented this culture that resulted from salaries and job security tied to specific metrics?

As Mr. Kahlil reports in his closing paragraphs… it was none other than Michelle Rhee:

Critics view the problems, particularly the attendance issue, as an indictment of the entire data-driven evaluation system instituted a more than a decade ago when then-Mayor Adrian Fenty took over the school system and appointed Michelle Rhee as the first chancellor. Rhee’s ambitious plan to clear out dead wood and focus on accountability for teachers and administrators landed her on the cover of Time magazine holding a broom. But now analysts question whether Rhee’s emphasis on performance metrics has created a monster.  

Readers of this blog know the answer to that question: there is no doubt that the test-and-punish methods supported by Ms. Rhee and her follow reformers created a monster… but it’s serving their purposes: it is creating the impression that public schools are not only “failing” based on those test scores, but they are now “corrupt” because of the actions of a handful of administrators whose continued employment required them to boost them.

And here’s one fact that remains the same today as it was in 2008: the teachers who work in poverty stricken urban and rural districts like DC are giving their hearts and should to the jobs and the administrators in those same schools are being over backwards to support them. But a cover article lionizing public school teachers and principals is not nearly as compelling as one showing that an inexpensive one-size-fits-all solution is the best way to fix schools.

 

 

 

 

 

 

Studies Show No Connection Between Test Scores and Life Outcomes… But, Nevertheless, Testing Persists!

March 16, 2018 Comments off

Diane Ravitch exhumed a blog post by Jay P. Greene, a charter school advocate, who begrudgingly acknowledged that there was no demonstrable link between standardized test scores and “later outcomes in life“. He writes:

If increasing test scores is a good indicator of improving later life outcomes, we should see roughly the same direction and magnitude in changes of scores and later outcomes in most rigorously identified studies.  We do not.  I’m not saying we never see a connection between changing test scores and changing later life outcomes (e.g. Chetty, et al); I’m just saying that we do not regularly see that relationship.  For an indicator to be reliable, it should yield accurate predictions nearly all, or at least most, of the time.

This is unsurprising. As Ms. Ravitch has noted in her books and blog posts the US has lagged in international test scores from the time they were initially issued and yet our economy has thrived and the general well-being of our population has improved. As many researchers have noted there is no correlation between SAT scores and success in college and yet post-secondary schools continue to use those test scores as a primary metric for accepting students.

And Mr. Greene, a proponent of charters and choice and heretofore a proponent of using standardized tests as a metric for measuring school effectiveness, was compelled to re-think his position on tests after examining rigorous research on their effectiveness as a predictor:

If we explored the most common use of test scores — examining the level of proficiency — there are no credible researchers who believe that is a reliable indicator of school or program quality.

It would be helpful if other advocates of charters and choice looked beyond test scores as well.