Archive

Posts Tagged ‘value added’

Studies Show No Connection Between Test Scores and Life Outcomes… But, Nevertheless, Testing Persists!

March 16, 2018 Leave a comment

Diane Ravitch exhumed a blog post by Jay P. Greene, a charter school advocate, who begrudgingly acknowledged that there was no demonstrable link between standardized test scores and “later outcomes in life“. He writes:

If increasing test scores is a good indicator of improving later life outcomes, we should see roughly the same direction and magnitude in changes of scores and later outcomes in most rigorously identified studies.  We do not.  I’m not saying we never see a connection between changing test scores and changing later life outcomes (e.g. Chetty, et al); I’m just saying that we do not regularly see that relationship.  For an indicator to be reliable, it should yield accurate predictions nearly all, or at least most, of the time.

This is unsurprising. As Ms. Ravitch has noted in her books and blog posts the US has lagged in international test scores from the time they were initially issued and yet our economy has thrived and the general well-being of our population has improved. As many researchers have noted there is no correlation between SAT scores and success in college and yet post-secondary schools continue to use those test scores as a primary metric for accepting students.

And Mr. Greene, a proponent of charters and choice and heretofore a proponent of using standardized tests as a metric for measuring school effectiveness, was compelled to re-think his position on tests after examining rigorous research on their effectiveness as a predictor:

If we explored the most common use of test scores — examining the level of proficiency — there are no credible researchers who believe that is a reliable indicator of school or program quality.

It would be helpful if other advocates of charters and choice looked beyond test scores as well.

Advertisements

DC Schools Testing Scandal Proof of the Immutability of Campbell’s Law

February 11, 2018 Leave a comment

For the past several days, report-after-report has emerged from Washington DC where the public schools “fudged” the accountability data they were submitting in a way that made them look good. Here’s an excerpt from one of the latest reports of such “fudging”, which was brought to light by the local ABC station WJLA:

The DC Council Education committee held an accountability hearing with the DC Public Schools Chancellor, the State Superintendent and the Executive Director of the Charter School Board Thursday morning.

An investigative report from WJLA revealed the recording of a DCPS principal, telling teachers that the DCPS central office pressured principals to pass more students,” pointed out Councilmember Robert White.

White referenced a secretly recorded conversation with Roosevelt STAY High School’s principal in 2015 directing teachers to ignore DC attendance law, first reported by ABC7 News Monday.

“Here’s the thing: we have to pass and promote. If we are not then what are we here for? I’m sitting in a meeting to tell the chancellor you’ve got to give me more resources. I can’t sit in the meeting with the chancellor and I’m with big stats in red,” said Principal Young in the recording.

White called the recording, “a clear indication of widespread fraud in DCPS that was only brought to light because of investigative journalism.”

A few days ago, Diane Ravtich wrote a post on the immutability of Campbell’s Law, which was formulated by psychologist Donald Campbell at the end of the 1900s. Here’s the law in its entirety:

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

The Roosevelt STAY High School Principal’s directive to “pass and promote” illustrates the immutability of Campbell’s Law… and here’s the way to avoid Campbell’s Law coming into play: avoid the use of ANY quantitative social indicator for social decision-making. 

Article on MD’s New Rating System A Reinforces Notion that Parents are Consumers who PRESUMABLY have a Choice

July 1, 2017 Leave a comment

Liz Bowie’s Washington Post article describing the state of Maryland’s new rating system elicited a response that is a combination of resignation, dismay, and frustration.

Resignation because it is reinforces the notion that parents are “consumers” who presumably have a choice when it comes to school; dismay because that clearly is NOT the case for those parents who are living in poverty; and frustration because I don’t see either circumstance changing any time soon.

Ms. Bowie’s article describes the tension between the State Board, appointed by Republican Larry Hogan, who seek a greater emphasis on so-called “academic” measures and the largely Democratic legislature that passed a law limiting the percentage of a rating that can be based on “academic” measures (i.e. standardized test results) to 65%. But given the GOP leadership in the USDOE, the State Board’s desire to emphasize academics is likely to prevail. Why?

Basing just 65 percent of the weight on academics may be deemed too low by the U.S. Department of Education. The department has already told Delaware that its 80 percent weight for academics is too low.

And what, exactly, constitute “academic measures”? Here’s Maryland’s idea of “academics”, which currently constitute 65% of a school’s rating but, based on the USDOE’s reaction to Delaware’s submission, will need to constitute an even higher percentage:

… students’ annual academic growth and the percentage of students who pass the state English and math exams.

Meanwhile, those who celebrated the passage of ESSA might think again. Here’s how Ms. Bowie described ESSA as compared to NCLB, the law it replaced.

ESSA replaces the No Child Left Behind Act, which was in effect for more than a decade and was widely criticized for being too punitive because schools were judged almost solely on test scores. ESSA was designed to give more authority back to the states for how schools are held accountable.

The legislators who championed ESSA because it stopped the centralization of decision-making in Washington—especially the conservative GOP members who harp on this issue repeatedly— should ask themselves if that has really happened. It seems that the spirit of ESSA should allow Maryland to base 65% of it’s ratings on test scores. But the billionaires who love to demean public education and who help underwrite the campaigns of the “states-rights” Tea Party will likely look the other way on this issue.

Categories: Uncategorized Tags: , , ,

Who Paid to Support Pro-Charter Board Candidates in LA? The Usual Suspects

May 19, 2017 Leave a comment

Two news stories dominate the education blogs today: the fallout of the LA Board election that gave a majority of seats to pro-privatization candidates (covered in this post) and the DeVos-Trump budget (covered in the later post).

Diane Ravitch had two posts yesterday that had links to articles that dealt with the dark money funding “school reform”. Peter Dreier’s Huffpost article, “Big Money Wins in LA” delineates the huge amounts spent on that election which pitted pro-privatization candidate Nick Melvoin and incumbent Steve Zimmer, specifically identifies the donors to the pro-privatization candidate’s campaign, and briefly describes their backgrounds and home towns:

Among the big donors behind Melvoin and the CCSA were members of the Walton family (Alice Walton, Jim Walton, and Carrie Walton Penner) ― heirs to the Wal-Mart fortune from Arkansas, who’ve donated over $2 million to CCSA. Alice Walton (net worth: $36.9 billion), who lives in Texas, was one of the biggest funders behind Melvoin’s campaign. Reed Hastings, CEO of Netflicks (net worth: $1.9 billion), who lives in Santa Cruz, donated close to $5 million since last September to the CCSA’s political action committee, including $1 million a week before the election.

Other moguls behind Melvoin and the CCSA include Doris Fisher (net worth: $2.7 billion), co-founder of The Gap, who lives in San Francisco: Texas resident John Arnold (net worth: $2.9 billion), who made a fortune at Enron before the company collapsed, leaving its employees and stockholders in the lurch, then made another fortune as a hedge fund manager; Jeff Yass, who lives in the Philadelphia suburbs, and runs the Susquahanna group, a hedge fund; Frank Baxter, former CEO of the global investment bank Jefferies and Company that specialized in “junk” bonds; and Michael Bloomberg (net worth: $48.5 billion), the former New York City mayor and charter champion. Eli Broad (net worth: $7.7 billion), who hatched a plan to put half of all LAUSD students in charter schools by 2023 — an idea that Zimmer fought — donated $400,000 to CCSA last Friday, on top of $50,000 he gave in November. He made his money in real estate and life insurance.

Not surprisingly, most of these billionaires are big backers of conservative Republican candidates and right-wing causes. Several are on the boards of charter school chains.

After providing this rundown, Dreier poses the 6.6 million dollar question and offers an insightful answer, one that makes the distinction between “reform” and “privatization”:

What do the corporate moguls and billionaires want? 

They want to turn public schools into educational Wal-marts run on the same corporate model. They want to expand charter schools that compete with each other and with public schools in an educational “market place.” (LA already has more charter schools than any other district in the country). They want to evaluate teachers and students like they evaluate new products — in this case, using the bottom-line of standardized test scores. Most teachers will tell you that over-emphasis on standardized testing turns the classroom into an assembly line, where teachers are pressured to “teach to the test,” and students are taught, robot-like, to define success as answering multiple-choice tests…

The corporate big-wigs are part of an effort that they and the media misleadingly call “school reform.” What they’re really after is not “reform” (improving our schools for the sake of students) but “privatization” (business control of public education). They think public schools should be run like corporations, with teachers as compliant workers, students as products, and the school budget as a source of profitable contracts and subsidies for textbook companies, consultants, and others engaged in the big business of education.

And Dreier emphasizes that one thing the “reformers” did NOT want was someone like Melvin’s opponent, Steve Zimmer, to be on the school board. Why?

Like most reasonable educators and education analysts, Zimmer has questioned the efficacy of charter schools as a panacea. When the billionaires unveiled their secret plan to put half of LAUSD students into charter schools within eight years, Zimmer led the opposition….

Now the billionaires and their charter school operators will have a majority on the school board. LA will become the epicenter of a major experiment in expanding charter schools – with the school children as the guinea pigs.

In the coming weeks it will be interesting to see who turned out to vote for Mr. Melvoin and why the voters decided to put Mr. Zimmer out of office. As noted in a post yesterday, what is most telling is that Arne Duncan came out several weeks ago in support of Mr. Melvoin, advocating a need for a reformer to be elected to the board to allow a change to the status quo. If the likes of Mr. Duncan really sought a change to the status quo they would abandon the reliance on standardized test scores based on groupings of students by age cohorts… the reliance of which results in classrooms that are turned into “…an assembly line, where teachers are pressured to “teach to the test,” and students are taught, robot-like, to define success as answering multiple-choice tests.” That is hardly a change to the status quo: it reinforces the factory model that is failing children and creating failure where success might be possible.

VAM: The Mathbabe Declares The Death of a Bad Idea… But I’m Not So Sure!

May 15, 2017 Leave a comment

Cathy O’Neill, a.k.a the Mathbabe, is now writing a column on the use and misuse of statistics for Bloomberg News. Her latest piece for Bloomberg titled “Don’t Grade Teachers with a Bad Algorithm” opens with this heartening paragraph:

For more than a decade, a glitchy and unaccountable algorithm has been making life difficult for America’s teachers. The good news is that its reign of terror might finally be drawing to a close.

Ms. O’Neill then provides a concise history and analysis of VAM— an acronym for Value Added Model– one that has been offered in earlier posts on this blog but one that bears recounting:

The VAM — actually a family of algorithms — purports to determine how much “value” an individual teacher adds to a classroom. It goes by standardized test scores, and holds teachers accountable for what’s called student growth, which comes down to the difference between how well students performed on a test and how well a predictive model “expected” them to do.

Derived in the 1980s from agricultural crop models, VAM got a big boost from the education reform movements of presidents Bush and Obama. Bush’s No Child Left Behind Act called for federal standards, and Obama’s Race To The Top Act offered states some $350 billion in federal funds in exchange for instituting formal teacher assessments. Many states went for VAM, sometimes with bonuses and firings attached to the results.

Ms. O’Neill describes the flaws in VAM, the major one of which was it’s opacity. One of her friends, who was Principal of a school in Brooklyn asked to get a copy of the algorithm when VAM was instituted in NYC and was dismissively told it was unavailable and, anyway, “it’s math, you wouldn’t understand it.” So a building administrator, who was held accountable for the VAM results in her school, was not let in on the way VAM was calculated.

She concluded her article by offering two pieces of evidence supporting her contention that VAM is dead:

Happily, the tide appears to be turning. In 2015, a revamp of No Child Left Behind, called the Every Student Succeeds Act, removed the federal funding incentives that had supported the algorithm. In May 2016, a Long Island teacher named Sheri Lederman won a lawsuit against New York State in which a judge deemed the state’s VAM-based rating system “arbitrary and capricious.” And earlier this month, a group of teachers in Houston, where VAM had been used for firings and bonuseswon a lawsuit in which they successfully argued that the algorithm’s secretive and complex nature had effectively denied them due process.

VAM expert Audrey Amrein-Beardsley told me that the Houston decision, pertaining to the country’s seventh-largest school district, might have a “snowball effect,” influencing the outcome of other lawsuits across the country. Let’s hope so, because teachers deserve better.

While I sincerely hope Ms. Amrein-Beardsley is correct in her forecast of a “snowball effect”, given the inability of politicians to drive a stake through the heart of the Gaffer Curve myth (see my next post), I’m not at all certain this bad idea is dead just yet. Yes, ESSA DOES eliminate the federal funding incentives that supported the VAM algorithm… but there are several states (including NH, the State I reside in) that are led by Governors and legislators who believe in “hard data” provided by standardized tests and love the idea that these tests can prove that public education is “failing”…. and those Governors and legislators will be loathe to abandon simplistic ideas like VAM that demonstrate that “failing teachers” are the ultimate cause of “failing schools”. VAM won’t die until the public is willing to face the facts on public schools… that more money is needed to help the schools that serve the children raised in poverty.

 

Joe Nathan’s Rebuttal to CHARTER School Metrics Should Apply to ALL Public Schools

May 10, 2017 1 comment

A few weeks ago the Fordham Institute issued a report identifying “Three Signs That a Proposed Charter School is at Risk of Failing”. The report found three “risk factors” in approved applications that were significant predictors of a school’s future weak performance in its first years of operation. They were:

  1. Lack of identified leadership: Charter applications that propose a self-managed school without naming a school leader.
  2. High risk, low dose: Charter applications that propose to serve at-risk pupils but plan to employ “low dose” academic programs that do not include sufficient academic supports, such as intensive small-group instruction or extensive individual tutoring.
  3. A child-centered curriculum: Charter applications that propose to deploy child-centered, inquiry-based pedagogies, such as Montessori, Waldorf, Paideia, or experiential programs.

Further, when an application displayed two or more of these risk factors, the likelihood of low performance rose to 80 percent.

In a rebuttal to this report that was printed in the Fordham Institutes blog, Joe Nathan, one of the authors of Minnesota’s charter school laws and a longstanding advocate of charter schools disputed the Institute’s findings based on the fact that their sole metric for determining “success” was standardized test results. Mr. Nathan wants to see a wider range of factors taken into consideration, factors that he presents as “Four Key Questions”:

First, and most importantly, shouldn’t (the Fordham Institute) recognize that the public wisely cares about much more than a school’s “value added” on standardized tests? The short answer is: they should!

Second, what are the best measures to predict success as an adult? Mr. Nathan cites the findings ACT researchers who sought to determine which of four factors best predicted success as an adult: high grades in high school, high grades in college, high scores on their test, and participation in debate, speech, drama, and student government. They found that participation in those extracurriculars best predicted success in adulthood, as they defined it. Given those findings, Mr. Nathan rightfully suggests that this participation rate should be an important factor.

Third, given that schools in a democratic society are not just places to prepare students for work, don’t we want young people to graduate schools with the tools and attitudes needed to be active citizens? The short answer, again, is YES!

Finally, fourth, policymakers, educators, and authorizers should ask: “Do strong assessments exist beyond standardized tests that could help assess what’s happening with students in a school?” In responding to this question Mr. Nathan offers five examples of research-based metrics that are far superior to standardized testing.

Mr. Nathan asserts that Fordham Institute does a disservice to charter schools by limiting their metrics to standardized tests and suggests they broaden their scope of measurement to include other factors. He writes:

Strong reading, writing, and math skills are vital. But Americans wisely want more from their schools. Students, the charter movement, and the broader society will gain if we:

  • Recognize the importance of assessing a broad array of skills and knowledge, not just those that are measured by standardized tests.
  • Refine and encourage use by states and authorizers of valid assessments that measure a broader array of skills and knowledge.
  • Support and encourage development of schools, chartered and otherwise, that help students develop many strong skills and broad knowledge.

Chartering has grown in part because it builds on the fundamental American values of choice within some limits, and the belief that those creating new products and services should expect to be judged on results. 

I tend to begrudgingly acknowledge that the kinds of charters Mr. Nathan supports are needed in order to get US schools out of the rut they are in— away from the factory model and into one that emphasizes the “soft” skills needed to function in the emerging new economy and in a democracy. But I do not think that the drive for charters should be based on the consumerist values of “choice within limits”. Rather, we should drive for better schools based on the egalitarian values that our forefathers envisioned when they founded our government: that all citizens no matter what station they were born into would have an equal opportunity to achieve well being. By basing the charter movement on economic principals instead of egalitarian ideals we are playing into the hands of those who see schooling as job preparation and not a means of developing self-motivated lifelong learners,

Small Michigan District Sees the Light, Ends “Merit Pay”

April 18, 2017 Leave a comment

As readers of this blog realize, I oppose “merit pay” for teachers on a number of grounds, several of which were exemplified in the decision of Whitmore Lake Public Schools decision to end what they called “merit pay”— a laughable bonus of $100 for each teacher who was rated effective and $500 for teachers rated very effective. Based on an article by Lauren Slagter in Michigan Live, Whoitmore Public School Superintendent Tom DeKeyser announced to the Board that he was suspending the merit pay plan because “…while people are happy to receive it – has become negative” adding that “We’ll find another way to reward our highly effective teachers through collective bargaining.”

The article went on to note another problem DeKeyser encountered with his version of “merit pay”: it was linked to test scores and when the State changed their tests it became “…difficult to draw conclusions about teaching quality from students’ scores.” 

Patti Kobeck, president of the Whitmore Lake Education Association, offered her insights on merit pay:

“By taking the merit pay away and rewarding teachers in other ways, I think it will change the atmosphere. We’re here for the kids. Without merit pay, teachers can stop worrying about what another teacher is getting and worry about what they’re giving the kids.

In general, merit pay isn’t an effective way to motivate teachers to perform their jobs better. Small gestures of appreciation can be more meaningful, she said, because of the lack of respect for their profession many teachers feel.

After reading the closing paragraph of this article it is abundantly clear that an increase in base pay would go a long way to improving morale in Whitmore Lake:

Whitmore Lake teachers currently are under a one-year contract that granted them 1-percent raises, following a 4.9-percent pay cut they took under a 2014 to 2016 contract. The current contract expires June 30, 2017.

Hopefully other small districts will learn from Whitmore Lake’s misguided effort to offer bonuses based on test scores and restore the compensation levels before offering bonuses.