Posts Tagged ‘Measurement’

Fairtest’s State-Wide Assessment Model Should be Embraced by NEA, AFT, NSBA, and AASA

October 12, 2016 Leave a comment

As noted in earlier blogs, the potential curse of ESSA is that many states are currently controlled by pro-“reform” Governors who will use the “flexibility” built into the new federal law to continue and— ins some cases– exacerbate the current test-and-punish system. But Fairtest, an organization that “...advances quality education and equal opportunity by promoting fair, open, valid and educationally beneficial evaluations of students, teachers and schools” and “…works to end the misuses and flaws of testing practices that impede those goals” released a report late last week that recommends State’s replace standardized multiple choice tests with performance assessments. In the report they offer this description of “performance assessments” and a subsequent paragraph that described the most important reason for giving assessments:

Performance assessments are intended to improve learning in ways that may not show up on standardized tests. Ideally, they can narrow gaps in achievement in areas that really matter for students’ future success, such as designing an extended project and persevering to completion. The danger is that discrepancies with results from current tests could lead to dismissing other forms of learning gains that are more meaningful. This may be particularly harmful in schools that had most heavily focused on test scores, and thus for low-income children, children of color, English language learners and students with disabilities.

Comparability has value, but the great value of assessment is to enrich student learning. The dangers from comparability requirements could be lessened if districts are not forced to alter their local assessment scores to be comparable to state test results. However, as long as current standardized exams are falsely presented as the “gold standard,” the problem will remain.

Testing WILL happen under ESSA and unless educational organizations can get behind an alternative to the “gold standard” advocated by “reformers” with deep pockets States will continue to use the cheap, easy, and seemingly exact multiple choice tests that have been in place since the passage of NCLB.  I REALLY hope the NEA, AFT, NSBA and AASA unite behind the kind of testing Monty Neill advocates and actively discourage the kinds of testing we’ve witnessed under NCLB and RTTT. If they can do so there is a possibility of undercutting the corporations and foundations who DO have a united front on precisely the kinds of testing NCLB and RTTT were built on and who continue to crank out variations in the name of achieving a “gold standard” that is irreversible.

National organizations face several challenges in their fight to replace the current testing with the kind Mr. Neill recommends. One problem is that the corporate reformers have momentum now after more than a decade of the test-and-punish region imposed by NCLB and the public has become accustomed to the simple “grading” systems States use to rank schools and the VAM methods they’ve sold to politicians. Another is the desire for each of the national organizations to devise their own unique perspective on issues and represent their constituencies on issues like student assessment. And the biggest impediment is that while national education associations represent thousands of adults they cannot begin to raise the kinds of funds that hedge funders and billionaires have and are willing to throw at the issue of school reform. Consequently, a small group of pro-privatization and pro-technology investors have an outsized influence in determining the future direction of schooling. The kinds of assessments Fairtest advocates, based on practitioner-designed performance tasks and “…student-focused assessments that emerge from ongoing schoolwork” are difficult to design and complicated to implement but they DO result in the development of agency on the part of the student and promote opportunities for students and teachers to work together in learning activities.

The Fairtest report illustrates how one State, New Hampshire, has developed a State-wide performance assessment that could be replicated in other states and DOES meet the standards set forth in ESSA. Unless national organizations unify behind performance assessments the “gold standard” of computerized testing will continue.

A “Reformer” in Connecticut’s Superior Court

October 10, 2016 Leave a comment

Last month, Elizabeth Harris wrote an article in the NYTimes on a far reaching decision rendered by Connecticut Superior Court Judge Thomas Moukawsher, who issued a 200+ page ruling on that state’s funding formula. Mr. Harris’ article focussed on the primary issue that faced the judge, the fairness of funding, and noted in passing some of the other issues the judge touched on in his lengthy decision.

Last weekend, Wendy Lecker, a Hearst Connecticut Media Group columnist and senior attorney at the Education Law Center, wrote an op ed piece decrying some of the remedies embedded in the judge’s decision, remedies that are based on the popular misconception that exit examinations will ensure uniform success for all learners and VAM will ensure quality teaching. When groups filing lawsuits seeking equitable funding get a decision that affirms their assertion that the existing funding mechanisms are inherently inequitable, they don’t expect to receive decisions that call for practices that are destructive to the students who are raised in poverty or to the teachers who are willing to devote their careers to working with those students. But, as Ms. Lecker notes, that is exactly what Mr. Moukawsher did in his rambling decision.

On the issue of exit examinations, where the judge cited Massachusetts’ successes, Ms. Lecker writes:

The judge decided that because Connecticut does not have “rational” and “verifiable” high school standards, meaning standards measured by a high school exit exam, Connecticut diplomas for students in poor districts are “patronizing and illusory.” He concluded that the cure for this problem is standardized, “objective” exams that students must pass to graduate…

(H)ad the judge examined the evidence, he would have also learned that the actual major factor in Massachusetts’ improvement was the very measure he refused to order Connecticut to implement: school finance reform that dramatically increased the amount of school funding statewide. No fewer than three studies have shown that increasing school funding significantly improved student achievement in Massachusetts. Recent major studies confirmed those findings nationwide, demonstrating that school finance reform has the most profound positive impact among poor students.

When it came to teacher evaluations, which fall well outside the purview of a ruling on funding equity, the judge advocated VAM as a method. In response to that decision Ms. Lecker writes:

Courts that have actually examined the evidence on systems that rate teachers on student test scores have rejected these systems. Last year, a court in New Mexico issued a temporary injunction barring the use of test scores in that state’s teacher evaluation system. And in April, a court in New York ruled that a teacher’s rating based on her students’ “growth” scores — the foundation of New York’s teacher evaluation system — was “arbitrary and capricious;” the opposite of “rational” and “verifiable.”

Yet despite the reams of evidence debunking the use of student growth scores in evaluating teachers, and despite these two court rulings, Judge Moukawsher insisted that rating teachers on student “growth” scores would satisfy his demand that Connecticut’s system for hiring, firing, evaluating and compensating teachers be “rational” and “verifiable.” His ruling defies the evidence and logic.

A month ago when I wrote a post on the Connecticut ruling I surmised that, based on what’s happened in other states where the courts fond the funding inequitable, nothing would happen as a result the judges decision. I was wrong. In this case, as a result of the judge’s overreach, both sides on this issue are appealing the decision to a higher court… and as the case goes forward I share Ms. Lecker’s hopes:

One can only hope that that our highest court will steer this case back on course, away from these ill-advised educational policy rulings and toward a proper finding that the state is failing to provide our poorest schools with adequate funding and is consequently failing to safeguard the educational rights of our most vulnerable children.

Stay tuned… it will be another school year at best before anything happens… and likely another generation before change occurs in Connecticut… if it happens at all.


The Mathbabe Pushes Back Against VAM Critics— But Overturning VAM will be a Daunting Challenge

October 9, 2016 Leave a comment

In a blog post a few weeks ago that Diane Ravitch linked to yesterday Cathy O’Neill (a.k.a the Mathbabe) offered some counterarguments to critics who pushed back when she slammed VAM (Value Added Model) in her recent book Weapons of Math Destruction”. As one who was seeking a way to make use of the test scores that are generated due to the NCLB mandates that emerged in the early 2000s, I was drawn to the ideas that William Sanders proposed regarding “value added” testing. But I quickly saw that the rigorous methods he initially advocated were being oversimplified and in virtually all cases the tests that many “reformers” wanted to use to measure “value added” were NOT designed for that purpose. Moreover, as statisticians like Ms. O’Neill noted, VAM was a wrongheaded approach to begin with. Nevertheless, despite all the flaws in VAM, it gained traction among politicians who saw it as a means of “weeding out” bad teachers and saw the critics of VAM as either union apologists or etherial intellectuals. Consequently, when President Obama was elected and passed an overly modest stimulus package for public education, he used VAM as the centerpiece of his Race to the Top (RTTT) grant program, effectively requiring that it be used as the basis for teacher evaluations in order for States to receive any of the funding. The two States I was working in at the time, NH and VT, were among the last to seek RTTT funds, in large measure because the leadership in the State got pushback from either State Boards or Superintendents.

In her recent post, Ms. O’Neill responds to one of the frequent rebuttals she’s received as a result of her criticism of VAM, with my emphasis added:

Here’s an example of an argument I’ve seen consistently when it comes to the defense of the teacher value-added model (VAM) scores… Namely, that the teacher’s VAM scores were “one of many considerations” taken to establish an overall teacher’s score. The use of something that is unfair is less unfair, in other words, if you also use other things which balance it out and are fair.

Ms. O’Neill makes one clearly straightforward logical rebuttal to this “one of many considerations” argument, with my emphasis added:

The obvious irony of the “one of many” argument is, besides the mathematical one I will make below, that the VAM was supposed to actually have a real effect on teachers assessments, and that effect was meant to be valuable and objective. So any argument about it which basically implies that it’s okay to use it because it has very little power seems odd and self-defeating.

While the use of the “one of many” argument IS “odd and self-defeating”, it is also an argument that has intuitive appeal and one that would enable the use of a “valuable and objective” tool that is also— conveniently— cheap, easy, and seemingly exacting. But what if the exactitude is pointless and meaningless? As Ms. O”Neill notes, when everything else that constitutes a teacher evaluation yields very little variance, as is the case in teacher evaluations, the pointless and meaningless but exact measures can ultimately be the determining factor.

The VAM was brought in precisely to introduce variance to the overall mix. You introduce numeric VAM scores so that there’s more “spread” between teachers, so you can rank them and you’ll be sure to get teachers at the bottom.

But if those VAM scores are actually meaningless, or at least extremely noisy, then what you have is “spread” without accuracy. And it doesn’t help to mix in the other scores.

In a statistical sense, even if you allow 50% or more of a given teacher’s score to consist of non-VAM information, the VAM score will still dominate the variance of a teacher’s score. Which is to say, the VAM score will comprise much more than 50% of the information that goes into the score.

In the end, I have to believe that some statistician at the USDOE knew this whole concept was flawed but supported it anyway because VAM is easy to implement, relatively inexpensive, and intuitively appealing. The shame is that once a concept like this takes hold, correcting it is extremely difficult as is replacing it with something new. And with ESSA now in place, it will require a change of heart in 50 State capitols since virtually every state in the union embraced the VAM precepts when they accepted the RTTT funds. The “Weapon of Math Destruction” will be the Obama-Duncan legacy….

Restorative Justice Boosts Self-Awareness, Builds Community, and Builds Skills Needed in a Democracy

September 9, 2016 Leave a comment

This Sunday’s NYTimes will feature an article by Susan Dominus on how the use of restorative justice in an urban high school in NYC has dramatically lowered the suspension rates. Ms. Dominus’ article vividly describes the daunting challenges an administrator faces when trying to replace the criminal justice model of discipline with a restorative justice model. Teachers and deans who are accustomed to swift and automatic consequences for specific forms of misconduct are thrown when they are expected to deal with small offenses on their own and expected to help students learn to manage their own conduct. After reading the description of how the Principal at Leadership and Public Service High School in Manhattan’s Financial District implemented restorative justice model over a period of years, Ms. Santos noted that:

“While studies have shown that restorative practices curb suspensions, research on their influence on test scores and grades is inconclusive.”

It’s a sad reality that schools are assessed based on standardized test scores and students progress is measured by grades— because both are based on the premise that time is fixed and performance is variable. Moreover, test scores and grades measure what is easy to measure but ultimately not that important. Restorative justice, as this article shows, tackles the toughest and most important issues. If we want to graduate students who are ready to thrive in a community, who are self-actualized learners, who are self-aware, who understand the skills needed to function in a democracy, we need to ignore their standardized tests and change our thinking about grades. We need to show them the same patience in the mastery of academics as restorative justice affords them in the management of their emotions. If we continue to focus on seemingly objective and precise metrics like standardized tests and grades we will continue ignoring the emotional well-being of children. Given our obsession with tests and grades Is it any surprise that we are reading countless articles about disaffected and disengaged young adults?

Ms. Dominus illustrates the difficulty of changing the dominant paradigm of school discipline and, in so doing, illustrates how difficult it is to change the dominant thinking about test-based accountability. Her article is aptly titled “An Effective but Exhausting Alternative to High School  Suspensions”.  What Ms. Dominus fails to acknowledge is that our current practice with school discipline is IN-effective but equally exhausting. As is our practice in batching students in age based cohorts and expecting them to progress in lockstep.

“We don’t believe that we have failing schools… We’ve been failed.”

September 8, 2016 Leave a comment

The quote that serves as the title of this post comes from a Truthout article by Mike Ludwig titled “After Hundreds of School Closures, Black Families are Still Waiting for Justice”. In the article Mr. Ludwig describes the “reform” cycle whereby schools in poor urban neighborhoods are closed because they were deemed to be “failing” based on “...standards set by bureaucrats and lawmakers miles away”. But some parents are getting wise to what is happening in their neighborhoods and in their cities.

In cities across the country, hundreds of schools have shut down under so-called “reform” policies handed down by the Bush and Obama administrations, according to Journey for Justice. State and local officials use enrollment numbers, high-stakes testing scores and other metrics attached to state and federal funding incentives to identify and shut down schools considered to be “failing,” robbing neighborhoods of essential public resources and disrupting students’ academic life.

“We don’t believe that we have failing schools,” (Chicago activist Jitu) Brown told Truthout. “We think that’s a political statement. We’ve been failed.”

Brown says that taxpaying parents in Black neighborhoods deserve better-funded schools with more resources for learning, but the inequities in Chicago are sitting in plain sight. For example, schools in wealthier, whiter neighborhoods enjoy teacher’s aides in every classroom and librarians on staff at all times, while schools in lower-income neighborhoods of color do not.

Instead of providing more money for schools serving poor children, districts are consolidating failing schools or turning over their operation to private for-profit organizations. In both cases the students see no marked improvement in their performance and the neighborhoods where the schools close are disrupted. Jitu Brown, who is the national director of the Journey for Justice Alliance, an organization comprised of grassroots civil rights groups in 23 cities, is leading the fight to replace these kinds of policies that shut down schools and replace them with community-based solutions. But the fight is arduous, complicated, and time consuming. At this juncture the Department of Education is examining his organization’s complaints to see if Federal Laws have been violated. But the concluding paragraphs of Ludwig’s article offer a dispiriting conclusion:

Brown said he is grateful that the federal authorities agreed to investigate educational discrimination in New Orleans and Chicago, but now that two years have passed, he’s starting to doubt that federal civil rights officials are the “crusaders for justice” that he once hoped they would be.

“The wheels of justice, they are rusted,” Brown said. “And they don’t turn.”

The wheels of reform, however, oiled by the donations of billionaires, are gliding smoothly as privatized charters invade the neighborhoods and push public schools out of the picture altogether.


Getting the Lead Out A Good Example of Government Acting on Scientific Evidence

September 4, 2016 Leave a comment

Paul Krugman’s column on September 2 described our country’s decision to allow the poisoning of thousands of children due to lead contamination to continue. Why? Two reasons: the children are almost all poor and minorities; and the solution to the problem would involve government intervention and cost lots of money. And how do politicians explain this decision to ignore this state of affairs? By denying the scientific evidence or accepting evidence gathered by “scientists” funded by corporate donors.

If this sounds familiar, it’s because we’ve witnessed it before with cigarettes, with acid rain, with lead paint and tetra-ethyl lead in gasoline (both eventually outlawed), and with flouro-carbons, which were also eventually outlawed. But we are also witnessing it now in our denial of climate change, in our continued unwillingness to get the lead out of water in many communities across the country… and in our continued belief that we don’t need to address poverty in order to improve our public schools. Indeed, Krugman emphasizes the link between lead in the water and education:

But I’ve just been reading a new study by a team of economists and health experts confirming the growing consensus that even low levels of lead in children’s bloodstreams have significant adverse effects on cognitive performance. And lead exposure is still strongly correlated with growing up in a disadvantaged household.

But how can this be going on in a country that claims to believe in equality of opportunity? Just in case it’s not obvious: Children who are being poisoned by their environment don’t have the same opportunities as children who aren’t.

I guess it isn’t obvious to those who insist on administering standardized tests to children who are being poisoned by their environment and then concluding that their poor performance is the result of attending “over-regulated government schools”.

Mississippi Legislators Following Reform Playbook

September 3, 2016 Leave a comment

Retired English professor TJRay wrote an op ed piece for the Oxford (MS) Eagle decrying the recent action of the legislature and State Board in Mississippi, actions that follow the ALEC inspired “reform” playbook to a “T”. Mr. Ray’s essay describes how the legislature passed a bill that makes it possible for public schools to be closed and replaced with charter schools if they are graded lower than a “B”. And now, only weeks later, the State Board– appointed by the same political party that is in the legislature– is ready to enact a new rating system that limits the number of schools that can receive an “A” rating and mandates a minimum number of schools that must receive an “F” rating.

As Mr. Ray notes:

The object (of the bill that passed) was not to improve the public schools in question; it was to feather the nests of the corporations and groups that set up charter schools. An interesting inquiry might pose the question: How many names on those corporate charters match names on generous campaign donors? Well, obviously they’re getting their payback for putting the folks back where they can wreak havoc in the state.

And Mr. Ray also questions the rationale for the “reform” movement in Mississippi offered by the State’s Commissioner of Higher Education:

The Commissioner of Higher Education said that the foundation of education that students will need to succeed in universities is not being provided. One response might simply be that every young person doesn’t need to succeed at a university, may not even be suited to academics at all.

The oligarchs manufactured need to prepare all students for college leads to artificially high standards which leads to artificially difficult tests which leads to high failure rates in public schools which leads to the need to close those schools and replace them with privatized schools run by the oligarchs. And to make sure this machinery is well-oiled the oligarchs help elect politicians who support this “system” that keeps them enriched and a large number of children on a path to “failure”…. or at least on a path to work for lower wages.