“People and educators often deeply underestimate that it actually hurts to fail,” he explained. “The world is so much more open than any report card or any test score.”
An instagram post that had this message appeared on Facebook this morning. I left this comment:
If you’re moving to a red state you’re probably going to be working for a for-profit charter with a canned curriculum to prepare kids to pass standardized tests…
It’s where we are headed given ESSA and the fact that 33 States are under Republican control….
I was pleased to read in Politico that John King, Arne Duncan’s replacement as Secretary of Education, discovered that none of the 15 minority students in a classroom he was visiting in Coolidge Senior High School were familiar with the College Scorecard, the abominable metric devised by the Department of Education to measure the “good outcomes” students should expect from a college program. But I was also distressed to read that Mr. King found this “worrisome” because he thought this College Scorecard provided the kind of information students needed to make their decision about where to enroll in college. And what are the “good outcomes” measured by the College Scorecard? The descriptor of “Affordable Four Year Schools With Good Outcomes” offers the answer:
These four-year public colleges offer their students an affordable higher education, with relatively high salaries. As students weigh the costs and benefits of higher education, it’s especially important to find schools that can offer them the best possible outcomes. For students looking for a high return on investment, these institutions may offer good opportunities.
As progressive liberal arts majors dreaded, the government algorithm used to determine a “good outcome” is driven by the mean wages of graduates… and as a result schools offering technical degrees fare far better than those offering liberal arts degrees. This means that a school like Drexel University, my alma mater, where grads earn over $62,600 is presumably nearly twice as good as Evergreen State College, my daughter’s alma mater, where the average grad earns $32,800. This is, of course, absurd given that Drexel graduates are predominantly engineering and science majors and Evergreen graduates mostly liberal arts majors who work in social services and education.
Over the next several months we’ll read a lot about President Obama’s legacy. Sadly his legacy will include the fact that income is the primary measure of a “good outcome” when it comes to post-secondary education.
As noted in earlier blogs, the potential curse of ESSA is that many states are currently controlled by pro-“reform” Governors who will use the “flexibility” built into the new federal law to continue and— ins some cases– exacerbate the current test-and-punish system. But Fairtest, an organization that “...advances quality education and equal opportunity by promoting fair, open, valid and educationally beneficial evaluations of students, teachers and schools” and “…works to end the misuses and flaws of testing practices that impede those goals” released a report late last week that recommends State’s replace standardized multiple choice tests with performance assessments. In the report they offer this description of “performance assessments” and a subsequent paragraph that described the most important reason for giving assessments:
Performance assessments are intended to improve learning in ways that may not show up on standardized tests. Ideally, they can narrow gaps in achievement in areas that really matter for students’ future success, such as designing an extended project and persevering to completion. The danger is that discrepancies with results from current tests could lead to dismissing other forms of learning gains that are more meaningful. This may be particularly harmful in schools that had most heavily focused on test scores, and thus for low-income children, children of color, English language learners and students with disabilities.
Comparability has value, but the great value of assessment is to enrich student learning. The dangers from comparability requirements could be lessened if districts are not forced to alter their local assessment scores to be comparable to state test results. However, as long as current standardized exams are falsely presented as the “gold standard,” the problem will remain.
Testing WILL happen under ESSA and unless educational organizations can get behind an alternative to the “gold standard” advocated by “reformers” with deep pockets States will continue to use the cheap, easy, and seemingly exact multiple choice tests that have been in place since the passage of NCLB. I REALLY hope the NEA, AFT, NSBA and AASA unite behind the kind of testing Monty Neill advocates and actively discourage the kinds of testing we’ve witnessed under NCLB and RTTT. If they can do so there is a possibility of undercutting the corporations and foundations who DO have a united front on precisely the kinds of testing NCLB and RTTT were built on and who continue to crank out variations in the name of achieving a “gold standard” that is irreversible.
National organizations face several challenges in their fight to replace the current testing with the kind Mr. Neill recommends. One problem is that the corporate reformers have momentum now after more than a decade of the test-and-punish region imposed by NCLB and the public has become accustomed to the simple “grading” systems States use to rank schools and the VAM methods they’ve sold to politicians. Another is the desire for each of the national organizations to devise their own unique perspective on issues and represent their constituencies on issues like student assessment. And the biggest impediment is that while national education associations represent thousands of adults they cannot begin to raise the kinds of funds that hedge funders and billionaires have and are willing to throw at the issue of school reform. Consequently, a small group of pro-privatization and pro-technology investors have an outsized influence in determining the future direction of schooling. The kinds of assessments Fairtest advocates, based on practitioner-designed performance tasks and “…student-focused assessments that emerge from ongoing schoolwork” are difficult to design and complicated to implement but they DO result in the development of agency on the part of the student and promote opportunities for students and teachers to work together in learning activities.
The Fairtest report illustrates how one State, New Hampshire, has developed a State-wide performance assessment that could be replicated in other states and DOES meet the standards set forth in ESSA. Unless national organizations unify behind performance assessments the “gold standard” of computerized testing will continue.
Last month, Elizabeth Harris wrote an article in the NYTimes on a far reaching decision rendered by Connecticut Superior Court Judge Thomas Moukawsher, who issued a 200+ page ruling on that state’s funding formula. Mr. Harris’ article focussed on the primary issue that faced the judge, the fairness of funding, and noted in passing some of the other issues the judge touched on in his lengthy decision.
Last weekend, Wendy Lecker, a Hearst Connecticut Media Group columnist and senior attorney at the Education Law Center, wrote an op ed piece decrying some of the remedies embedded in the judge’s decision, remedies that are based on the popular misconception that exit examinations will ensure uniform success for all learners and VAM will ensure quality teaching. When groups filing lawsuits seeking equitable funding get a decision that affirms their assertion that the existing funding mechanisms are inherently inequitable, they don’t expect to receive decisions that call for practices that are destructive to the students who are raised in poverty or to the teachers who are willing to devote their careers to working with those students. But, as Ms. Lecker notes, that is exactly what Mr. Moukawsher did in his rambling decision.
On the issue of exit examinations, where the judge cited Massachusetts’ successes, Ms. Lecker writes:
The judge decided that because Connecticut does not have “rational” and “verifiable” high school standards, meaning standards measured by a high school exit exam, Connecticut diplomas for students in poor districts are “patronizing and illusory.” He concluded that the cure for this problem is standardized, “objective” exams that students must pass to graduate…
(H)ad the judge examined the evidence, he would have also learned that the actual major factor in Massachusetts’ improvement was the very measure he refused to order Connecticut to implement: school finance reform that dramatically increased the amount of school funding statewide. No fewer than three studies have shown that increasing school funding significantly improved student achievement in Massachusetts. Recent major studies confirmed those findings nationwide, demonstrating that school finance reform has the most profound positive impact among poor students.
When it came to teacher evaluations, which fall well outside the purview of a ruling on funding equity, the judge advocated VAM as a method. In response to that decision Ms. Lecker writes:
Courts that have actually examined the evidence on systems that rate teachers on student test scores have rejected these systems. Last year, a court in New Mexico issued a temporary injunction barring the use of test scores in that state’s teacher evaluation system. And in April, a court in New York ruled that a teacher’s rating based on her students’ “growth” scores — the foundation of New York’s teacher evaluation system — was “arbitrary and capricious;” the opposite of “rational” and “verifiable.”
Yet despite the reams of evidence debunking the use of student growth scores in evaluating teachers, and despite these two court rulings, Judge Moukawsher insisted that rating teachers on student “growth” scores would satisfy his demand that Connecticut’s system for hiring, firing, evaluating and compensating teachers be “rational” and “verifiable.” His ruling defies the evidence and logic.
A month ago when I wrote a post on the Connecticut ruling I surmised that, based on what’s happened in other states where the courts fond the funding inequitable, nothing would happen as a result the judges decision. I was wrong. In this case, as a result of the judge’s overreach, both sides on this issue are appealing the decision to a higher court… and as the case goes forward I share Ms. Lecker’s hopes:
One can only hope that that our highest court will steer this case back on course, away from these ill-advised educational policy rulings and toward a proper finding that the state is failing to provide our poorest schools with adequate funding and is consequently failing to safeguard the educational rights of our most vulnerable children.
Stay tuned… it will be another school year at best before anything happens… and likely another generation before change occurs in Connecticut… if it happens at all.
In a blog post a few weeks ago that Diane Ravitch linked to yesterday Cathy O’Neill (a.k.a the Mathbabe) offered some counterarguments to critics who pushed back when she slammed VAM (Value Added Model) in her recent book Weapons of Math Destruction”. As one who was seeking a way to make use of the test scores that are generated due to the NCLB mandates that emerged in the early 2000s, I was drawn to the ideas that William Sanders proposed regarding “value added” testing. But I quickly saw that the rigorous methods he initially advocated were being oversimplified and in virtually all cases the tests that many “reformers” wanted to use to measure “value added” were NOT designed for that purpose. Moreover, as statisticians like Ms. O’Neill noted, VAM was a wrongheaded approach to begin with. Nevertheless, despite all the flaws in VAM, it gained traction among politicians who saw it as a means of “weeding out” bad teachers and saw the critics of VAM as either union apologists or etherial intellectuals. Consequently, when President Obama was elected and passed an overly modest stimulus package for public education, he used VAM as the centerpiece of his Race to the Top (RTTT) grant program, effectively requiring that it be used as the basis for teacher evaluations in order for States to receive any of the funding. The two States I was working in at the time, NH and VT, were among the last to seek RTTT funds, in large measure because the leadership in the State got pushback from either State Boards or Superintendents.
In her recent post, Ms. O’Neill responds to one of the frequent rebuttals she’s received as a result of her criticism of VAM, with my emphasis added:
Here’s an example of an argument I’ve seen consistently when it comes to the defense of the teacher value-added model (VAM) scores… Namely, that the teacher’s VAM scores were “one of many considerations” taken to establish an overall teacher’s score. The use of something that is unfair is less unfair, in other words, if you also use other things which balance it out and are fair.
Ms. O’Neill makes one clearly straightforward logical rebuttal to this “one of many considerations” argument, with my emphasis added:
The obvious irony of the “one of many” argument is, besides the mathematical one I will make below, that the VAM was supposed to actually have a real effect on teachers assessments, and that effect was meant to be valuable and objective. So any argument about it which basically implies that it’s okay to use it because it has very little power seems odd and self-defeating.
While the use of the “one of many” argument IS “odd and self-defeating”, it is also an argument that has intuitive appeal and one that would enable the use of a “valuable and objective” tool that is also— conveniently— cheap, easy, and seemingly exacting. But what if the exactitude is pointless and meaningless? As Ms. O”Neill notes, when everything else that constitutes a teacher evaluation yields very little variance, as is the case in teacher evaluations, the pointless and meaningless but exact measures can ultimately be the determining factor.
The VAM was brought in precisely to introduce variance to the overall mix. You introduce numeric VAM scores so that there’s more “spread” between teachers, so you can rank them and you’ll be sure to get teachers at the bottom.
But if those VAM scores are actually meaningless, or at least extremely noisy, then what you have is “spread” without accuracy. And it doesn’t help to mix in the other scores.
In a statistical sense, even if you allow 50% or more of a given teacher’s score to consist of non-VAM information, the VAM score will still dominate the variance of a teacher’s score. Which is to say, the VAM score will comprise much more than 50% of the information that goes into the score.
In the end, I have to believe that some statistician at the USDOE knew this whole concept was flawed but supported it anyway because VAM is easy to implement, relatively inexpensive, and intuitively appealing. The shame is that once a concept like this takes hold, correcting it is extremely difficult as is replacing it with something new. And with ESSA now in place, it will require a change of heart in 50 State capitols since virtually every state in the union embraced the VAM precepts when they accepted the RTTT funds. The “Weapon of Math Destruction” will be the Obama-Duncan legacy….
Restorative Justice Boosts Self-Awareness, Builds Community, and Builds Skills Needed in a Democracy
This Sunday’s NYTimes will feature an article by Susan Dominus on how the use of restorative justice in an urban high school in NYC has dramatically lowered the suspension rates. Ms. Dominus’ article vividly describes the daunting challenges an administrator faces when trying to replace the criminal justice model of discipline with a restorative justice model. Teachers and deans who are accustomed to swift and automatic consequences for specific forms of misconduct are thrown when they are expected to deal with small offenses on their own and expected to help students learn to manage their own conduct. After reading the description of how the Principal at Leadership and Public Service High School in Manhattan’s Financial District implemented restorative justice model over a period of years, Ms. Santos noted that:
“While studies have shown that restorative practices curb suspensions, research on their influence on test scores and grades is inconclusive.”
It’s a sad reality that schools are assessed based on standardized test scores and students progress is measured by grades— because both are based on the premise that time is fixed and performance is variable. Moreover, test scores and grades measure what is easy to measure but ultimately not that important. Restorative justice, as this article shows, tackles the toughest and most important issues. If we want to graduate students who are ready to thrive in a community, who are self-actualized learners, who are self-aware, who understand the skills needed to function in a democracy, we need to ignore their standardized tests and change our thinking about grades. We need to show them the same patience in the mastery of academics as restorative justice affords them in the management of their emotions. If we continue to focus on seemingly objective and precise metrics like standardized tests and grades we will continue ignoring the emotional well-being of children. Given our obsession with tests and grades Is it any surprise that we are reading countless articles about disaffected and disengaged young adults?
Ms. Dominus illustrates the difficulty of changing the dominant paradigm of school discipline and, in so doing, illustrates how difficult it is to change the dominant thinking about test-based accountability. Her article is aptly titled “An Effective but Exhausting Alternative to High School Suspensions”. What Ms. Dominus fails to acknowledge is that our current practice with school discipline is IN-effective but equally exhausting. As is our practice in batching students in age based cohorts and expecting them to progress in lockstep.