Posts Tagged ‘Testing’

PISA results are in… And Neither the “Reformers” or Politicians Will Like Researchers’ Conclusions

December 7, 2016 Leave a comment

Amanda Ripley’s NYTimes Upshot article on the PISA results will not go over well with the “reform” crowd or the politicians who fail to face the facts on equitable funding. The PISA tests, (Programme for International Student Assessment) is an international study of 15-year-old school pupils’ scholastic performance on mathematics, science, and reading.conducted by the Organisation for Economic Co-operation and Development (OECD). It has been given every three years since 2003 and the results of the assessments are publicized a year later. During the intervening years, statisticians and psyshometricians analyze the results and draw conclusions about the effectiveness of various national strategies for improving schools… and the findings are not particularly helpful for the “reformers”. Here’s why: the only piece of good news in the results was an improvement in equity where: “One in every three disadvantaged American teenagers beat the odds in science, achieving results in the top quarter of students from similar backgrounds worldwide.” But Ms Ripley could not link this to anything associated with the “reform” movement. Her synopsis of the PISA analysis was:

Generally speaking, the smartest countries tend to be those that have acted to make teaching more prestigious and selective; directed more resources to their neediest children; enrolled most children in high-quality preschools; helped schools establish cultures of constant improvement; and applied rigorous, consistent standards across all classrooms.

Of all those lessons learned, the United States has employed only one at scale: A majority of states recently adopted more consistent and challenging learning goals, known as the Common Core State Standards, for reading and math. These standards were in place for only a year in many states, so Mr. Schleicher did not expect them to boost America’s PISA scores just yet. (In addition, America’s PISA sample included students living in states that have declined to adopt the new standards altogether.)

So Ms. Ripley concludes that the US has only employed one of the proven methods “at scale”… and then goes on to note that this “at scale” improvement was NOT adopted by all the States and had not been adopted in time for it to have any impact on the test results. So what DID result in the improvement of the performance by our disadvantaged students? We know it was’t more money… we know it wasn’t an effort to make teaching a more selecting and honored profession…. we know it wasn’t an upgrade of our virtually non-existent preschool program… and it wasn’t the Common Core. Is it possible that our teachers are doing a better job out of sheer pride in the craft? I believe that is the case, but that idea will never see the light of day in the NYTimes because it contradicts the “reform” narrative that teachers are the problem and more money isn’t needed.

Despite Ms. Ripley’s misplaced enthusiasm for the Common Core and failure to acknowledge the good work of teachers in our country, she does draw the right conclusion at the end of her article:

As we drift toward a world in which more good jobs will require Americans to think critically — and to repeatedly prove their abilities before and after they are hired — it is hard to imagine a more pressing national problem. “Your president-elect has promised to make America great again,” (PISA administrator) Mr. Schleicher said. But he warned, “He won’t be able to do that without fixing education.”

And the fix Mr. Trump is proposing has nothing to do with the need to make teaching more prestigious and selective; to direct more resources to their neediest children; to enroll most children in high-quality preschools; to help schools establish cultures of constant improvement; or to apply rigorous, consistent standards across all classrooms. The PISA results in 2018 will likely reflect his efforts… and they are unlikely to show that we are on the right track.


Chinese Bands and US Charters: Sorting by Eugenics and Sorting by Parent Engagement

December 2, 2016 Leave a comment

I read an article in yesterday’s NYTimes with a mix of astonishment and revulsion. The article, by Didi Kirsten Tatlow, describes a music program in China where students are enrolled in band programs and assigned musical instruments in the band based solely on their physical attributes. Titled “In China, Eugenics Determines Who Gets in School Band”, Ms. Tatlow’s article describes the method “Teacher Wang” uses to identify prospective musicians. Here is an excerpt from the article that describes his meeting with the parents of the future band members:

Mr. Wang, whom parents addressed only as “Teacher,” (a sign of respect common here) stood before a giant white screen on which he projected a power point full of instrument images. “I’ve chosen your kids, one by one, out of a thousand kids.” Mr. Wang was referring to band C, the third in the school which trained the youngest students, some of whom would eventually rise through the ranks to band B and on to A, at which point they would perform at overseas gigs.

“I’ve looked at their teeth, at their arms, their height, everything, very carefully,” Teacher Wang said. “We don’t want anyone with asthma, or heart problems, or eye problems. And we want the smart kids; the quick learners.”

“Your kids were chosen not because they want to play this or that instrument, but because they have long arms, or the right lips, or are the right height, say for the trumpet, or the drums,” he said.

This sounded appalling to Ms. Tatlow, but ultimately she accepted the program in large measure because her daughter wanted to be a part of it and evidently possessed the physical and intellectual qualities Teacher Wang was seeking.

In some respects US schools in the 50s and 60s were no different: students were sorted into homogeneous batches based on their intellect and upbringing— and until 1954 they were also sorted based on race, a vestigial method of sorting that remains in place today on a de facto basis. As an elementary student I was among the group in my PA elementary group that were “smart kids”. I was in the highest reading group and did well in math without much effort. When my father was transferred to Oklahoma I was identified as “gifted and talented”, largely because 4th grade in that state was comparable to 3rd grade in PA. When he got transferred back to PA, though, I was in for a rude awakening. I was no longer deemed to be a “smart kid”. Rather, I was a “kid from Oklahoma” and was consequently placed in a mid-level section of students. I excelled in my classwork, but when the team of teachers met with my parents to discuss my placement in one of the higher groups they were told there was no room in those classes. And so for the next five years I remained in the “second tier”.

Schools today avoid that kind of rigid homogeneous grouping within the school… but they achieve homogeneity in a different fashion. Schools in affluent communities effectively screen out the “middling” students because their parents cannot afford housing in those towns. Charter schools in cities can screen out children of indifferent or working parents because their enrollment procedures require a level of engagement that is virtually impossible in a single parent household or in a household where both parents work. So the schools in less affluent areas and the non-charter schools in the city tend to have students whose parents are less engaged. And here’s where our sorting arrangement and that of the Chinese music teachers are similar: a child born into a US family where the parents are unwilling or unable to engage in their schooling has no more chance at success than a child born in China who lacks the physical and intellectual qualities sought by Teacher Wang. The result in both cases is a tremendous waste of talent.

Don’t Move to Canada, Move to A Red State and Teach Kids About Racism, Sexism, and Evolution

November 13, 2016 Leave a comment

An instagram post that had this message appeared on Facebook this morning. I left this comment:

If you’re moving to a red state you’re probably going to be working for a for-profit charter with a canned curriculum to prepare kids to pass standardized tests…

It’s where we are headed given ESSA and the fact that 33 States are under Republican control….

Money Matters in Massachusetts… And More Charters Don’t Matter NEARLY As Much

November 5, 2016 Leave a comment

Yesterday’s NYTimes feed on “Tests and Testing” featured an article by David Leonardt that will appear in this Sunday’s paper. In the article Mr. Leonardt describes the demonstrable and unequivocal evidence of the kind of charter school that succeeds with all students… a charter school he describes as “high expectations, high support”. He offers this article as evidence that voters should support Question #2 in Massachusetts, a proposition that calls for the lifting of a cap on charter schools. But, as I noted in a comment I left on the article, Mr. Leonardt’s analysis overlooks several points:
1 – The expansion of charters in Massachusetts does not mean the expansion of demonstrably successful “high expectations, high support” charters… it means the expansion of ALL charters, including those the “many charter schools (that) fail to live up to their promise”.
2 – The study used to “prove” that high expectations, high support” charters work doesn’t examine the cost per pupil required to provide the “high support”, which must either be higher than the public school, necessitate lower compensation for teachers and administrators who work there, or be offset by private funds that underwrite the additional services.
3 – The metric used to determine “success” is standardized tests… a narrow means of determining success and a means that can be manipulated by shedding low achieving students.
4 – MA, like every state in the union, has many “high expectations, high support” schools. They are located in the affluent suburbs that surround urban areas and exurban college towns…. and the public gladly pays high taxes to support them.

5 – Money matters, Mr. Leonardt.


Fairtest’s State-Wide Assessment Model Should be Embraced by NEA, AFT, NSBA, and AASA

October 12, 2016 Leave a comment

As noted in earlier blogs, the potential curse of ESSA is that many states are currently controlled by pro-“reform” Governors who will use the “flexibility” built into the new federal law to continue and— ins some cases– exacerbate the current test-and-punish system. But Fairtest, an organization that “...advances quality education and equal opportunity by promoting fair, open, valid and educationally beneficial evaluations of students, teachers and schools” and “…works to end the misuses and flaws of testing practices that impede those goals” released a report late last week that recommends State’s replace standardized multiple choice tests with performance assessments. In the report they offer this description of “performance assessments” and a subsequent paragraph that described the most important reason for giving assessments:

Performance assessments are intended to improve learning in ways that may not show up on standardized tests. Ideally, they can narrow gaps in achievement in areas that really matter for students’ future success, such as designing an extended project and persevering to completion. The danger is that discrepancies with results from current tests could lead to dismissing other forms of learning gains that are more meaningful. This may be particularly harmful in schools that had most heavily focused on test scores, and thus for low-income children, children of color, English language learners and students with disabilities.

Comparability has value, but the great value of assessment is to enrich student learning. The dangers from comparability requirements could be lessened if districts are not forced to alter their local assessment scores to be comparable to state test results. However, as long as current standardized exams are falsely presented as the “gold standard,” the problem will remain.

Testing WILL happen under ESSA and unless educational organizations can get behind an alternative to the “gold standard” advocated by “reformers” with deep pockets States will continue to use the cheap, easy, and seemingly exact multiple choice tests that have been in place since the passage of NCLB.  I REALLY hope the NEA, AFT, NSBA and AASA unite behind the kind of testing Monty Neill advocates and actively discourage the kinds of testing we’ve witnessed under NCLB and RTTT. If they can do so there is a possibility of undercutting the corporations and foundations who DO have a united front on precisely the kinds of testing NCLB and RTTT were built on and who continue to crank out variations in the name of achieving a “gold standard” that is irreversible.

National organizations face several challenges in their fight to replace the current testing with the kind Mr. Neill recommends. One problem is that the corporate reformers have momentum now after more than a decade of the test-and-punish region imposed by NCLB and the public has become accustomed to the simple “grading” systems States use to rank schools and the VAM methods they’ve sold to politicians. Another is the desire for each of the national organizations to devise their own unique perspective on issues and represent their constituencies on issues like student assessment. And the biggest impediment is that while national education associations represent thousands of adults they cannot begin to raise the kinds of funds that hedge funders and billionaires have and are willing to throw at the issue of school reform. Consequently, a small group of pro-privatization and pro-technology investors have an outsized influence in determining the future direction of schooling. The kinds of assessments Fairtest advocates, based on practitioner-designed performance tasks and “…student-focused assessments that emerge from ongoing schoolwork” are difficult to design and complicated to implement but they DO result in the development of agency on the part of the student and promote opportunities for students and teachers to work together in learning activities.

The Fairtest report illustrates how one State, New Hampshire, has developed a State-wide performance assessment that could be replicated in other states and DOES meet the standards set forth in ESSA. Unless national organizations unify behind performance assessments the “gold standard” of computerized testing will continue.

The Mathbabe Pushes Back Against VAM Critics— But Overturning VAM will be a Daunting Challenge

October 9, 2016 Leave a comment

In a blog post a few weeks ago that Diane Ravitch linked to yesterday Cathy O’Neill (a.k.a the Mathbabe) offered some counterarguments to critics who pushed back when she slammed VAM (Value Added Model) in her recent book Weapons of Math Destruction”. As one who was seeking a way to make use of the test scores that are generated due to the NCLB mandates that emerged in the early 2000s, I was drawn to the ideas that William Sanders proposed regarding “value added” testing. But I quickly saw that the rigorous methods he initially advocated were being oversimplified and in virtually all cases the tests that many “reformers” wanted to use to measure “value added” were NOT designed for that purpose. Moreover, as statisticians like Ms. O’Neill noted, VAM was a wrongheaded approach to begin with. Nevertheless, despite all the flaws in VAM, it gained traction among politicians who saw it as a means of “weeding out” bad teachers and saw the critics of VAM as either union apologists or etherial intellectuals. Consequently, when President Obama was elected and passed an overly modest stimulus package for public education, he used VAM as the centerpiece of his Race to the Top (RTTT) grant program, effectively requiring that it be used as the basis for teacher evaluations in order for States to receive any of the funding. The two States I was working in at the time, NH and VT, were among the last to seek RTTT funds, in large measure because the leadership in the State got pushback from either State Boards or Superintendents.

In her recent post, Ms. O’Neill responds to one of the frequent rebuttals she’s received as a result of her criticism of VAM, with my emphasis added:

Here’s an example of an argument I’ve seen consistently when it comes to the defense of the teacher value-added model (VAM) scores… Namely, that the teacher’s VAM scores were “one of many considerations” taken to establish an overall teacher’s score. The use of something that is unfair is less unfair, in other words, if you also use other things which balance it out and are fair.

Ms. O’Neill makes one clearly straightforward logical rebuttal to this “one of many considerations” argument, with my emphasis added:

The obvious irony of the “one of many” argument is, besides the mathematical one I will make below, that the VAM was supposed to actually have a real effect on teachers assessments, and that effect was meant to be valuable and objective. So any argument about it which basically implies that it’s okay to use it because it has very little power seems odd and self-defeating.

While the use of the “one of many” argument IS “odd and self-defeating”, it is also an argument that has intuitive appeal and one that would enable the use of a “valuable and objective” tool that is also— conveniently— cheap, easy, and seemingly exacting. But what if the exactitude is pointless and meaningless? As Ms. O”Neill notes, when everything else that constitutes a teacher evaluation yields very little variance, as is the case in teacher evaluations, the pointless and meaningless but exact measures can ultimately be the determining factor.

The VAM was brought in precisely to introduce variance to the overall mix. You introduce numeric VAM scores so that there’s more “spread” between teachers, so you can rank them and you’ll be sure to get teachers at the bottom.

But if those VAM scores are actually meaningless, or at least extremely noisy, then what you have is “spread” without accuracy. And it doesn’t help to mix in the other scores.

In a statistical sense, even if you allow 50% or more of a given teacher’s score to consist of non-VAM information, the VAM score will still dominate the variance of a teacher’s score. Which is to say, the VAM score will comprise much more than 50% of the information that goes into the score.

In the end, I have to believe that some statistician at the USDOE knew this whole concept was flawed but supported it anyway because VAM is easy to implement, relatively inexpensive, and intuitively appealing. The shame is that once a concept like this takes hold, correcting it is extremely difficult as is replacing it with something new. And with ESSA now in place, it will require a change of heart in 50 State capitols since virtually every state in the union embraced the VAM precepts when they accepted the RTTT funds. The “Weapon of Math Destruction” will be the Obama-Duncan legacy….

Don’t Medicate or Punish: Meditate

September 23, 2016 Leave a comment

My daughter shared a post from the Free Thought Project web page by John Vibes on Facebook that described how a school in Baltimore completely eliminated suspensions by sending children to a “mindful moment room” to wind down and meditate. Mr. Vibes writes of the project which has been underway at Robert W. Coleman Elementary in West Baltimore:

The new policy has been in place for over a year, and in the time that the meditation room has been set up, there has actually been no suspensions throughout the entire year.

The program is an initiative organized by the Holistic Life Foundation, a Baltimore-based nonprofit organization committed to nurturing the wellness of children and adults in underserved communities.

Andres Gonzalez, one of the organizers of the project, says that children are even bringing home what they are learning to their families.

“That’s how you stop the trickle-down effect, when Mom or Pops has a hard day and yells at the kids, and then the kids go to school and yell at their friends,” he says. “We’ve had parents tell us, ‘I came home the other day stressed out, and my daughter said, Hey, Mom, you need to sit down. I need to teach you how to breathe,‘” Gonzalez said.

As one who witnessed an explosion in the use of medications to “control” impulsive behaviors and ADHD in children and one who has witnessed the positive effects of meditation, I am heartened to see that schools are applying the research on meditation in classrooms. And, as Vibes notes in his post, meditation isn’t necessarily limited to sitting on a cushion:

A bike ride, a walk under the stars, writing poetry, or any practice that offers individual quiet time within your own heart and mind can be considered a form of meditation.

Sadly, in many cases, children raised in poverty seldom experience quiet time when they can look within their own hearts an minds and be in the present moment. I believe if schools spent more time focusing on the present moment and less on preparing for tests that the well-being of children would improve dramatically… and that the test scores would improve as the children’s well-being improves. We know this: it doesn’t work the other way around.

Categories: Uncategorized Tags: ,