Posts Tagged ‘Measurement’

Cuomo Sees the Light: the NYTimes… not so much

November 26, 2015 Leave a comment

In an article in today’s NYTimes, Kate Taylor reports that NY Governor Andrew Cuomo has let it be known that he is no longer in support of tying teacher evaluations to test scores and his recently announced Task Force on the Common Core is expected to incorporate such a recommendation in its findings. The Times infers that by creating the Task Force the governor is giving himself political cover to reverse his thinking on testing and now with the abandonment of the Race to the Top waivers that required such a shift he is free to do so.

One intriguing paragraph suggests that some of the Governor’s “school reform” donors have also accepted the political reality that tests are too dominant, but they repeated their bogus charges about the success rate of students:

It also appears that the advocates and donors to the governor who praised his call last winter for a more rigorous teacher evaluation system would not criticize him if it were now unwound.

StudentsFirstNY, an advocacy group that promotes charter schools and other education reforms, on whose board several of those donors sit, strongly endorsed the governor’s campaign to make test scores matter more in evaluations, saying the existing system bore “zero resemblance” to how students themselves were performing across the state.

Asked this week about a possible reversal, the organization’s executive director, Jenny Sedlis, said in an email, “When only a third of students in this state are performing on grade level, even without evaluations, we know that there’s ineffective teaching going on.”

A key fact the article neglects to mention: the passing grade on the test is not based on a percentage of students mastering a set of predetermined standards, it is determined by the setting of an arbitrary cut score. Cuomo’s reliance on tests to “prove” that “there’s ineffective teaching going on” put him in a box as more and more parents realized the tests were driving the joy out of their child’s schooling and the test results “proving” that school were “failing” were determined by state officials, not by their children’s performance on tests.

I keep hoping that someday someone in political office will stand up to this whole test-and-punish scheme and acknowledge that it is a failed policy. As noted in earlier posts, the reauthorization of ESEA was a golden opportunity for someone to step forward. Alas, we will have to wait for another decade or so to have the debate on testing.

In Defense of Reform, Personalized Learning, and Mastery

November 19, 2015 Leave a comment

Today Diane Ravitch cross posted an excerpt from a blog written by ME 4th grade teacher Emily Talmage, who decries the reforms being introduced into her state, reforms funded by Bill Gates and the Nellie Mae Foundation. In her post, which is a letter to Mark Zuckerberg imploring him to dissociate himself from Bill Gates,  and in the comments that follow, there are several negative comments about “reform”, “personalized learning” and “mastery learning”—all terms that have been expropriated by those billionaires who see education as the next place to make even more money.

The term “reform” and all its variants, which I at one time associated with Ted Sizer and other progressive educators, is now used by profiteers and the neo-liberal politicians who take their money and promote their ideas.

“Personalized learning”, a term embraced by the technology-based crowd who want to disrupt education, has also lost its original intention which was brought to life in Vermont as part of the Education Quality Standards (EQS). In Vermont, the idea was that all seventh grade students would develop a personalized learning plan (PLP) that would be the basis for their course selection in high school as well as learning activities that would take place outside of school. The notion was that this plan would help bring purpose to the coursework in school and tie the students out-of-school learning and activities to those happening in school. While you might use a computer to write this plan down and modify it annually (or as needed), it had nothing to do with the kind of computer-driven instruction described in the Talmage’s post or the comments that followed.

I also find it maddening that “mastery learning” is being conflated with behaviorism, because boiling that concept down to answering a series of multiple choice questions will corrode that phrase as well. In my way of thinking, mastery learning requires benchmark assessments that are administered when the student is ready to demonstrate that they have mastered a skill. As noted in previous blog posts, the driver’s test is the paradigmatic mastery test: If you fail it the first time– or the first five times— once you demonstrate mastery you get the same “certificate” as an individual who passed it the first time. In mastery learning, time is the variable and content is constant.

In looking back on my 40+ year career in public education I can see the source of these ideas in my first assignment. As a rookie 8th grade math teacher in an overcrowded junior high school in Philadelphia in 1970 I was handed a schedule that assigned me to four sections of roughly 35 homogeneously grouped students who I taught in over 20 different “classrooms” that included a small gymnasium, a section of the cafeteria, and a science lab with 24 lab stools. The book I was issued matched the city’s 8th grade pre-algebra curriculum. Most of my students could not perform basic operations… and given that I had 8-24, 8-30, 8-34, and 8-36… all sections in the lowest 1/3 of the age cohort… this was not surprising. The mismatch between my student’s skills and the expectations of the text book was gaping. My solution– which it took me several months to stumble upon– was to borrow a 3rd grade book of mimeo masters from my father’s best friend, who sold textbooks in the suburbs, and integrate them into a mimeograph “textbook” I wrote and issued to one of my classes. The discipline problems in that class diminished markedly from that point forward… and I began using modifications of this in the other three sections to the same effect.

The name for that kind of approach in 1970 (according to the observation report that was written by an observer of that class) was “individualized instruction”. Today, with the data collection and analysis capabilities, “individualized instruction” has been renamed “personalized learning”. The application of this aspect of “personalized learning” is limited to curricula that are hierarchical in nature, requires adept and timely intervention by a teacher, and is not intended to replace school. It could be a means of getting students out of the lockstep age-based groupings that lead to 8th grade teachers being required to offer pre-algebra to students who have not learned the basic skills.

Mastery Learning, a concept that was taking root as I was in graduate school in the  early 1970s, has always seemed unattainable because of the massive amounts of paperwork required to track each student’s progress. By definition it requires a hierarchy of skill levels and focusses on the assessment of those skills… but hierarchical skills exist in virtually every area of learning and performance and assessing learning and performance is a skill that cannot be delegated to a computer… unless, that is, the goal of the school is to measure only easy-to-measure content.

When “reformers” who advocate “personalized learning” claim that computerized assessments measure “mastery” three terms lose their meaning… and the factory school model is reinforced because the effectiveness of all of this “reform” is determined by whether students progress at a uniform rate based on their age… they are widgets moving along an assembly line, not sentient beings who need connection with others.


Florida School Board Chair Promotes an Alternative to Test-and-Punish Reform

November 15, 2015 Leave a comment

Washington Post blogger Valerie Strauss used her post on Saturday to reprint a thoughtful four page memo written by Steve Luikhart, the chair of the Pasco County (FL) School Board. In his memo, Luikhart decries the direction FL has taken in emphasizing standardized testing over formative individualized feedback provided by classroom teachers. Much of the memo outlines an ambitious plan he has to replace the current test-and-punish regimen, a plan based on research on how children learn. While I am not completely confident that his ideas could be implemented as presented, I am 100% certain his ideas are FAR superior to what is in place today and welcome any template that moves us away from arguing against the reform movement by arguing for a better paradigm.

The legislative picture on replacing the “reform” agenda is grim, especially in states like Florida where profiteering charter schools have gained a substantial foothold.  In states where for-profit schools are gaining momentum, legislators are not delivering the schools parents want, the kinds of schools their leaders believe are needed, the kinds of schools where teachers are valued, or the kinds of schools where the lives of children are the focal point of everyone’s attention. Legislators in these states are selling the public on the notion that there is a quick, easy and cheap way to “fix schools” and the for profit charter school lobbyists and their allies in the testing and technology fields are helping them with their sales pitch in two ways: by giving them the tools to promote the test-and-punish reform model and by making donations to the campaigns of any legislator who uses those tools to impose “scientific equations” onto educators.

As long as voters ignore the local and state elections— and any examination of the voter turnout for those elections makes it difficult to prove otherwise— those with money will find a way to get sympathetic legislators elected to office and the current factory paradigm with it’s reliance on testing of students in age-based cohorts will continue. We need to elect more thoughtful school board members like Steve Luikhart and replace the State legislators who promote the panaceas peddled by the reform movement. Until that happens, schools will continue to “fail” and more children, especially those raised in poverty, will be left behind.

Harvard Business Review Synthesizes Peter Drucker’s Perspectives on Automation: Life-Long Learning is Essential

November 14, 2015 Leave a comment

Roughly 20 years ago I read several books on management theory and found that almost all the writers on this topic based much of their thinking on the ideas of Peter Drucker, a prolific and insightful writer and thinker. Rick Wartzman, the Executive Director of the Drucker Institute in Claremont, CA wrote an essay for the Harvard Business Review titled “What Peter Drucker Had to Say About Automation” that synthesized his perspective on that topic, and they ultimately boil down to one idea: life-long learning is essential for any worker who wants to avoid becoming obsolete.

In the essay, Wartzman cites Drucker’s writings from a 1946 Harpers essay on the mechanization of cotton harvesting and concluding with quotes from his 1993 book The Post-Capitalist Society.  In 1946 Drucker wrote:

“It is easy—and very popular in the Deep South today—to see only one aspect of the technological revolution through which the Cotton Belt is passing: the removal of the dead hand of the cotton economy and plantation society, the establishment of a sound agriculture and of a better balance between industry and farming, higher incomes, better living standards, the end of sharecropping—in short the final emancipation of both white and colored from slavery. It is also easy to see only the other aspect: dislocation, the suffering, the uprooting of millions of people who will lose their homes and their livelihood.

However, the full picture, as in all technological revolutions, emerges only if both—the better life for those who can adjust themselves and the suffering of those who are pushed out—are seen together and at the same time.

In 1986, nearly four decades after observing the impact of the mechanization of cotton harvesting, Drucker observed the same phenomenon in the rust belt:

The “shrinkage of jobs in the smokestack industries and their conversion to being capital-intensive rather than labor-intensive, that is, to automation, will put severe strains—economic, social, political—on the system,” Drucker warned in his 1986 book The Frontiers of Management.

In 1993, nearly five decades later, Drucker underscored the need for everyone to adapt to technological advances by learning new and different skills, envisioning a role for both traditional schooling and corporations:

“School,” Drucker wrote in 1993’sPost-Capitalist Society, “has traditionally been where you learn; job has been where you work. The line will become increasingly blurred.”

Employers also have their role, including “active and energetic attempts at retraining for specific new job opportunities,” as Drucker put it. And each employee must step up and be ready to embrace what’s being taught—over and over and over again. “People have to learn how to learn,” Drucker advised. “No one is allowed to consider himself or herself ‘finished’ at any time.”

The highlighted sentences from Post-Capitalist Society resonate with me the most, and should be the basis for determining if a high school graduate is “ready to work”. If “learning how to learn” is the ultimate goal of schooling, the use of a single test to determine if a student is “ready to work” or “ready for higher education” is preposterous. Passing a test implies that the student is a “finished” product. The ability to “learn how to learn” cannot be measured with a single test. It requires initiative, independence, and insight, traits that a teacher can observe and, if given the chance, could document. The ability to pass a test measuring a prescribed set of skills requires compliance and conformity. It doesn’t require a teacher’s observation, only the grading by a machine. Compliance and conformity might land a job, but being able to remain in the workplace in the future requires a combination of initiative and the ability to learn independently. And only a skilled teacher can motivate a student to want to learn… but instead of training and valuing teachers who can motivate independent learning we are trying to replace teachers with computers that train compliant students to pass tests.

Goldman Sachs PreSchool Success Problematic on Three Counts

November 10, 2015 Leave a comment

A succession of articles appeared recently on Goldman Sachs’ “success” in preventing large groups of preschool students from requiring special education services through innovative approaches funded by a social impact bond. In a typically insightful post on her Mathbabe blog, Cathy O’Neill explains the design of a social impact bond and how they can be abused to the detriment of those who are presumably helped, by investors, and of science itself.

In the opening paragraphs O’Neill outlines how a social impact bond works:

The idea is that people with money put that money to some “positive” purpose, and if it works out they get their money back with a bonus for a job well done. It’s meant to incentivize socially positive change in the market. Instead of only caring about profit, the reasoning goes, social impact bonds will give rich people and companies a reason to care about healthy communities.

So, for example, New York City issued a social impact bond in 2012 around recidivism for jails. Recidivism, which is the tendency for people to return to prison, has to go down for the bond to pay off. So Goldman Sachs made a bet that they could lower the recidivism rate for certain jails in the NYC area.

Sounds like a good concept on paper… except for a few issues:

  1. Who defines what constitutes “recidivism”
  2. Who looks to see if Goldman is investing in other instruments that bet against the success of the bond
  3. When money is involved, who makes sure the “experiment” involving recidivism is done in a scientifically rigorous fashion?

Later in her post, O’Neill describes the social impact bond “success” touted by Goldman Sachs as evidence that these new instruments can solve thorny educational problems:

Here’s a big red flag on the whole social impact bond parade: Goldman Sachs was caught rigging the definition of success for a social impact bond in Utah. It revolved around a preschool program that was supposed to keep kids out of special ed. Again, it was hailed by the Utah Governor as “a model for a new way of financing public projects.” But when enormous success was claimed, it seemed like the books had been cooked.

Basically, Goldman Sachs got paid back, and rewarded, if enough kids who were expected to go into special ed actually didn’t. But the problems started with how find the kids “expected to go into special ed.”

Namely, they administered a test known as the PPVT, and if the kid got a score lower than 70, they were deemed “headed to special ed.” But the test was administered in English, when up to half of the preschoolers didn’t speak English at home. And also, the PPVT was never meant to measure kids for special ed needs in the first place. In fact, it’s a vocabulary test.

Unsurprisingly, many of the non-English speaking children did NOT require special education services when they got older because… they learned how to speak English more proficiently. That is, they didn’t require special education services later because they never should have been identified as likely candidates for those services to begin with!

MAYBE these social impact bonds would be a good idea if someone in the Utah State Department of Education was asked to determine the method for predicting future special needs children… but my hunch is that the Governor of Utah wanted to see this method work to “prove” that financial incentives can leverage solutions to difficult social problems and so avoided seeking advice from social scientists on the best means of screening for potential special needs students.

And here’s my other hunch: IF someone in the Utah State Department of Education determined the method for predicting future special needs children the funds needed to solve the problem through intervention would erode the profit margin… because early intervention requires a net increase in funding for schools and/or social services and not a reallocation of existing funds. Investors and politicians who think that there is enough money in the “inefficient” government system to solve problems rooted in poverty are engaged in magical thinking. The only way to reduce special education spending is to increase spending in regular education.

Eduardo Porter’s Tough Question for Public School Critics

November 4, 2015 Leave a comment

“School vs. Society in America’s Failing Schools”, Eduardo Porter’s column in today’s paper poses several tough questions for the critics of public education, all of which are framed in the initial two paragraphs:

Here’s the good news: American schools may not be as bad as we have been led to believe.

Ah, but here’s the bad news: The rest of American society is failing its disadvantaged citizens even more than we realize.The question is, Should educators be responsible for fixing this?

Throughout the column Porter offers evidence supporting the assertions made in the opening paragraph, drawing heavily on a report released last week by Martin Carnoy from the Graduate School of Education at Stanford, Emma García from the Economic Policy Institute in Washington and Tatiana Khavenson from the Institute of Education at the Higher School of Economics in Moscow, which suggests “…that socioeconomic deficits impose a particularly heavy burden on American schools.”

After outlining the impact of socioeconomic deficits on test scores, Porter offers counterarguments to the findings by Andreas Schleicher, the O.E.C.D.’s top educational expert, who runs the organization’s PISA tests whose results triggered the research by Carnoy et al.

“When you look at all dimensions of social background, the United States does not suffer a particular disadvantage.”

…As part of the PISA exercise, the O.E.C.D. collects information about parental education and occupation, household wealth, educational resources at home and other measures of social and economic status — and combines them into one index.

By that standard, fewer than 15 percent of American students come from the bottom rung of society. And yet, Mr. Schleicher found, 65 percent of principals in American schools say at least 30 percent of their students come from disadvantaged families, the most among nations participating in the PISA tests.

“I found this contrast between actual and perceived disadvantage so interesting that I intend to publish it shortly,” he told me.

This discrepancy is relatively easy to explain since “disadvantage” is often defined in schools by whether or not a student qualifies for free or reduced lunch… and over 50% of US school students now meet that threshold.

Schleicher and Carnoy do agree on one issue: parents of disadvantaged students should expect more from their children. This is a glib recommendation that is easy for policy makers to advance but far more difficult for schools to implement, especially when the parents of disadvantaged children have heard and absorbed the message that they are failures and heard and absorbed the message that the schools their children attend are failures.

Near the conclusion of the article, Carnoy contends that policy makers could learn more from comparisons between States that do well on assessments than countries that do well. But Scheicher disagrees:

Comparing the United States with other countries, he notes, allows researchers to identify particularly egregious deficits of American education.

There’s the wide disparity in resources devoted to education, which flows naturally from a system of school finance based on local property taxes. There’s the informal tracking that happens when smart children are grouped separately in gifted and talented classes while the less able are held back a year.

Teachers are paid poorly, compared to those working in other occupations. And the best of them are not deployed to the most challenging schools.

In a country like the United States, with its lopsided distribution of opportunity and reward, social disadvantage will always pose a challenge. What’s frustrating, Mr. Schleicher said, is “the inability of the school system to moderate the disadvantage.”

In this case, I wholeheartedly agree with Mr. Schleicher’s perspective, and the bottom line question they pose for politicians and policy makers: What steps are we willing to take to address the “particularly egregious deficits” identified as a result of the tests given to our students? And Mr. Schleicher’s identification of these deficits offers a clear answer to Mr. Porter’s initial question of whether educators should be responsible for fixing these deficits: NO!

NY Times Editorial Board’s Preposterous and Hypocritical Position on Tests

October 29, 2015 Leave a comment

Today’s NYTimes has an editorial that is based on the flawed logic they and legislators have used since the advent of NCLB and the “high stakes tests” that spawned the thinking behind NCLB. I couldn’t recount all of the flawed thinking in the essay in the space allowed for comments, but did offer this rejoinder:

President Obama’s 2% solution will only matter if tests are not used as the basis for closing schools and firing teachers. If a homeowner was told they would lose their home if they failed a test given in June why WOULDN’T they prepare for that test by studying the material on the test and taking preparatory tests that match the format on the June test? If the editors of the NYTimes were suddenly told they would lose their jobs based on a standardized test administered in June, why WOULDN”T they prepare for the test by studying the material on the test and taking preparatory tests that match the format on the June test? If the President wants to require fewer tests, he needs to abandon the notion that any one test is the basis for drastic actions like school closures and the firing of teachers.

Take the stakes out of the tests and all of the other tests will disappear.

Categories: Uncategorized Tags: , , ,