“Giving Doctors Grades“, an op ed article in today’s NYTimes by Sandeep Jauhar, describes the consequences of using simplistic metrics to determine the effectiveness of a complex operation: heart surgery. In the early 1990s, NYS decided to issue “Report Cards” to surgeons in an effort to provide easy-to-understand information on the ability of various medical practitioners. The result?
(T)he report cards backfired. They often penalized surgeons, like the senior surgeon at my hospital, who were aggressive about treating very sick patients and thus incurred higher mortality rates. When the statistics were publicized, some talented surgeons with higher-than-expected mortality statistics lost their operating privileges, while others, whose risk aversion had earned them lower-than-predicted rates, used the report cards to promote their services in advertisements.
This was an insult that the senior surgeon at my hospital could no longer countenance. “The so-called best surgeons are only doing the most straightforward cases,” he said disdainfully.
This sounded VERY familiar to me… and I left the following comment:
This wrongheaded method of measuring the performance of surgeons is analogous to the “Value Added” evaluation methods promoted by “school reformers” and adopted by Arne Duncan, Andrew Cuomo, the Regents, and host of other governors and State Boards. The standardized test scores used to “measure” teacher performance mirror the economic standing of the parents. Consequently teachers who choose to work with the most challenging students, like the surgeons who tackle the riskiest cases, could lose their jobs. Grading schools using test scores only serves to humiliate the entire faculty who choose to work with children raised in poverty. Both of these failed metrics have one thing in common: they are attempts to bring mathematical precision to fields of endeavor that are crafts more than sciences.
The notion that service organizations should be run like businesses leads to the need for “precise” metrics like mortality rates and VAM to be used in lieu of “the bottom line” so revered by businessmen. But service enterprises do not provide neat and tidy outcomes: they defy the kinds of measures that can be used to develop “stack ratings” or “grades” because they serve individuals who have different backgrounds, temperaments, and physical compositions. The desire to reduce everything to a single number to rank employees using some kind of “objective criteria” is ultimately a means to replacing the judgement of human managers with algorithms. It has not worked in the past and is unlikely to work in the future— unless the future is led by robots.
Fairborn, OH, Loses a Committed and Dedicated Teacher. Somewhere in Ohio An Affluent District Would Welcome Him
Valerie Strauss’ turned over her Washington Post Answer Sheet blog to Scott Ervin, a Fairborn OH third grade teacher who outlined his reasons for quitting as a third grade teacher after 15 years. From his description of his work ethic and dedication to working with the most challenging students in a school that serves children raised in poverty I am confident that there is an affluent school district within driving distance that will be happy to hire him… and in that district Mr. Ervin won’t have to put up with Ohio’s laws that pertain to “failing schools”. As I wrote in an essay published in Education Week several years ago, this is the form of “merit pay” that is already in place in public education.
I base my assertion that Mr. Ervin could land a job in an affluent district on my experience as the former Superintendent of an affluent district in NH surrounded by several districts that had “failing schools” full of dedicated teachers, some of whom would jump into our applicant pools whenever we had an opening. Why? Because they knew that teachers in our district did not have to worry about test results because our students scored at the high end of the bell curve and their year-to-year performances never put the school in jeopardy of failing. Mr. Ervin’s experience brought to mind a teacher we recruited from a nearby district to work with students who were not eligible for special education services but did require one-on-one attention because of their inability to “fit” in the classrooms. Through behavioral interventions we were able to provide these students with the support they needed to do the kind of independent work teachers assigned and parents expected. Such a position was affordable in our district in two respects. First, we had the resources to pay for the position (though it was questioned whenever we needed to consider budget cuts) and second, we did not have to devote any resources to “test preparation”.
I consulted in financially strapped areas of the state after I retired in 2011 and worked in many under-resourced school districts in the pre-NCLB era. In less affluent districts after NCLB the focus was on avoiding designation as a School In Need of Improvement (a “SINI” status) or, as happened over time, working to get out of a SINI designation. The SINI focus meant that every class was dedicated to preparing for the NECAP, the standardized test used to determine whether a school was “failing” or “succeeding”. Many of the administrators and teachers I worked with thought the use of tests to measure their schools was preposterous but they all accepted it as a “given” and worked tirelessly to get enough of their students over the NECAP “cut score” so that their school could get out of the SINI status…. but the practical reality was that even when the school was out of the SINI status it was still obsessed with maintaining that status by, you guessed it, doing well on the next round of NECAPS. I found this vicious cycle astonishing and completely wrongheaded since I had spent seven years in a district that effectively paid no attention to NECAP scores. I also saw that the focus on NECAP scores took time away from the focus on what was most important: the cultivation of the love of learning and the ability of students to work independently on projects that interested them. The obsession with testing was taking the joy out of school for students as well as teachers.
I hope Mr. Ervin continues to teach and has applied to districts in his region that are not “failing” and that a Superintendent in Ohio reads Mr. Ervin’s post, looks through their applicant pool, and invites Mr. Ervin in for an interview. He may be good enough to merit a job in that district… THAT’s the kind of “merit pay” we have in America today.
Over the past four decades there has been a trend in employment in our country: corporations, in an effort to improve their bottom line have moved away from full-time employees in the direction of “contingent workers”. An article in todays NYTimes by Noam Scheiber titled “Rising Economic Insecurity Tied to Decades Long Trend in Employment Practices” describes how this shift played out in corporate America this way:
Far-flung business units were sold off. Many other activities — beginning with human resources and then spreading to customer service and information technology — could be outsourced. The corporate headquarters would coordinate among the outsourced workers and monitor their performance.
The article described at length how this shift was advantageous for some highly skilled workers whose talents enabled them to earn high wages while working flexible hours. But it downplayed the impact on unskilled employees whose work was scheduled “efficiently” to help the corporations achieve a higher level of profit while viewing their labor as easily replaceable. And here’s the rub: as technology advances more and more iterative jobs will be eliminated and more and more unskilled laborers will be marginalized. Worse, as technology advances, so-called “robots” will be able to perform more and more tasks that now require “skilled” workers, further diminishing the workforce. Those “robots” may take the form of DIY devices like ATMs and scanners that enable shoppers to replace check-out clerks or may be literal robots that take orders and deliver food at restaurants replacing the waitstaff.
Public education has not been exempt from the outsourcing phenomenon. Just as the organization of schools mirrored the corporate organizations in place through the 1950s, today’s schools, particularly larger school districts, are moving toward the corporate models when it comes to the provision of support services. It is not unusual for a school district to outsource food services, custodial services, maintenance, technology support, bussing, payroll, and a wide array of testing programs. All of the jobs associated with these services were once staffed by school district employees who resided in the district and willingly paid taxes to support their local schools. Once the jobs were outsourced, however, the employees no longer had an allegiance to the school district and no longer had the assurances that their jobs would be secure.
Until the past decade or so, teachers have been exempt from this outsourcing phenomenon… but no more. One of the results of NCLB and RTTT is the “takeover” of “failing schools” by non-government organizations who, in many cases, were staffed by uncertified college graduates who could provide the iterative instruction needed to improve test scores which are the primary metric used to determine the effectiveness of schools. And here’s the rub: as teaching-to-the-test increases the skill level required by a teacher diminishes and the availability of on-line instruction increases. This vicious cycle mirrors what is happening in corporate America… and the results have not been beneficial for employees or children.
I believe there is a way out of this cycle for public education. If schools embrace the opportunity to use on-line instruction to provide iterative instruction and demand that teachers use their talents and skills to diagnose individual student needs, to counsel students, and to provide the “soft” skills that cannot be measured by standardized tests but are critically important in our democratic society they can emphasize the craft of teaching and provide teachers with the same opportunities (and wages) that skilled workers receive in the “contingent economy”. If teaching is reduced to training students to pass tests, however, teachers will quickly become as replaceable as Walmart employees and burger flippers.
As noted in many previous posts, public schools have been collecting massive amounts of data on individual students for decades… data that has been stored in stuffed file folders and various generations of microfiche and computer formats. This inconvenient and inconsistent method of data collection made it impossible to use group data to determine the effectiveness of teaching methods, to track an individual student’s learning, or to do systematic research in education.
The advent of cloud storage, the adoption of uniform learning standards, and the extensive use of standardized tests makes it possible to gather and analyze data systematically. This should be nothing but good news for teachers and parents… but as we’ve seen with the NSA, data collection has a dark side as well. Recent articles in the NYTimes and Atlantic describe the dilemma researchers and practitioners face in making use of the data that is now available: the reluctance of parents to have information about their children stored on line.
The Times article, “When Guarding Student Data Endangers Valuable Research” looks at the Data Dilemma from the research angle. As the writer Susan Dynarski notes, the data gathered is invaluable:
Educators parse this data to understand what is working in their schools. Advocates plumb the data to expose unfair disparities in test scores and graduation rates, building cases to target more resources for the poor. Researchers rely on this data when measuring the effectiveness of education interventions.
Noting that despite the fact that no one has hacked into the student data and despite the fact the student data is not a likely target for marketers, many legislators are proposing laws that would hamstring the efforts of researchers to draw on the data to gain a better understanding of what works and the efforts of teachers to use the data to personalize instruction. To use a phrase of one of my colleagues in Maryland, the legislators are using a shotgun to kill a mosquito. Her solution to this is to provide the Department of Education with the ability “…to impose serious penalties on districts and states as soon as they are found to have violated privacy regulations” noting that “…the states, districts and the courts then need to do the hard work of enforcing laws that protect student privacy.” A noble idea, but a non-starter in Congress who, even if they passed such a law to pacify indignant parents, would fail to provide the funding for enforcement.
The Atlantic article by Andrew Giambrone describes one way to solve this data dilemma. Given the government’s seeming inability to deal with this problem, and the given the demand for data analytics on the part of schools (e.g. a 2012 survey of educational professionals indicated that 80% of the respondents “…believed analytics would become more important in the future”), developing an acceptable means of defining appropriate use of data may fall to local districts working with eager vendors. Giambrone describes how this is happening across the country… and it calls to mind a Ted Sizer quote I used frequently: “How does change occur in education? Slowly, Carefully, and All At Once”. His concluding paragraphs underscore why the systematic collection of student data is a good idea… and why this change will happen slowly and carefully:
Jose Ferreira, the founder and CEO of Knewton, a New York-based company that develops adaptive-learning tools, says a lot of student data is going to waste right now; rather than being forgotten at the end of each school year or semester, it could be harnessed responsibly to drive learning outcomes. His company tracks students’ proficiencies across a variety of subjects, but will not share that information—even with teachers—unless explicitly authorized to do so by a student’s legal guardians.
“If you’re going to touch people’s data, it’s very important that the benefits be clear,” he explains. “‘Why should I let you collect my data? The benefits are fantastic? Now you have to reassure me you’re going to use it in a way I’m comfortable with.’”
Like Ferreira, I am convinced that reams of student data is going to waste.. but like the majority of parents, I am not yet comfortable with the way the data could and might be shared. That will take some time.
I read with dismay Elizabeth Harris’ article in today’s NYTimes titled “Tough Tests for Teachers, With a Question of Bias”. The article describes a nascent movement to require that teachers pass rigorous tests in order to get licensure. The article outlines the pros and cons of testing and indicates the racial disparity in the Praxis test results, and offers this paragraph as rebuttal to the critic of teacher testing:
But many public education officials view rigorous entrance requirements as crucial to improving student performance and ensuring a qualified teaching force in the face of uneven preparation programs. In a court document, an expert defending the ALST on behalf of the state is quoted as saying, “The purpose of a teacher licensure test is to protect the public from incompetent teachers.”
It has been four years since I was a Superintendent, but in the 29 years I served in that capacity I can only think of a handful of my colleagues who saw the Praxis test or tests like it as being a valid means of measuring teacher competence. Testing IS cheap and fast means of determining a candidate’s knowledge base of general content, but it was never clear to me that the general knowledge base required for physical education, secondary science, kindergarten, and art had much in common. Moreover, I had at least two instances where we hired a teacher who did extraordinary work as a substitute in the district on the condition that they pass the Praxis tests only to find ourselves releasing them when they could not pass the test: one was a PE teacher and the other was a Special Education teacher. Alas, the talents– the “competence”— they brought to the classroom were immeasurable by a pencil and paper test but I’m sure an expert in the State department and the State Board members slept well knowing they had protected the public from an “incompetent teacher”.
A recent article by Tampa Bay Times reporter Mariene Sokol summarizing the recent findings of a teacher survey done in Hillsborough County caught my eye. I hope it also caught the eye of data driven education reformers across the country because Sokol’s article provides hard evidence that teaching in schools serving children raised in poverty, particularly those with disengaged parents, is far more difficult than teaching in schools with affluent and/or actively engaged parents. The message of the findings is summarized in the first three paragraphs:
School districts offer cash bonuses. They hire teacher coaches. They appeal to the idealism of educators who want to make a difference.
But the proof is in their own data: It’s hard to teach at a high-poverty school.
There’s less buy-in from parents. Kids don’t follow the rules. There aren’t even enough computers. And staff turnover is sky high.
So the if the favored approach of the “reformers”, giving bonuses and assistance to teachers doesn’t improve morale in a school, what does? Near the end of the article is the answer:
Dunbar, a West Tampa medical and science magnet school, had some of the happiest teachers, with a composite score of 96 percent. But the percentage of its students receiving free or reduced-price lunches was relatively high at 83 percent.
It’s a small school, with only 287 students. Principal Sarah Jacobsen Capps also said she is deliberate about maintaining a culture of collaboration.
“We have constant conversations and reflections on what we’re doing,” she said. “We always talk about it all the time. Even after we saw the survey results, we asked, ‘Where else should we focus?’ “
Some reformers will read this and conclude that “choice” is the key because Dunbar is a magnet school. Others will read it an say that keeping schools small is the key. I read the article and come to the conclusion that three factors are at play here:
- Parent engagement: I am never surprised to read that magnet schools have better learning environments because one of the de facto entry requirements to a magnet school is parent engagement. When parents are engaged in the lives of their children and interested in their current well-being and future, children thrive in school. Note that parent engagement is actionable. It is something schools can foster and support and in the article it noted that schools who made an effort to engage parents saw an increase in their teacher’s satisfaction and an increase in the percentage of students who followed the rules in school.
- Student focus: I know from experience leading large districts that smaller schools like Dunbar do not have to focus on logistical issues nearly as much as large schools. With fewer buses, fewer mouths to feed in the cafeteria, fewer names to learn, and fewer opportunities for students to be disruptive it is easier for teachers to direct their attention to children. Indeed, in a small school with limited transience it is common for teachers to know the names and families of each and every child in the school. While size makes it easier to focus on students as opposed to logistics, it is possible for larger schools to keep the focus on teaching and learning each student with the right kind of leadership, which is the third element.
- Teacher-centered leadership: The principal at Dunbar seeks a “culture of collaboration”, which was illustrated by the way she handled the information from the survey. Instead of using a top-down method whereby the omniscient administrator explains and interprets data to the staff, the principal engaged her staff in “constant conversations and reflections“.
Small schools and magnet schools are easy to replicate and maintain the traditional separation of school and family life and the hierarchical organizational structure that is familiar to business leaders and politicians. Engaging disengaged parents, maintaining a focus on each and every student, and nurturing teachers are soft skills that are difficult to measure and require a change in the orthodoxy in schools…. but my experience and, I would contend, these data support that direction going forward.
An op-ed article by Mark Bauerlein in today’s NYTimes poses the question: “What’s the Point of a Professor?” The article contrasts the role of the professor in the 1960s and even the 1980s with the role today, and he finds that in today’s world the role is significantly diminished. Why?
When college is more about career than ideas, when paycheck matters more than wisdom, the role of professors changes. We may be 50-year-olds at the front of the room with decades of reading, writing, travel, archives or labs under our belts, with 80 courses taught, but students don’t lie in bed mulling over what we said. They have no urge to become disciples.
He laments the fact that more students view college as a means to earn more money, citing a finding of a survey given to entering freshmen:
One prompt in the questionnaire asks entering freshmen about “objectives considered to be essential or very important.” In 1967, 86 percent of respondents checked “developing a meaningful philosophy of life,” more than double the number who said “being very well off financially.”
…Since then… finding meaning and making money have traded places. The first has plummeted to 45 percent; the second has soared to 82 percent.
What Bauerlein fails to note in his essay is that the federal government wants to use job placement and lifetime earnings as the primary metric for determining the quality of a college. And he also overlooks the fact that efforts to monetize public education and post-secondary education has emphasized the utility of schooling over its inherent virtue. In effect, both the government and the private sector are reinforcing the anti-intellectual notion that college is only worthwhile if it is financially rewarding.
He concludes his essay with this lament:
When it comes to students, we shall have only one authority: the grades we give. We become not a fearsome mind or a moral light, a role model or inspiration. We become accreditors.
After reading the essay, I left this comment:
Our country’s obsession with standardization reinforces the notion that teachers and professors are accreditors and not mentors. When we place a higher value on test scores than on interpersonal relationships between teachers and students we set the stage for automated instruction to replace in-the-flesh teaching and learning. When we advocate measuring the value of a college degree by the earnings of graduates and/or workforce preparedness we replace idealism with utility. The good news for those who want to make colleges profit centers who serve customers is that these trends make operating a college more efficient. You can devise computerized curricula that guide students toward lucrative jobs and develop computerized assessments that determine if the students meet the entry standards for those jobs. When that’s completed you can eliminate those pesky professors who want decent wages and benefits and assurance of long term employment and put crazy ideas in student’s heads… ideas like critical thinking, for example.
There is a way teachers and professors could use computerized instruction to augment their classroom instruction and be more than accreditors. Indeed, in an ideal model the teachers could be the mentors, the individuals who “become the fearsome mind or moral light” for students, and it would be through providing the kind of writing tutorials that Bauerlein describes in his essay…. because writing more than any discipline requires intense human interaction and cannot be done algorithmically.