Posts Tagged ‘value added’

The Newly Appointed NYS Commissioner Favors VAM and “Collaborative Approach”

May 27, 2015 Leave a comment

I wish MaryEllen Elia well… but based on the article in the NYTimes announcing her appointment, I sense that in the coming weeks I may be finding some areas of serious disagreement with her OR she may be finding herself in disagreement with the Regents and/or the Governor.

In Ms. Elia’s most recent assignment as Superintendent of Hillsborough County Superintendent in FL, she was the recipient of a $100,000,000 grant from the Gates Foundation to improve the teacher evaluation system. Under the new system she implemented, 40 percent of teachers’ evaluations were based on “…their students’ improvement on tests” (VAM), and 60 percent was based on “….observations by principals and peers“. During Ms. Elia’s ten year tenure she Hillsborough also instituted a merit pay system that “…allowed some new teachers to earn more than veteran teachers”.  These “reforms” would clearly warm the hearts of the Regents and Governor Cuomo. However, Ms. Elia was also part of “…a group of Florida superintendents who asked the State Board of Education to suspend consequences for schools and students in the first year of the tests“, presumably because the transition was too fast. THAT position would contradict the previous Commissioner’s stance and would fly in the face of the Governor’s impatience. Her willingness to advocate for a phasing in DID win her the qualified support of the union leadership in NYS:

The president of the American Federation of Teachers, Randi Weingarten, offered tempered praise, saying in a statement that while the union was “opposed to high-stakes testing” and grading teachers on students’ test performance, “even when MaryEllen applied it as required under Florida law, she made collaboration her mantra.”

If past actions are the best predictor of her future, Ms. Elia is more likely to lose Randi Weingarten’s support than the Regent’s support… and VAM is likely to proceed on schedule in NYS. This paragraph in the Times article explains why:

She will make these decisions against a backdrop of low test scores, well-funded special-interest groups, angry parents and a State Legislature that has become more active on education policy and has been in conflict with Gov. Andrew M. Cuomo.

The Times may believe that the “well funded special interest groups” are supportive of teachers… but anyone keeping a balance sheet knows that the for-profit charter lobby has much more money than the unions and much more support in the legislature than those pesky parents who want their public schools to provide something more than test preparation exercises. I hope I’m wrong about VAM… but I’m afraid money will trump common sense and the needs of children.

Preparation for Tests is the “Work” of Schools. Everything Else is “Play”

May 17, 2015 Leave a comment

The NYTimes op ed section today features an article by David Kohn titled “Let the Kinds Learn Through Play“. In the essay, Kohn describes the recent push to make pre-school and Kindergarten more “academic” in an effort to address the (sic) failing public schools as described in this paragraph:

By many measures, American educational achievement lags behind that of other countries; at the same time, millions of American students, many of them poor and from minority backgrounds, remain far below national norms. Advocates say that starting formal education earlier will help close these dual gaps.

He then describes how this notion of “starting formal education earlier” has the opposite effect on students, citing one study that showed early academic gains are short lived and another that showed early childhood students who had “academically oriented” programs did worse than students who had “child initiated” learning experiences.

From my perspective the article’s main message was important and helpful to those of us who favor experiential student centered instruction over didactic teacher led approaches. But I felt the article had two overarching flaws: it reinforced the “failing public schools” meme (the above phrase “By many measures, American educational achievement lags behind that of other countries” is a case in point); and it underemphasized the fact that “play” is neglected at ALL levels.

The emphasis on standardization and efficiency do not support “play” of ANY kind. The authors of the Common Core assume that teachers and schools are accountable only for the development of those skills that can be measured with standardized tests. The Common Core also clings to the 1920s paradigm that the most efficient way to educate children is to batch them by age cohorts and measure their progress using standardized tests that are administered annually. To make matters worse, NCLB and Race to the Top assume that the students’ group performance on these annual tests is a reliable, valid and efficient way to measure school performance and teacher performance, and that each student’s performance on these tests is a reliable, valid and efficient means of determining their ability to learn. Because these tests have such an impact on the ratings of the school, teacher, and student, preparing for them becomes the focal point of schooling and anything else is superfluous and inefficient. The “work” in school is test preparation. Everything else is “play”.

Consequently, PE, Music, Art, libraries, and recess are all bundled together as unworthy of attention in school and, therefore, unworthy of funding. They’re not in the Common Core, they don’t have a battery of standardized tests to measure performance, and they all look like “play”. Maddeningly, the teachers who provide instruction in these “non-academic” courses are evaluated based on the student performance on standardized tests. The result? PE, Music, Art, Libraries all inject “Common Core” activities into their curricula so that students can do well on the tests. There is nothing sadder than witnessing students completing bubble tests in an empty gym, an art room with paints and clay in the cabinets, a silent music room, and a library with books on the shelves. Nothing sadder except a playground that is empty throughout the school day because the children have “work” to do.

And here’s what is especially frustrating: the tests the children are “working” to do well on are NOT valid or reliable measures nor do they measure what is IMPORTANT to learn in school… yet their importance to the lives of students, teachers and parents cannot be understated.

Standardization, tests, and efficiency are the enemy of creativity and are undercutting the future of public education. Here’s hoping that eventually the public will see the need to change our emphasis in public schools and allow us to move in a new direction.

Opt Out Movement Creating MAJOR Problems for Reformers

May 6, 2015 Leave a comment

Carol Burris, a NYS Principal who is a respected voice in the national education community, wrote a post for the Washington Post explaining how and why the opt out movement could lead to the unravelling of the “reform” movement. The bottom line is this: if enough students do not take the test the results will be invalidated and the basis for measuring school performance and teacher performance will collapse. This is why Arne Duncan and Meryl Tisch made threats to withhold funding and why Governor Cuomo is perturbed. And if you need an example of why NYS’s tests overreached, take a look at this excerpt from Burris’ post:

Here is a sample from the Grade 6 Reading test that was given in Virginia last year to measure the state’s Standards of Learning (SOL):

“Julia raced down the hallway, sliding the last few feet to her next class. The bell had already rung, so she slipped through the door and quickly sat down, hoping the teacher would not notice.

Mr. Malone turned from the piano and said, “Julia, I’m happy you could join us.” He continued teaching, explaining the new music they were preparing to learn. Julia relaxed, thinking Mr. Malone would let another tardy slide by. Unfortunately, she realized at the end of class that she was incorrect.”

That is certainly a reasonable passage to expect sixth-graders to read. You can find the complete passage and other released items from the Virginia tests here.

Contrast the above with a paragraph from a passage on the sixth-grade New York Common Core test given this spring.

 The artist focuses on the ephemerality of his subject. “It’s there for a brief moment and the clouds fall apart,” he says. Since clouds are something that people tend to have strong connections to, there are a lot of preconceived notions and emotions tied to them. For him though, his work presents “a transitory moment of presence in a distinct location.”

I will let readers draw their own conclusions.

I, too, will let you draw your own conclusions…

Parents of Gifted Children Leading Opt Out Movement. The Result? Lower Test Scores!

May 4, 2015 Leave a comment

Alejandra Matos’ recent Minneapolis StarTribune article describes an unexpected finding regarding the opt out movement in MN and the unintended consequence that resulted from that finding. It seems that in Minnesota the bulk of the opt out movement has come from parents of gifted children— which any experienced administrator could have predicted but seems to have caught MN politicians by surprise. And, as any statistician would the predict, the test scores in the schools attended by opt outs dropped and the scores across the state dropped.

I am not at all surprised that parents of “gifted” children are supporting their children’s decision to opt out of the tests— especially the 11th grade tests that have no academic consequences for them. The typical “gifted” 11th grade student is likely enrolled in multiple AP or honors classes, is striving to do well in each and every course to ensure that they have a sufficiently high class rank to get into a prestigious college, and is engaged in multiple extra-curricular activities. I imagine that instead of attending school to take a simple-minded pencil-and-paper examination that the “gifted” students are staying home to study for test that matter and/or spending time in the school library working on term papers.

I am not at all surprised that parents of “gifted” children and their children’s teachers are leading this, for they can see through what is happening and by their actions can make certain that the schools their children attend remain under public control and retain the wide array of course offerings that engage their children. As Matos notes:

The opt-out effort is being led by a growing group of parents and teachers skeptical of testing’s results and who believe that the push for more testing is driven largely by the nearly $1 billion testing industry.

“The system that we have is pitting teachers against students,” said Valerie Olsen-Rittler, a National Board Certified teacher in Minneapolis and a parent. “If you have teachers that are going to be judged based on students’ scores, that destroys the relationship between the teacher and the child.”

Matos later quotes pro-testing advocates’ concerns:

State and district officials say students have a legal right to opt out of exams, but say they are concerned schools will be unable to accurately track student progress or evaluate how well schools are doing in closing achievement gaps between white and minority students.

These concerns are based on the flawed premise that standardized tests can:

  • Determine which students are “gifted” (which is why is put the term “gifted” in quotations in earlier paragraphs)
  • Accurately track student progress
  • Evaluate how well schools are doing in closing achievement gaps between white and minority students
  • Determine if a particular teacher is meeting the needs of the students they teach

Matos writes that Parents and teachers who support the movement say this is an ideal moment to draw attention to the negative effects of testing on students, teachers and the entire educational system.”

And she is right— but some charter and pro-privatization advocates are seizing on the lower test scores to “prove” that Minneapolis Schools are ALL failing.

A new problem for Minneapolis school leaders is that the opt-outs have caused misleading student achievement data. Minneapolis schools have already been a target of charter schools and other groups that for years have seized on the district’s lagging test results.

Better Ed, a nonprofit group that has relentlessly advocated for the dismantling of the Minneapolis School District, highlighted the drop in Southwest’s scores almost immediately.

“Trouble at Minneapolis’ best high school,” the group wrote, highlighting the drop in math scores. “It’s troubling to see that Southwest’s students are regressing in a subject so critical for their futures.”

Better Ed Vice President Daniel J. Lattier said he was later told why the scores had dipped and agreed that the opt-out movement is concerning.

“I don’t think anyone claims standardized tests are perfect,” Lattier said. “I understand some of the reasons for the movement, but I haven’t heard what other form of accountability really looks like.”

I have a suggestion for Mr. Lattimer: read some of the articles in progressive blogs! Every blog I read has MANY ideas on how public schools can be held accountable… but most of them think that the first step along that path is to address poverty, which ALL tests indicate as the root cause of “failure” as measured by standardized tests.


NYTimes Op Ed Acknowledges Limits of Big Data… Now Politicians Need to Follow Suit

May 3, 2015 Leave a comment

Two data scientists, Alex Peysakhovich (Facebook) and Seth Stephens-Davidowitz (formerly of Google) wrote an op ed piece for today’s NYTimes titled “How Not to Drown In Numbers“. The article describes how Big Data ideas have swept the country using the concepts described in “Moneyball” to underscore the limitations of data and the importance of human observation and insight in making key decisions on personnel. As I read the opening paragraphs I hoped that the authors would take on education… and they did so near the end of the article. After offering the cheating scandals in Chicago and Atlanta as evidence of teachers spending their time “...worrying about gaming the test, (instead of) what was happening in class“, Peysakhovich and Stephens-Davidowitz suggest that times might be changing in education:

Education, despite all the debate about test scores, the Common Core and value-added methods, is actually moving in a similar direction as baseball and tech companies.

It’s gotten much less press than the test score debate, but there is also a huge national effort to collect and evaluate small data. Student surveys have proliferated fast. So have parent surveys and teacher observations, where other experienced educators watch a teacher during a lesson.

Thomas Kane, a professor of education at Harvard, told us, “School districts realize they shouldn’t be focusing solely on test scores.”

A three-year study by the Bill and Melinda Gates Foundation bears out the value of both big and small data. The authors analyzed value-added models, student surveys and teacher observations. They tested how to best predict student outcomes on both traditional state tests and more cognitively demanding challenges in math and English. When they put the three measures together into a composite score, they got the best results. “Each measure adds something of value,” the report concluded.

As at Facebook and in baseball front offices, small data can find holes in the big data. If a teacher raises her students’ test scores but students say she wastes a lot of time, and outside observers rank her poorly, this raises big questions. Conversely, if a teacher does not improve test scores but students say she inspires them and principals think she is imparting profound lessons, we may give her the benefit of the doubt. Most important, while big data can tell us whether certain teachers are helping their students, small data gives us the best hope to answer a crucial question: How are they doing it?

While Bill Gates is seen as the primary promoter of Value Added Metrics, this is not the first article I’ve read citing this research finding— one that effectively undercuts the notion that VAM should be the sole or even primary metric for determining a classroom teacher’s effectiveness. I am inclined to give people the benefit of the doubt in general, and I am inclined to think that Bill Gates, who has pushed mightily to eliminate polio and help fight mundane diseases in impoverished foreign countries is NOT trying to destroy public schools. I do have one bone to pick with Mr. Gates, however, and that is his failure to trumpet changes in his positions on education as loudly as he broadcasts his initial findings. Several years ago Gates saw the creation of small high schools as the best means for engaging students. When he found that size of the school was an incomplete metric he neglected to share that conclusions loudly… and so many districts are continuing to scale down schools despite recent evidence that it makes no substantial difference. Now we have governors like Cuomo claiming that Big Data should be used for 50% of the teacher’s rating based on Gates’ initial research, we learn that such a weighting is not based on science… it’s a politically convenient number.

But Peysakhovich and Stephens-Davidowitz’ article ultimately does a disservice to those of us who believe there is an overemphasis on testing. Their use of Kane’s quote and Gates’ research without informing readers that the vast majority of statisticians fail to see any evidence that VAM is valid leads readers with the impression that VAM is unquestionably the best means of using Big Data. VAM is bad science the same way that climate change denial “science” and “creation science” are bad science… and two data scientists would realize that and share that with their readers.

How Standardized Tests Undercut Parenting, Teaching, and Schools

April 29, 2015 Leave a comment

Huffington Post blogger Stacy Steinberg’s latests post, “Set Up to Fail: High Stakes Testing in Public Schools” describes the impact of testing on administrators, teachers, and especially parents. Steinberg asserts that standardized tests are designed to “prove” schools are failing knowing that the tests will prove what we already know: the schools serving children raised in poverty will “fail” while the schools serving children raised in affluence will “succeed”.:

High stakes testing sets a child up to fail.

Even when a school employs hardworking teachers and has the support of parents and local business partners, some schools serve students that live in homes that fall below the poverty line. Research tells us that the number one indicator of how a child will perform on a standardized test is whether or not that child lives in poverty. Our flawed standardized testing model sets even the most talented teacher up to fail.

Unlike many critics of the standardized testing regimen, Steinberg appreciates the bind administrators face, noting that they:

…are caught between trying to provide quality education to… students and maintaining the financial backing from the state to do so. Unfortunately, these goals are often mutually exclusive. High stakes testing sets up even the most altruistic school administrator up to fail.

But the lion’s share of her post is directed at parents. Noting that parents work hard to provide opportunities for their children to thrive and be creative before they enter school, once they arrive to school they find an environment that stifles creativity and focuses on test preparation:

You watch as important subjects like history, art, and music drop from the curriculum as these subjects are not covered by the common core curriculum. You know your child needs an empathetic and patient teacher, yet you watch as teachers shift focus because of standardized testing standards. Assets like empathy and patience work against our best teachers who know their performance review will be based on how much and how quickly your student is able to learn. Perhaps worst of all, your child takes her first standardized test and comes home reporting that she was required to sign a piece of paper promising not to talk to you about the test or the testing procedure itself.
This leaves Steinberg with a sense that she, as a parent, is being set up to fail! She concludes by encouraging parents to write to their legislators sharing their concerns and urging them to do whatever is possible to put an end to this testing. This compelled me to leave the following comment:
Before you write, you might want to see what the Presidential candidates and your legislators think of this approach to education… and in the case of the Presidential candidates who is funding them. Are they claiming “schools are failing” based on these test results? Are they supporting the state or corporate takeover of “failing schools”? Are they claiming the solution to “failing schools” is to give parents more choices WITHOUT requiring that ALL schools open their doors to ALL students?
At this juncture (Bernie Sanders has not officially announced his candidacy), every single Presidential candidate supports the continued emphasis on testing and at least 33 Governors insist that standardized tests are the best way to measure school performance. At this juncture NCLB is the law of the land and the pending federal legislation will not end the testing regimen. And, at this juncture states have invested heavily in the new computerized tests and expect to use those tests for the foreseeable future. At this juncture, it is not too late to change the course of public education, but the odds of making the change are approaching the odds of us taking action to address climate change.

The Media’s Responsibility to Improve Voters’ and Parent’s Mathematical Reasoning

April 25, 2015 Leave a comment

I hate to sound like a broken record when it comes to calling out the NYTimes… but I intend to take every chance I get to make the point that VAM is flawed and the Times is complicit in the public’s misunderstanding of that mathematical and statistical fact. Sunday’s column by Nick Kristof, “Are You Smarter than an Eighth Grader” gives me such a chance. The column offers three questions from a recent international test of eighth graders and used them as examples of how poorly our students fared as compared to students in other countries. This led me to offer the following comment:

This paper contributes to the public’s misunderstanding of mathematics and statistics by supporting flawed ideas like “value added” measures as a basis for measuring individual teacher performance despite the rebuke of the methodology by the American Statistical Association and by publishing test data on individual schools without explaining their statistical significance.

I could have made the response more political by noting that the Times reports clearly incorrect and/or incomplete mathematical information when it comes to budget proposals, giving column inches to budget balancing ideas that lack specifics or, in some cases, don’t add up at all. When the “newspaper of record” supports statistical measures that are rebuked by professionals in the field and fails to provide its readers with mathematically accurate facts it is failing the public far more than its schools who need to defend themselves against baseless and inaccurate charges of “failure”.

And then this morning I read a letter to the editor to the Lubbock Avalanche Journal from George McFarland, a local superintendent, pointing out how their media have jumped onto the “failing schools” meme without looking at the facts, which are:

For example, news media like to grab onto quotes that public schools are clearly failing because there are 146,000 students trapped in almost 300 failing public schools. However, considering that 146,000 students is 2.8 percent of the 5,151,925 Texas students, simple math can identify more than 97 percent of Texas public school students are not enrolled in “failing” schools.

Likewise, 300 schools represent 3.5 percent of the 8,574 public school campuses in Texas, meaning 96.5 percent of campuses are not “failing.” These numbers might suggest there are areas where public education can improve but certainly don’t necessitate the need to completely trash an entire system which is serving so many successfully.

Thankfully the newspaper published the article… but if they were doing their job every time a politician said schools were failing they would note that 96.5% are NOT failing… but that FACT undercuts the narrative that is stuck in the minds of readers and voters.