Posts Tagged ‘Testing’

Reformers’ Worries About Standardized Tests Too Little and Too Late

November 17, 2018 Leave a comment

Two days ago Chalkbeat’s Matt Barnum posted an article describing the epiphany of many reformers regarding standardized tests titled “In a shift, more education reformers say they’re worried about schools’ focus on testing“.

The epiphany is summarized in the opening paragraphs:

“If there is one office in every state I would want to get rid of, it’s the accountability office,” said Andre Perry, a fellow at the Brookings Institution who previously led a charter school in New Orleans. “I would replace that office with some kind of statewide coordination around personalized learning.” No one on the panel with him disagreed.

I think too much time, attention, and resources have been devoted to accountability systems that don’t produce outcomes for students that historically struggled,” Lewis Ferebee, the head of Indianapolis Public Schools, said later.

“The way we’re doing [assessment] now — that is so time-, age-, grade-based — is really constraining for those innovators that are developing models that will support all kids,” said Susan Patrick of iNACOL, an organization that promotes technology-based personalized learning in schools.

To no educators surprise, once No Child Left Behind mandated the use of standardized tests to determine whether a school was succeeding or failing and then tied those test results to the compensation of teachers and made the continued operation of the schools contingent on performing well on the tests, MOST of the teachers time, attention and resources were devoted to passing those tests. And to no educators’ surprise the students who struggled the most to pass those tests were children who came from homes where education was not as important as, say, figuring out where the family would sleep or where their next meal would come from.

And once standardized tests became the basis for judging schools, it became evident to reformers and politicians who were claiming these tests would “prepare students to enter the workforce” that it was necessary to ensure that the tests in every state were based on the same set of skills… which opened the door to the Common Core.

By the time the Obama administration had every state engaged in a Race to the Top, standardized tests were entrenched in the DNA of every school system in the nation and their importance was magnified.

So, nearly two decades later, the “reformers” who wanted a cheap, simple, and fast way to measure “school effectiveness” and “student success” have come to the conclusion that standardized tests, while cheap, simple, and fast, do NOT measure the effectiveness of schools or do an adequate job of measuring individual student learning.

But the tests cannot be abandoned as quickly as they were imposed… because there is not a quick, cheap and easy metric to take their place when it comes to measuring schools or individual student performance… and if it is impossible to do so how can a school be given a low grade and recommended for takeover by a private for profit charter chain? And how can a parent make an informed choice about the school they want to attend.

Mr. Barnum’s article concludes with this offer from a group that has long opposed standardized testing:

“I’m happy to hear that these groups are in fact grappling with and realizing some of the same problems we are,” said Andre Green, the executive director of FairTest, a group that pushes for a smaller role for testing. “Come talk to us.”

I doubt that FiarTest’s phone is ringing off the hook… and that’s too bad because they might have some insights based on what makes sense to teachers and administrators.


Those Opposing Personalization Based on Data Collection Fail to See Technology’s Insidious Trade-off

November 17, 2018 Leave a comment

Earlier this week I read a post by Diane Ravitch about a group of Brooklyn HS students who are protesting “Mark Zuckerberg’s Summit platform” used to personalize education in their school. Their protest was based on the following: some students played games on their computers; cheating was easy; teachers’ over-used computers; there were all kinds of technical difficulties, and the platform “… is collecting a huge amount of personal data from thousands of students without their knowledge or consent or that of their parents.”

Here’s a few reality check based on my experience in high school in the early 1960s:

  • My friends and I used graph paper we secured from the math classroom to play five-in-a-row tic-tac-toe throughout classes, engaging in tournaments we developed in homeroom
  • Some of my friends (not me, I swear!), devised ways to cheat on quizzes and tests… but almost everyone I knew (including me) used “flexible grading” for the “individualized” SRA reading programs that one progressed through by passing self-graded tests that were periodically audited by teachers.
  • Some teachers, especially social studies teachers, overused films to “teach” us about the wars that constituted their course of study

The equipment glitches that plague “Zuckerberg’s Summit platform” didn’t exist, but there were some days where we had more than one substitute teacher which meant we could play tic-tac-toe openly.

What we DIDN’T have was the privacy issue… but then we didn’t have the conveniences that come with the technology that students, parents, and teachers rely on today. And here’s the irony about those who complain about invasions of privacy: while they complain about “Zuckerberg’s Summit Platform” they are probably walking around with their cell phones inter pockets, purses or backpacks and, in doing so, providing all kinds of data. And if they are making any on-line purchases with any company, or streaming any videos or music of any kind, or using any social media of any kind, students and parents are providing a treasure trove of information to potential sellers.

This just in privacy advocates: We have evidently unwittingly made a trade-off: we get all the goodies technology offers us in exchange for information that can be used to market stuff to us.

My thought: We need to develop a new curriculum that teaches children how to ignore the propaganda that is the basis for advertising and the noxious politics in our country…. Maybe the tech billionaires can develop it, we personalize it, and develop a standardized test to see how well the children are learning it. Or maybe teachers can do that without the standardized testing part.

My Son-in-Law’s Description of the NYC High School Application Process Underscores Requirement for Parent Engagement

November 11, 2018 Leave a comment
My Grandson, Evan, is in eighth grade in NYC. This means he is in the throes of applying to high schools in that city, a process that requires much more parent engagement than applying for college, and a process that is much more involved than applying to college because there is no common application form. My son-in-law who has been fully engaged in the process took the time to provide a written synopsis of the process for his sister who lives in Colorado and was mystified by the fact that her nephew had to “apply for high school”. I know from my daughter’s sharing of her experiences on Facebook that most people in the country are unaware of how “choice” plays out in NYC… and when they see what parents are required to do they are astonished. Here’s my son-in-law’s overview in italics with some notations I’ve inserted in bold green and some phrases I’ve emphasized underlined in bold.
In NYC there are 8 “specialized” public high schools that are all very good schools and are spread across the 5 boroughs, some huge (Brooklyn Tech is the largest high school in the US with almost 6000 students) some relatively small (400-500 students). What these have in common is that admission is 100% based on a single 3-hour test–the SHSAT. Evan took that test a couple of weeks ago, and on the test form submitted our ranking of the 8 schools. Next spring the board of ed will run their algorithm on the test scores. Person with the highest score will be offered a spot at their #1 choice, then the next highest scoring person will be placed at their #1 choice, etc., until one or more schools fill all their spots, and some people start getting their 2nd choice, etc.. At the end of this process when each of the 8 schools have awarded all their slots, you can retroactively see a minimum “cutoff” score for each school. Evan took about 3 sample SHSAT tests, and his scores on two of them would likely be above the cutoff for at least a couple of these schools, so we’ll see in the spring whether he did well on the actual test. Of course the large majority of kids who take the test don’t get an offer from one of these schools. To summarize: one test determines whether a child qualifies to attend a “specialized” public school and not all children who take the test get into any school of their choice. 

Meanwhile, there are over 200 other public high schools in NYC, at least two dozen of which are also very good. These good schools are all “screened” schools, meaning they look at kids’ 7th grade report cards and state test scores and attendance records (and a few add their own admission test, essay, and/or interview into the mix) and rank applicants accordingly. Some of them also give preference based on what borough or neighborhood you live in, while some of them judge kids without considering where in the city they live. All NYC 8th graders have to submit a ranked preference list of 12 of these (non-specialized) high schools, regardless of whether they also took the SHSAT for the specialized schools.So the open houses I mentioned were a mix of specialized and non-specialized schools to help us submit the two lists of ranked schools. It’s a grueling process, and every kid comes home from school with a paper copy of this giant directory of schools to pore through:

To summarize: my son-in-law and daughter need to become familiar with all 200 high schools in the city to make an informed choice and, having done that background work, need to schedule visits to open houses to both determine and demonstrate their interest in the “screened” schools… and they need to have monitored my grandson’s work and attendance for the years leading up to 8th grade. 
The deadline for submitting Evan’s list is Dec. 3… Basically Evan has a shot at qualifying for Brooklyn Tech, which due to having the most slots generally has close to the lowest cutoff score for the specialized schools. And we have at least 8 smaller, boutique-y non-specialized schools to rank (mostly in Manhattan with one or two in Brooklyn) that we’d be very pleased with, as well as a couple of large Brooklyn “safety schools” that are more comparable to his middle school.. Of all the 20 schools ranked on the two lists, only one is in walking distance from our apt, and most are 20-80 minute subway rides away.  To summarize: there is no such thing as a neighborhood high school in Brooklyn. 
After all this hubbub, nothing much more will happen until March, when we’ll get a letter telling us: 1) SHSAT score 2) specialized school offer (if any) 3) regular school offer (if any). It is possible to end up with a choice to make between offers at a specialized school and a non-specialized school. It’s not impossible that when all is said and done he’ll be walking to that nearby school (Brooklyn Millennium HS), which is relatively young but very impressive. It’s also possible that he’ll be one of the hundreds of city kids going to really engaging schools butspending over 2 hours a day total on the subway to get there and back. Almost-worst case is that he’ll end up in a fine but ego-bruising safety school like his middle school. Worst-worst case is that he wouldn’t get into any of the schools ranked on either list, and would be arbitrarily placed in some other school that didn’t fill up during the first round of applications–… but I think its unlikely. To summarize: a young man who has a good attendance record, done well in his school work, and will presumably do well on the SHSAT, COULD end up not getting into any schools of his choice and may possibly have to settle for a school that is way down on his list of choices. 
The real bottom line in all of this is that the 10% of homeless children in NYC schools are highly unlikely to complete this daunting process…. nor are many of the 74% of the NYC students who qualify for free and reduced lunch.  It requires one resource that those parents lack: time. If a parent is looking for a place to sleep or looking for a better paying job they are unlikely to have the time to perform the kind of analysis my daughter and son-in-law did or the time to visit schools with their child that both my daughter and son-in-law devoted.
According to the Princeton Review, “Out of more than 28,000 students who took the SHSAT in 2016, only about 18% were offered a seat at a Specialized High School.” What the Princeton Review DIDN’T report and what has been underreported in the mainstream media and underemphasized by the “reformers” is this fact: there are roughly 75,000 students in the 8th grade cohort, which means that only 37% of the cohort took the test to qualify for the eight specialized schools and only 6.7% will get into one of those schools. How can “reformers” tout choice when only 37% of the children are taking the test that enables them to HAVE a choice and only 6.7% of those children will attend one of the “top schools” in the city?
I am grateful that my grandson has two parents who are willing and able to take the time to do the in depth research necessary to make an informed choice on his behalf. I wish those who espouse “choice” would realize that making a choice for schooling is inherently inequitable and unfair and stop insisting that it is a civil rights issue or a means of leveling the playing field for students. If “reformers” wanted to level the playing field they would advocate for fair housing, decent wages, and enough money to support the children in all schools in New York City…. and my Grandson might need to apply to a small group of schools that provide specialized programs for gifted and talented students but be confident that if he failed to gain acceptance there would be a high school within walking distance that would provide him with a robust, high quality college preparatory curriculum and a wide range of activities to participate in.

Medium Blogger Wendy Buchholz Provides the Naked Truth About Standardized Tests

November 10, 2018 Leave a comment

Medium blogger Wendy Buchholz offers a humorous but accurate analysis of standardized testing in her satirical riff on Hans Christian Anderson’s fairy tale titled “The Emperor is Naked! Hegemony in Education“. After recounting the familiar story of the wealthy emperor who is duped into walking naked in a parade by being convinced that he is wearing a glamorous outfit that only the intelligentsia can observe, Ms. Buchholz draws a parallel to standardized tests:

There is a similar fairytale being told in the public education system. It is labeled “standardized testing,” and it is, in fact, naked educational hegemony. This can be defined as a leadership or dominance of the policy makers and testing corporations over their consumers, advocating a standard of knowledge or ideology that is based on that which maintains their power. In this case, standardized tests are advocated and promoted as adequate measures of intelligence, knowledge and capability, and are maintained as the status quo among public educational leadership. This educational hegemony is dominating a generation of children.

The roots of cultural hegemony are found in the writings of Antonio Gramsci, one of the most important Marxist thinkers in the 20th century. Gramsci defined cultural hegemony as an ideology that maintained a capitalist state, which thereby normalized the ideas of those in power and maintained the current power structure.Power is not achieved through force, but rather the advancement of an ideology that becomes the “common sense” of the masses. Educational hegemony in the public system is built on an ideology that is largely constructed for the purpose of maintaining a power structure, using the vehicle of standardized tests.This practice of testing and test preparation has now fully clothed the public education system, enveloping a large portion of the time that a student spends in school. According to the Washington Post article, Confirmed: Standardized Testing Has Taken Over our Schools. But Who is to Blame? (October 24, 2015), this initiative, brought about in 2002 by the No Child Left Behind Act, consumes 20–25 hours of child’s life, every year. This does not include test preparation time. The average student will take 112 standardized tests from Pre-K through 12th grade. The standardized testing industry is a 1.4 billion dollar industry (Buchholz’ emphasis). That is not including the test prep industry, computer industry, tutoring, coaching, and the assortment of services that are necessary for the implementation of the standardized tests. And the results of standardized tests have never been shown to improve student achievement or teacher performance. (again, Buchholz’ emphasis) In short, the Emperor is, in fact, naked!

Ms. Buchholz is spot on in this analysis of how NCLB led to the takeover of public education, and I think she is correct in her view that standardized testing has the effect of reinforcing the current economic system since affluent children tend to score higher than children raised in poverty. But I am not convinced that most affluent parents see “the vehicle of standardized testing” serving as a means of maintaining the economic status quo nor do they fully appreciate how the political use of testing to advance privatization in lower income communities and neighborhoods ultimately works to the advantage of their children. The effects of the standardized testing paradigm are invisible, and, as Peter Senge would observe, we are prisoners to paradigms we cannot see. I am glad that Ms. Buchholz is showing how this paradigm plays out in public schools.


Campbell’s Law Confirmed as Public Colleges Chase High US News and World Report Ratings

November 1, 2018 Comments off

As noted repeatedly in this blog, what gets measured is a crucial element in what gets done, and, as  the use of flawed metric has had a devastating impact on the availability of affordable colleges. In “Undermining Pell, Volume IV“, New America researcher Steven Burd explains how a combination of reductions in State funding and the U.S,News and World Report’s ranking systems have diminished the opportunities for children raised in poverty to attend college, thereby undercutting the opportunities for them to achieve middle class earnings.

Mr. Burd opens the report describing how the state of Maine’s repeated budget cuts compelled it’s flagship college to seek out of state students by offering them scholarships. In doing so, the college attracted more out of state students, but it also witnessed a plunge in the number of “Pell grant” students, that is students who qualify for federal subsidies due to the low income of their parents. This, in turn, meant that fewer children raised in poverty were able to attend the State subsidized colleges, which often serve as the best opportunity for advancement.

Later in the report, he highlights how the US News and World Report’s ratings impact college admissions, with colleges offering more “merit” scholarships that effectively balloons the number of affluent students at the expense of low income students. The result: greater inequality of opportunity. As Mr. Burd writes in one of the concluding sections of the report:

To be sure, a significant number of public colleges and universities are staying true to their mission by keeping their prices low and/or providing generous amounts of need-based aid to the substantial number of low-income students they enroll. These schools are run by leaders like Nancy Cantor, the chancellor of Rutgers University at Newark, and Matthew Holland, the former president of Utah Valley University, who believe that public institutions should value inclusiveness over exclusivity.

But, unfortunately, for every Rutgers-Newark and Utah Valley, there are more public universities like Temple University that have lost track of their historic mission while worshipping at the altar of the U.S. News rankings and pursuing greater prestige. Many of these schools spend tens of millions of dollars lavishing scholarships on upper-middle income, mostly white students from the suburbs or other states, while students with the most financial need are charged a hefty price…

Mr. Burd offers several ways college aid could be transformed to reverse this trend before concluding with this call for policy makers and politicians to make colleges more affordable to low income Americans:

Regardless of which approach policymakers take, it is absolutely vital they act to put the brakes on the merit-aid arms race, which has done great harm to the college aspirations of low-income and working-class students.

For the good of the country, we must do all we can to ensure that colleges live up to their commitments to serve as engines of opportunity, rather than as perpetuators of inequality.

Will it happen? Unless States are willing to spend more for higher education and colleges are willing to ignore the bogus ratings system set up to “rank” them it won’t.

Campbell’s Law CAN Be Repealed… but Only By Introducing Multiple and Soft Measures

October 28, 2018 Comments off

Decades ago, in 1985 to be precise, I gave a presentation to teachers at the beginning of the school year that featured a slide that read “What Gets Measured Gets Done”. At the time, NH was about to launch some form of standardized test that was intended to be the end all for accountability and, at the time, I was advocating that our district devise multiple measures for accountability in order to avoid being held accountable based on a single, flawed measure. At the same time, my Assistant Superintendent and I conferred with the consultants from the standardized test company to see if there was some way we could use their norm-referenced test as a criterion referenced test (short answer: it was possible but only through convoluted calculations) and we persuaded the school boards to look at the test results through a criterion-referenced lens as opposed to the norm-referenced lens that we felt mis-represented the effectiveness of our schools since what was tested did not match what we were teaching. I don’t think we were familiar with Campbell’s Law at that time, but without being aware of it we were determined not to fall prey to it. What is Campbell’s Law? Fred Hess offers this definition in a recent Medium post:

Formulated in 1976 by social psychologist Donald Campbell, it reads, “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” Put simply: “When a measure becomes a target, it ceases to be a good measure.”

Without knowing of Campbell’s Law at the time, my Assistant and I both understood the aphorism “What Gets Measured Gets Done” and we both believed that the national norm referenced standardized tests being used in New Hampshire did NOT measure what we were teaching and, therefore, should NOT be used as a valid accountability measure. Fortunately, the district we worked in was led by school board members who understood this and, therefore, did not pay that much attention to the test results that were reported in the local media. As I witnessed throughout my career, one reason the school boards did not pay that much attention to the test results was that our schools invariably scored very high on the tests: parents were generally well educated, engaged with the schools, and pushed their children to succeed.

Throughout my career as a Superintendent I tried to get the school boards to develop multiple measures for the schools, because if the only measure of success is a nationally normed test score the only thing students will be exposed to in class is a curriculum that is focussed on what is tested… and standardized tests cannot measure what is really important: the development of a joy for continuous learning thought one’s life.

I still believe “What Gets Measured Gets Done” and still believe that norm-referenced standardized tests have a place in measuring a school’s effectiveness… but given the high correlation between parent income and education and test scores it seems foolish to equate high test scores with quality. And given the very tenuous correlation between high test scores and college success and/or earnings, it seems even more preposterous to use those scores as the sole metric for measuring quality. But we love to rank and compare in our country, and norm-referenced tests give us an easy way to do so…. and so the beat goes on….

Categories: Uncategorized Tags: ,

ACT Results Show Bi-Partisan Premises Behind NCLB, RTTT, ESSA are Flawed… But They Will NOT Be Easily Overturned

October 18, 2018 Comments off

The latest ACT results are worse than ever, which John Merrow believes might drive a final nail in the coffin of the premise that annual high stakes tests will improve schools… but if last year’s debates in Congress and the ongoing debates in state legislatures are any indication there is no likelihood of changing the thinking on accountability any time soon. Why? Because ESSA delegated accountability to States and at this writing 33 states are controlled by GOP legislatures, many of whom are using the ALEC playbook to craft legislation and frame the debates about public education in their states. Add to that the ongoing debates about how best to “harden” schools and the bandwidth for debates about public education is used up.

New Hampshire where I live is a good case in point. In 2016 voters elected GOP candidate Chris Sununu as governor and elected GOP dominated legislature. Once elected, Mr. Sununu replaced widely respected Commissioner of Education Virginia Barry, a Ph.D educator, with Frank Edeblut, a business executive who homeschooled his seven children and ran to the right of Mr. Sununu in the primaries. As a result of the 2016 election there has been no discussion whatsoever about moving forward with a creative accountability plan Dr. Barry developed, a plan that was not exclusively reliant on standardized testing. Instead, the GOP Governor and GOP controlled State Legislature are trying to pass laws that would expand the use of Education Savings Accounts (ESAs) for parents who want to educate their children in private sectarian schools. To fund these ESAs, the GOP planned to divert funds from an equalization formula developed by previous legislatures in response to a court order that would provide more support to property poor districts so their students could meet the “adequate education” mandated by the State Supreme Court. The GOP governor’s solution? Pass a bill that would preclude the courts from intervening on issues involving public school funding and expand choice. The Governor’s thinking? If the parents of students who resided in property poor towns had the opportunity to use tax free savings to take their children out of “failing government schools” and place them in any school they wished their children would ultimately benefit.

Added to the mix of ALEC bills designed to facilitate vouchers that will supposedly allow children who reside in property poor towns to escape the “failing schools” in their community is the ongoing debate on how much to spend to “harden” public schools to make them safe from shooters. This debate about school safety is a double whammy for public education: it inevitably results in diverting funds away from making capital improvements in outdated schools, many of which are located in property poor towns; and it reinforces the notion that public schools are inherently unsafe, making the push for de facto vouchers to attend private schools more politically acceptable.

Because of the ongoing debates on vouchers and school safety, debates on the virtue of standardized testing are pushed to the sidelines. Indeed, the need for these tests is largely settled in minds of most voters. Didn’t voters need to pass test to pass courses that got them promoted to the next grade level and earn a diploma? Didn’t voters who went to college have to attain a minimum score on the SAT to gain acceptance to their higher education? Doesn’t the military and civil service use tests to sort and select applicants? Why, then, doesn’t it make sense to use tests to determine if schools are successful?

At the conclusion of his article on the decline in ACT scores, Mr. Merrow writes:

It’s past time for progressives to speak loudly in support of strong public education….as well as other social initiatives that will address homelessness, hunger, and lack of health care.  Schools don’t function in isolation, not when–for example–about 10 percent of New York City’s public school students are homeless.

I completely agree and persist in writing this blog to that end… but, I don’t see many politicians at any level speaking up for public education or “…social initiatives that will address homelessness, hunger, and lack of health care“. The Social Darwinists in the GOP want to drown government in a bathtub and the neo-liberals who dominate the DNC are comfortable with privatization of public services or relying on the goodwill of philanthropists… and NO politician in EITHER wants to advocate for the higher taxes that would be needed to underwrite these social needs. And alas, as the cold analysis outlined above indicates, I do not see much sentiment today among rank and file voters for “social initiatives that will address homelessness, hunger, and lack of health care” because they know that such initiatives will cost them money.

My bottom line is that unless we reframe the debates about public education away from “choice” and the debates about social initiatives away from their cost we will continue on the path we are traveling and inequality will persist. We need to talk more about the common good and less about the virtue of selfishness.