Posts Tagged ‘Measurement’

NYC Tracking System in the Spotlight… and It Is NOT a Pretty Picture

June 18, 2018 Leave a comment

Over the past several days, numbers articles have appeared in the NY media praising or assailing Mayor Bill De Blasio’s call to expand the number of minority students in NYC’s elite schools by de-emphasizing the SHSAT tests that serve as the de facto sole metric for admission. Today’s NYTimes features an article by Winnie Hu and Elizabeth Harris that brings to light the fact that the kind of screening the exists to gain entry to the elite high schools permeates the entire city school system…. and that screening underpins the re-segregation that is taking place in that city and across the nation. Titled “A Shadow System of Tracking by School Feeds Segregation, Mss. Wu and Harris’ article opens with these startling paragraphs:

No other city in the country screens students for as many schools as New York— a startling fact all but lost in the furor that has erupted over Mayor Bill de Blasio’s recent proposal to change the admissions process for the city’s handful of elite high schools.

One in five middle and high schools in New York, the nation’s largest school district, now choose all of their students based on factors like grades or state test scores. That intensifies an already raw debate about equity, representation and opportunity that has raged since Mr. de Blasio proposed scrapping the one-day test now required to gain entry into New York’s eight elite high schools. Black and Hispanic students are underrepresented in many of the most selective screened middle and high schools, just as they are in the specialized high schools.

I’ve witnessed this screening mechanism as a grandparent of a NYC seventh grader. My grandson is one of the 20% of middle schoolers whose parents “chose” a school for him, a process that was arguably more daunting than applying for college since there is no common application form and one of the factors for admission to a de facto selective middle school is a parents willingness and ability to attend evening orientation sessions for each school a child is considering. This phenomenon is described in the article, using one parent’s experience as a proxy for thousands of parents cross the city:

Edwin Franco, a father of two girls who lives in the Bronx, said that too many selective schools cherry pick the best students — and deprive everyone else of opportunities. “They’re almost like a factory,” he said. “They’re churning out high-performing kids who are doing great while the rest of the kids are trying to figure it out on their own because they don’t have the same resources.”

….And now as many coveted middle schools screen, the competition has moved down to that level as well. Mr. Franco attended neighborhood schools in Washington Heights, and he only went through a selection process for high school. Both his daughters have already been through screening for middle schools.

“As a parent, I’m seeing the same level of intensity to get into middle school,” he said. “That’s what baffles me, middle schools are just as competitive as high schools.”

Mss. Wu and Harris provide a “history” of this tend toward screening, attributing its acceleration to the Bloomberg administration when all eighth grade students were compelled to “choose” their high school and the high schools marketed their programs in an effort to entice parents to select them:

Students rank up to 12 choices, and then get matched to one school by a special algorithm. The idea was to allow students to escape failing neighborhood schools and apply anywhere they chose.

…But as students increasingly chose their schools, the system evolved so that many schools became the ones choosing the students.

The number of high schools that admitted students only through academic screening — including the specialized high school exam, other tests and grades, or auditions — has more than tripled to 112 schools in 2017 from 29 schools in 1997, according to an analysis by Sean P. Corcoran, an associate professor of economics and education policy at New York University. Screening requirements vary from school to school, but the most sought-after schools often require at least a 90 average.

“You’ve set up a system of competition among high schools in which the easiest way for a principal to win is to select the students who are best prepared,” Richard D. Kahlenberg, a senior fellow at the Century Foundation, said. “Certainly having that market-based ideology — without guidelines for equity — appears to have accelerated the growth of screening.

From my perspective, even if Mayor De Blasio’s effort to limit the use of standardized test scores fails in the NYS legislature as appears to be the case for THIS session, by raising this topic now he could conceivably make the commodification of schools a campaign issue in 2018… and THAT would be a tremendous public service. The more parents understand the inter-relationship between choice and screening and the consequences of screening, the more likely it is that public schools might abandon the practice of sorting and selecting and replace it with funds to improve all public schools.


GreatSchools” Not So Great Premise: Standardized Test Mirror “Greatness”

June 11, 2018 Leave a comment

I have often blogged about the absurdity of rating schools based on easy to collect data, especially when that data is standardized test scores. A recent Medium post by Ali McKay, The Problem with “GreatSchools”, describes the flaws with the rating algorithm that “service” uses. Ms. McKay, who describes herself as “A white lady with kids digging into the practices of equity and anti-racism”, decries the Great Schools ratings. After describing the warmth and inclusiveness of her low rated racially and socio-economically integrated school, (by GreatSchools scheme), she offers this insight:

So what, exactly, is GreatSchools measuring? Mostly socioeconomic status, it turns out. In fact, Jack Schneider, an historian and researcher who studies schools, has written that factors the schools can control usually explain only about 20% of test scores. That means at least sixty percent of test scores is determined by socioeconomic status. Low income students will tend to score lower and high income students will score higher — and this is regardless of where they go to school. Much has been written about why, but, as just one example, researchers have found that poverty affects kids’ language environments. And, middle and upper class parents are, from day one, cultivating their kids’ language and other skills, setting them up to stay in the middle or upper class.

Ms. McKay, in the spirit of fairness, does note that GreatSchools is aware of the problem and attempting to address it:

GreatSchools seems to be aware that there may be a problem, and changed their ratings late in 2017 to include an equity component. This component accounts for 28% of a school’s rating… Their website says:“We believe that every parent — regardless of where they live or how much money they make — needs reliable information in order to ensure their child is being served by their school.” They have many pictures of Black and Brown families on their site.

Ms. McKay doesn’t “do the math”, but clearly the 28% factor is mathematically unlikely to identify a “low performing school” that effectively differentiates instruction into a higher classification. It DOES provide a fig leaf to indicate they are open to data beyond standardized test. But, as Ms. McKay notes elsewhere in her essay, it is a very small fig leaf given that:

…(standardized test) scores… account for 47% of GreatSchool’s school rating for elementary schools (and a whopping 72% if you add in their ‘Student Progress’ on tests factor). This means that (their ratings) are mostly telling you to find high socioeconomic students and avoid lower socioeconomic students (and English language learners, kids who qualify for special education services, and so on . . .).

So if these scores are only a proxy for affluence, what is a parent to do if they are seeking a school that includes a mixed demographic? Ms. McKay offers a common sense approach:

Take the two tour pledge: set foot inside two schools. You wouldn’t buy a house without going in it, so why do so with your child’s education? When we were deciding on our current school, we toured and we talked to teachers and parents. It didn’t take that much time, and walking around and seeing the actual people in the building was the most important factor for us.

Second, remember that parents tend to pass along the dominant narratives, whether they are actually true or not.They will tell you a school is “good” or “bad”, even though they might not have ever been in the school they are talking about… Researchers like Jennifer Jellison Holme and others have found this to be true(i.e. that families listen to and value a school based on what other privileged parents say about it).

And then, investigate your values and your goals for your kids. I am guessing your goals for your kids when they are 50 is not that they had high test scores. Like me, you probably want a lot more than that for them. Like me, you might be anxious about academics or anxious that not being around high achieving peers or watching screen time at school sometimes (gasp!) will hurt their prospects as adults in a competitive world. Anxiety is a small price to pay for seeking justice and dismantling systems of segregation and racism. And, it makes me feel icky but it bears repeating: socioeconomically advantaged kids will get high test scores wherever they are, because of the luck of their birth.

From my perspective we need more parents to take on that icky feeling and acknowledge that where their kids go to high school will have less bearing on the household they come from and the friends they make when they are in school… and that friendships with children of different races and socio-economic status are only possible if their children attend schools that are not economically and racially homogenous.

And here’s the challenge for GreatSchools and the education reformers who help underwrite it: choices about schools would vanish IF public education was funded adequately and affluent parents acknowledged that their children would not suffer if they attended school with those of other races and economic backgrounds. That was the vision of our founders, who hoped that democracy and upward mobility would be maintained through a public school system that served ALL children equally.

Bad Metrics Not Limited to Education: Employment Rates Mis-measure Our Economy Too

June 9, 2018 Leave a comment

Earlier this week, President Trump effectively released the employment figures before the official announcement and, in so doing, reinforced the notion that low unemployment rates are a sign of economic well-being. But, as Paul Constant wrote in Civic Skunk Works immediately after the release of the employment figures, that is not necessarily the case and, of late, has increasingly NOT been the case. Here’s the nub of his argument:

…if you just report on numbers, it’s very easy to fall into a Trump-friendly video-game mindset, in which larger numbers are an unalloyed good to be accrued at all costs…all these… journalists didn’t ask the most important question of all: we know the quantity of jobs. But what about the quality of those jobs?

Mr. Constant then produces reams of evidence that the quality of jobs in the “new economy” is awful:

As Derek Thompson argued at The Atlantic back in 2012, America’s postwar economy has shifted dramatically. Since the 1950s, he reports, “The manufacturing/agriculture economy shrunk from 33% to 12%, and the services economy grew from 24% to 50%.” And as most anyone who’s worked in the service economy knows, there are an awful lot of awful jobs—low-wage, part-time, no-benefit kinds of jobs—in service.

But this is not just about Walmart. Service jobs don’t have to suck—and many don’t. But I could sit here and list stats all weekend long proving that quality jobs in America are disappearing:

And on and on and on.

The fact is, sometimes in the 1970s America made the switch from high-quality, high-wage employment to low-quality, low-wage employment—and the shift is getting progressively worse.It’s gotten so bad that Axios recently revealed that CEOs openly admitted that the American worker isn’t getting a cut of the economic prosperity anytime soon: “executives of big U.S. companies suggest that the days of most people getting a pay raise are over, and that they also plan to reduce their work forces further.”

The report that Donald Trump touted today only counted the number of jobs created, not the quality of those jobs.

The truth is, this isn’t a jobs story at all. It’s an inequality story.

Mr. Constant concludes his essay with this compelling insight:

By blindly promoting economics numbers as though the highest score is all that matters, we as Americans are agreeing that the most important thing, above all else, is being employed. Never mind if you have to work two or three part-time gigs to pay the rent. Never mind if none of your employers provide health insurance. Never mind that workers are too tired and stretched too thin to find a new job, or to get training that might improve their conditions. Never mind that jobs which were once considered good careers are now paid a pittance.

When we blare the news of a great new jobs report—no matter which party is in power—we are advancing the narrative that as long as we hit our marks, nothing else matters. A job is a job is a job is a job.

Except that’s not true. Gradually, over the last half-decade, and without our consent, the deal has changed. Eventually, no amount of deft media manipulation will be able to hide that fact.

What does this have to do with public education policy? A paraphrase of that first paragraph answers that question:

By blindly promoting standardized test scores as though the highest score is all that matters, we as Americans are agreeing that the most important thing, above all else, is doing well on those tests. Never mind if you forfeit art, music, PE, and play for test preparation. Never mind if none of your school excludes students who score poorly on tests. Never mind that students are taught only what can be tested and fail to learn the soft skills that are needed in a well functioning democracy. Never mind that in the quest for high test scores we sacrifice childhood completely. 

Gradually, over the last decade-and-a-half we have made a decision to conflate good schools with high test scores and no amount to deft media manipulation can hide that fact.


NYC Mayor De Blasio Promises Change to Elite HS Admissions— Will State Allow it?

June 3, 2018 Comments off

In the byzantine governance structure of NYC, changing ANY policy is extraordinarily daunting and fraught with not only local politics but also State politics. In the coming weeks, NYC residents (and bloggers interested in social justice) will get a lesson this governance structure as NYC Mayor De Blasio attempts to overhaul the existing admissions system for eight “elite” specialized high schools in the city, a system that effectively denies minority students an equal opportunity for admissions.

When he ran for office in 2013, Mr. De Blasio pledged to change the admissions procedures for the eight “elite” high schools in the city where the sole screening mechanism is the Specialized High School Admissions Test or SHSAT. As Elizabeth Harris reported in yesterday’s NYTimes, the mayor is now planning to make good on that promise:

“The Specialized High School Admissions Test isn’t just flawed — it’s a roadblock to justice, progress and academic excellence,” Mr. de Blasio wrote in an op-ed published Saturday on the education website Chalkbeat.

“Can anyone defend this?” he continued. “Can anyone look the parent of a Latino or black child in the eye and tell them their precious daughter or son has an equal chance to get into one of their city’s best high schools? Can anyone say this is the America we signed up for?”

The most significant change Mr. de Blasio proposed was replacing the test, called the SHSAT, with a new method that would admit students based on their class rank at their middle school and their scores on statewide standardized tests. That change would require approval from the State Legislature, which has shown little appetite for such a move. A bill outlining those changes was introduced in the Assembly on Friday.

Mr. de Blasio announced another, smaller change on Saturday, one the city can do on its own. Beginning in the fall of 2019, the city would set aside 20 percent of seats in each specialized school for low-income students who score just below the cutoff; those students would be able to earn their spot by attending a summer session called the Discovery program. Five percent of seats for this year’s ninth graders were awarded this way, the city said.

Mr. de Blasio said that if both reforms were enacted, 45 percent of students at the eight specialized schools would be black or Latino.

If you asked a typical NYC resident who controls their schools, they would answer: “the mayor”. But as I’ve read about NYC’s schools (in part as an interested grandparent) I’ve discovered that nothing is straightforward in the way schools are governed… and this situation is a perfect illustration.  To paraphrase Mr. De Blasio’s question in his op ed piece: “Can any Upstate legislator look the parent of a Latino or black child in the eye and tell them why they would not support a policy change that would enable their precious daughter or son to have an equal chance to get into one of their city’s best high schools? Can anyone say this is the America we signed up for?”

There is a gubernatorial election underway in NYS. The DNC, which pledged to stay out of State races, broke its promise in NY, throwing their support behind the incumbent Governor, Andrew Cuomo. Mr. Cuomo has repeatedly stymied Mayor De Blasio’s efforts to avoid the expansion of privatized charter schools and his efforts to secure more funds for schools. Mr. Cuomo and the DNC are about to face a litmus test. The Governor and the DNC now have an issue before them that would not cost the taxpayers in the State a dime and would provide social justice for impoverished minorities in the city. Will they offer full throated support for the change Mr. De Blasio is advocating even though “…the State Legislature… has shown little appetite for such a move.” or will they remain silent, thereby supporting the status quo argument that test-and-punish reform is the best way to achieve social justice? From where I sit, anything less than full-throated support from the Governor’s office will send a message to every teacher in the state that standardized tests are the best and only way to measure “success”. I am sure I am not the only one who will be monitoring this legislation in the weeks ahead. I hope the Governor and the DNC do the right thing and give the Mayor their unqualified support— because he IS doing the right thing.

John Merrow Identifies as a “Progressive”, Offers Advice on How “Progressives” Can Gain Traction

June 2, 2018 Comments off

John Merrow’s Changing Conversations post on May 30 divided the education world into four identifiable groups: “Devosians,” ‘School Reformers,’ those who aren’t involved at all, and progressives and briefly defines the first three groups before offering an extended analysis of how “progressives” need to change their approach in order to gain more traction in the ongoing political debate about how schools should function. Here’s a brief overview of each group Mr. Merrow defines:

1) ‘The DeVosians,” are “…supporters of Education Secretary Betsy DeVos and her campaign to redefine ‘public education’  to include every type of school known to man–plus home schooling.”  Their ultimate goal is to end public education as we know it.

2) The ‘School Reform’ crowd is the bi-partisan group who imposed data-driven approaches to public education in the name of providing equal opportunity for all children. As Mr. Merrow writes, their legacy after nearly two decades “ nothing to be proud of: declining test scores, widespread rigging of both achievement and graduation data, an exodus of teachers from the field, and a dramatic reduction among young people in interest in becoming teachers.” With NCLB, RTTT, and now ESSA, this group is still steering the direction for public school.

3)  Those who are not paying attention to the ongoing struggle to control public education, which is the largest group of voters. Mr. Merrow writes: “At most, only 25% of households have school-age children, and most of the 75% pay little attention to education issues.  They are the key to real change, in my opinion.”

4) And the “progressives”, who believe in public schools but also want to see them change direction from where they are now. By Mr. Merrow’s definition, “progressives” oppose sorting and selecting based on test results and favor instruction driven by the curiosity of children as opposed to the imposition of topics and timelines by adults.

As readers of this blog realize, like John Merrow I fall into the “progressive” category. And I concur with his call to action for those of us who desire a change in direction away from the test-and-punish-sort-and-select model in place now:

Now let’s get to work on creating a genuine paradigm shift. For that to occur, at least three things have to happen.  One, we need to reject the language of ‘school reformers’ in favor of a more precise vocabulary.  Two, we need to change the conversation from hackneyed terms like “learning for all” to more dynamic language like “discovery” or “knowledge production.” And, three, we must get outside our own echo chamber and engage with the 75% of the population that does not have a direct stake in schooling.

While I DO agree with Mr. Merrow’s call to action, I fear that it will not be enough. Two things I’ve witnessed over my 29 year career as a public school superintendent and six+ years as a blogger: first, those who control the debate in the media will expropriate any meaningful terminology and make it into hollow phrases; and second, engaging the disengaged may require those of us seeking change to consider expropriating the arguments of the Devosians and reformers that our schools are failing.

To illustrate my first point, one need only recall that in the early 1990s if one favored “reform”, they supported the ideas about schooling advanced by Ted Sizer and other educators who favored student-centered approaches that are very much like those “progressive” ideas in this post. A “reformer” in that era also favored racial and economic de-segregation, funding equity, and more spending on public education. At some point the Democrats joined the Republicans in reaching a consensus that “throwing money” at the problem was unnecessary and that the Technology Gods would provide a more efficient (and less politically inhospitable) means of addressing the high-minded ideals of civil rights leaders in the 1960s and early 1970s. This effort to address “the soft bigotry of low expectations” was called “reform” by the technocrats who saw an opportunity to profit from the data-driven direction schools would head and especially by those who saw the potential for an emerging market when public schools inevitably failed to meet the standards.

In order to gain the attention of the disinterested and ultimately win their hearts and minds, I fear it will be important to accept the argument that schools are failing but offer a different and persuasive rationale WHY this is the case. As a progressive, I find it very easy to make the case that our current paradigm IS failing children. By emphasizing sorting and selecting over unifying and edifying we are creating the alienated children who see their only means of achieving power is to isolate themselves and engage in video games where they can control imaginary worlds. We are creating a world where “success” is determined by seemingly precise mathematical algorithms and not by “sloppy” metrics like the ability to get along with others or empathizing with others. And here’s the worst part: we are telling children who attend underfunded public schools that do poorly on standardized tests that they must accept a world where they live in austerity while those who attend well funded public schools that are successful on tests live in a world where “frills” like recess, drama, music, and the arts are a given. In short, getting the disinterested off the sidelines may require progressives to show the general public that the world we have now is the result of the world we created when we decided to determine “success” by test scores.

Jay Matthews Analysis of Special Education Based on Flawed Premise: NAEP Tests Are “Gold Standard”

May 31, 2018 Comments off

Yesterday’s local newspaper, the Valley News, reprinted a Washington Post article by Jay Matthews that asserted that special education as it is currently conceived in failing. In the article Mr. Matthews draws on the opinions and findings of Kalman R. “Buzzy” Hettleman who has served two terms on the Baltimore City school board and been deputy mayor of Baltimore and Maryland state secretary of human resources. Assuming that Mr. Hettleman’s political experience gives his perspective credence, Mr. Matthews writes:

Hettleman does not believe that most students in special education are truly disabled. Fewer than 20 percent, he says, have clearly defined conditions, such as Down syndrome, severe autism, or visual and hearing impairments. The rest, he says, are struggling learners, especially in reading. Their difficulties were sadly not identified and addressed in the crucial early grades.

So as a last resort, he says, they “are dumped into special education. Reading experts estimate that, in the absence of timely interventions, between 50 and 75 percent of struggling readers wind up unnecessarily in special education.”

The notion that schools over-identify Learning Disabled students is not completely off base, nor is the idea that their difficulties were not identified and addressed in early grades. But the underlying cause of all of this pressure to “dump students” into special education is our continued belief that the rate of learning is constant and linked to age… a belief that is reinforced by our reliance on standardized tests that “measure” performance based on that premise.

(Mr. Hettleman) says school officials are reluctant to be candid about how far behind special-education students are. The gold standard of education statistics, the National Assessment of Educational Progress, reported in 2017 that only 11 percent of fourth-graders and 7 percent of eighth-graders in special education were proficient in reading. The parents he has worked with and the ones I know often aren’t told those scores.

“Rather, school systems conceal actual performance through grade inflation; social promotion from grade to grade, though the student is not close to meeting grade-level standards; bogus graduation diplomas; and other means,” he says.

The so called “gold standard” Mr. Matthews and most education writers and policy makers revere is based on the premise that age-based grade cohorts are the only way to group children and their “progress” is best measured by standardized multiple choice tests.

The best way to address the “struggling learners” who are allegedly not identified in the early grade levels it to recognize that not all children who are five years old have the same skill sets and give children the time they need to develop those skill sets. Let time be the variable and learning be constant….

Categories: Uncategorized Tags: ,

My Annual Rant Against US News and World Report’s Ratings

May 13, 2018 Comments off

It’s the time of year when newspapers across America trumpet the schools in their states who achieve the highest ratings in their State and, in some cases, in the entire nation based on the US News and World Report’s metrics… and it’s the time of year when bloggers like me remind readers that these ratings are completely bogus because they are primarily based on standardized tests which, in turn, are inextricably linked to family affluence and education.

To find out how the US News and World Report calculates their rankings, one has to scroll all the way to the bottom of the page touting the importance of the ratings past the click-bait headlines listing the top high schools overall, the top charter schools, the top STEM schools, to a hot-link in the lower right hand corner. Once the link is clicked, the reader is led to another series of links where eventually the reader learns that in calculating the rankings:

…U.S. News & World Report teamed with North Carolina-based RTI International, a global nonprofit social science research firm.

RTI implemented the U.S. News comprehensive rankings methodology, which is based on these key principles: that a great high school must serve all of its students well, not just those who are college bound, and that it must be able to produce measurable academic outcomes to show it is successfully educating its student body across a range of performance indicators.

This sounds very high-minded, but the four step process is ultimately based on standardized tests and/or family income. .

Step One, for example, purports to determine “...whether each school’s students were performing better than statistically expected for students in that state (in standardized tests).” How is this done? “...(B)y looking at reading and math results for all students on each state’s high school (standardized) proficiency tests”Schools scoring in the top 10% were automatically carried forward, those scoring in the lowest 10% were dropped, and some manipulations were applied to identify schools serving disadvantaged students that performed “…much better than statistical expectations.” 

Step 2 “...assessed whether their historically underserved students – black, Hispanic and low-income – performed at or better than the state average (on standardized tests) for historically underserved students.

Step 3 looked at graduation rates, eliminating any schools that failed to graduate 80% of the cohort that entered the school. This is indirectly linked to standardized tests since 12 states require the passage of such a test to earn a diploma. But it is inextricably linked to family income since more than one third of all drop outs were raised in poverty.

Step 4 is the clincher. For schools whose students score well on State standardized tests and whose students graduate at an 80% rate, the ultimate benchmark is “college-readiness performance” – which is determined by “...using Advanced Placement and International Baccalaureate (standardized) test data as the benchmark for success.” 

So how does a school assure itself of high ratings in the annual US News and World Report’s index? Easy: it attracts students who aspire to college, students who enroll in AP and IB courses, students who pay for the AP and IB assessments, and students who do well on those tests. This creates a barrier to entry that precludes hundreds if not thousands of schools since schools or students must pay fees for each test they take, schools must pay to have teachers trained to offer AP and IB courses, and IB certified schools must pay annual fees in excess of $11,000.

And, as noted in earlier posts decrying these rankings, the whole system is based on the assumption that schools enrolling students who score well on standardized tests are meritorious. One would hope that US News and World Report writers realize that quality should be based on something more than standardized tests scores, but the test scores are seemingly precise and objective, readily attainable from public data-bases that are relatively inexpensive to glean data from from, and provide an easy means for sorting and selecting schools. In the end, the selective public and charter schools and the public schools serving the children of affluent and well educated parent achieve medals… and the vicious cycle of poverty continues.