Posts Tagged ‘Measurement’

Economists Weigh in on K-12 and Higher Education… and Their Conclusions Are a Mixed Bag

October 21, 2017 Leave a comment

Lacrosse (WI) Tribune writer Nathan Hansen reported on a two day gathering of four economists at University of Wisconsin-LaCrosse. He detailed their reports on a host of education issues and, after reading them, I concluded that their findings can best be summarized in this sentence from Christopher Walters from the University of California-Berkely, who, when asked about the value of standardized tests said:

“Having test scores is better than nothing, A researcher would like more measures and different kinds of tests.”

All of the economists in one former another echoed the sentiment that they wished for more data, but one conclusion that none of the four economists challenged was the impact of poverty, with Matthew Wiswall from UW-Madison and Susan Dynarski from the University of Michigan, being especially outspoken on the issue. In examining the impact of pre-school education Mr. Wiswall was particularly forthright. He noted that family background has a disproportionate effect on childhood development, likely due to those families having access to more resources to provide better nutrition, schooling, early education resources or even ability to spend more time with their children. He advocated for government intervention in the form of income redistribution such as the Earned Income Tax Credit and advocated for funding of early childhood programs such as preschool or Head Start, which targets low-income families with children between the ages of 3 and 5. He was particularly disparaging of “choice”:

“One of the questions people ask is why the government should do something. In the education policy sphere, one major motivation is you don’t get to pick your parents or sign contracts before you are born.”

Ms. Dynarski noted that additional education funding can only be the answer if that funding is targeted to those children raised in poverty and targeted to their essential needs. She was critical of Wisconsin’s recent legislation that provided vouchers to students who were already enrolled in private schools at their parents expense and technology initiatives might be an unwise use of scarce funding if the goal is to provide an equal opportunity for all. Mr. Hansen offers this quote from Ms. Dynarski to summarize her thinking:

“For a kid in good shape, adding another dollar is probably not a good investment. If your family is having trouble putting food on the table, adding a nifty laptop isn’t going to make a big difference.”

The economists all lamented the use of test scores as the primary metric for “school quality”, but being driven by data felt that the scores at least provided a means of capturing the inequities in schools.

In the second day the four economists tackled higher education… and their analysis there was flawed by the limited data as well and particularly muddied by the fact that they effectively bought into the notion that post-secondary education is all about earning more money. That may be because earnings is the only available hard metric for post-secondary education in the same fashion that standardized tests are the only hard metric for K-12. But in both cases, using the hard data as the sole rationale for schooling is wrongheaded: it assumes that anything that can’t be measured is unimportant, which is clearly not the case in a democratic nation.

There is one area where hard data is can inform education policy, and that is in the area of student loans. In examining the student loan crisis, Ms. Dynarki noted that data she’s gathered indicates that “...interest rates don’t have as much impact on monthly payments as they do on longer loans, such as a mortgage.” Instead, she suggests that policymakers focus on the repayment process or reducing student borrowing.

After reading Mr. Hansen’s article I conclude that the economists’ ability to inform policy making is limited by the hard data available to us… and because of that economists have thus far provided more mischief than assistance. Enamored of the power of mathematical models, it is economists who helped develop VAM and who use complex algorithms rooted in standardized test scores and demographics to assess the effectiveness of charter schools…. and it is the economists and statisticians who are promoting the gathering of hard data on soft skills, thus leading to a time where educators might be held accountable for flawed test results in those areas that same way they have been held accountable for test results for two decades in public schools. My thought: anthropologists would be more helpful than economists.


Kindergarten Assessments: What Gets Measured Gets Done… and in SOME Cases That ISN’T a Bad Thing

October 14, 2017 Leave a comment

When I read the title of Christina Samuels Education Week article, “Kindergarten Assessments Begin to Shape Instruction”, I immediately assumed I would deplore the content. After all, when assessments drive instruction teachers invariably teach to the test and if that test is a traditional pencil-and-paper assessment one can expect a narrowing of instructional techniques. But the first three paragraphs quickly disabused me of my misgivings and recalled my experience at changing the mindset of elementary teachers in a New Hampshire district in the mid-1980s:

In the not-too-distant past, the kindergarten classrooms at Pleasant Grove Elementary in Heflin, Ala., looked much the same as classrooms for older children.

Desks were arranged in rows. Children worked on worksheets. “There wasn’t a lot of differentiation in your instruction,” said Kristi Moore, a kindergarten teacher at the school, located halfway between Birmingham, Ala., and Atlanta. “Most of all your children were taught the same way.”

But in recent years, the school has tried to shift instruction in a way that they say works better for young children. And they credit the use of a comprehensive method of evaluating kindergarten students, called kindergarten entry assessment, as one of the tools that allowed them to do that.

When a colleague and I were appointed to lead a New Hampshire district in 1983, we quickly determined that the early elementary instruction in one of the communities was far too rigid and traditional. Like the Alabama district described in the first paragraph, the veteran first grade teachers and their principal who valued traditional lockstep rote-learning methods dictated the format of instruction in the early grades. We quickly learned that the underlying tension in that school was the result of a tension between teachers on the staff who valued more open and innovative methods and the “traditionalists” and a tension between parents in the community who wanted to see more differentiation and the “traditionalists”. Our workaround was to introduce a pre-first grade “Readiness” level for children who were “not ready” to thrive in the traditional classrooms that required them to sit still for direct instruction and work tirelessly on drills provided on mimeo worksheets. The teachers who wanted to use “non-traditional” approaches supported this effort because they would be able to demonstrate how differentiated instruction could reach children of all temperaments and abilities while the “traditionalists” supported it because their classrooms would be rid of the “troublemakers” who could not adhere to their approach. After the “traditionalist” Principal of the school was replaced by one who championed a developmental approach, the school began shifting more and more to a developmental approach.

The introduction of entry assessments mandated by Race To The Top seems to have had the same influence in school districts where the lockstep factory model was in place— the only element of that misbegotten program that did anything to change that paradigm! The assessment driven approach has some drawbacks, though, as Ms. Samuels notes:

Others have criticized the assessments as an additional burden that doesn’t let teachers know what they should do with all the data they’re expected to collect. And the assessments also raise concerns for some that they’ll be used for high-stakes purposes, like evaluating teachers or sorting children into educational tracks.

It is the use of assessments for sorting and electing that was problematic for us in New Hampshire as well. We realized that the assessment we used to assess “readiness” was imperfect and we therefore used it to guide parents’ decisions to defer entry into first grade or to proceed with entry based on their child’s chronological age. The deciding factor was often an invitation for the parent to look at the configuration of the “traditional” classroom as compared to the configuration of the “Readiness” classroom at which point the parents could determine which approach would work best for their child. Because the assessments were imperfect, we deferred to the parent’s choice… but more often than not the parents saw that the developmentally driven “Readiness” classroom was a better match.

If assessments can be used to reinforce a developmental approach to instruction they can be a force for true reform… but if they are used to reinforce the age-based cohorts that constitute the factory school model, they will be a force for the status quo.

Categories: Uncategorized Tags: , ,

This Just In: Homelessness Contributes to Academic Challenges… and in NYC 10% of Students were Homeless at some point in 2016-17

October 11, 2017 Leave a comment

Elizabeth Harris’ article in today’s NYTimes matter-of-factly reports that in the NYC School District one in every 10 public school students was homeless at some point during the 2016-17 school year and matter-of-factly describes the impact of homelessness on the students:

The upheaval in the home lives of students in temporary housing often follows them into school. Many of them frequently change schools as they bounce from one temporary living situation to another. Many are placed in shelters far from their original school, which means they must either transfer midyear or commute long distances each day. Many students regularly arrive late or miss days of school altogether.

Those stresses harm their academic performance. A report released this summer by the Institute for Children, Poverty, and Homelessness found that homeless students passed the state English tests at about half the rate as their peers who had permanent homes. Homeless students who were designated as English Language Learners generally took longer to become proficient in the language. On average, the report found that one-third of homeless students miss the equivalent of a month of school. Students living in homeless shelters had the highest rates of chronic absenteeism, meaning they missed more than 10 percent of school days.

Liz Cohen, chief of staff at the Institute for Children, Poverty, and Homelessness, said that while most of the city’s homeless policies are aimed at getting people housed, the academic damage can linger long after students find a place to live.

“The data shows that for multiple years after a student becomes housed, they have increased rates of chronic absenteeism and decreased academic performance,” Ms. Cohen said. “That experience stays with them.”

The data on the absenteeism and the transience of homeless children is only part of the story. The schools attended by these children are “graded” based on how well all of their students perform on standardized tests. Unsurprisingly, if a school is located in a neighborhood where there is a lot of churn in the housing market due to parents failing to pay rents, or a neighborhood where there is a homeless shelter that provides temporary housing, or a neighborhood where previously homeless children reside, their test scores will be lower because of the impact of homelessness on academic performance. Moreover, the parents of homeless children, who are concerned with keeping a roof over their heads, food on the dinner plate, and clothing on the backs of their children are unlikely to have the wherewithal to apply for the charter schools that “reformers” see as the solution to those stuck in a vicious cycle of poverty. Those who advocate “choice” as the solution to poverty often have a choice as to where they live, what kinds of clothes their children will wear to school each day, and what kind of food they will serve. Children and adults need food, clothing, and shelter first… “choices” can come later.


ESSA Was Supposed to “Unleash a Wave of Innovation” in Metrics… It’s Fallen Short of the Mark

October 2, 2017 Leave a comment

Education Week writer Darrel Burnette II posed this question as the title of a blog post over the weekend: “”  Why Did So Many States Choose to Use These Two Indicators? The two indicators: chronic absenteeism and “college and career readiness”. The answer to the question Mr. Burnette poses should be obvious to anyone who’s read this blog: they are cheap and easy to measure. Data on absenteeism has been collected for eons and determining what constitutes “chronic absenteeism” requires a minimal debate. Data on “college and career readiness” is also easy to collect. Virtually all high schools collect data on how many students applied for college and the percentage of students who graduated. Moreover, State Departments could make a straight faced argument that their graduation standards define “college and career readiness” and thus graduation rates, which are also collected, could be used to determine that metric. Mr. Burdette, like me, was skeptical that ESSA would yield any imaginative metrics, as he noted in his post:

It’s an issue I wrote about a year ago as state departments started rejecting outright some pretty unusual and innovative ideas from parents and teachers about how best to measure their schools. This caused consternation and confusion amongst advocates who wanted to break away from heavy reliance on testing.

From my story:

One big issue: whether states and districts are able to retrofit their data-collection systems to answer new and increasingly difficult questions, a potentially arduous and expensive task.

For many measures, state officials say they lack the infrastructure to collect enough reliable information to attach high stakes. Many districts’ data-collection sytems are scattershot and outdated. Scores of technicians responsible for processing data have been laid off in recent years amid budget cuts. And local superintendents have complained that they’re already required by states to collect an inordinate amount of data.

In addition, states must navigate a myriad of data privacy laws passed in recent years.

So what will become of ESSA’s promise to provide new and creative metrics? Mr. Burdette does not forecast any substantive changes, but he does note that ESSA does require the collection of new data points:

In the meantime, ESSA requires the collection and public reporting of several new data points, including student arrest rates, teacher experience and average pay, and school-by-school spending. While these data points will be collected and reported, schools will not be held accountable for disparities.

What do these data points have in common? They are easy to measure and cheap to collect… And what is the biggest problem with these data points? “…Schools will not be held accountable for disparities.”  And when it comes to arrest rates, teacher pay, and per pupil spending school districts shouldn’t be held accountable because they are all beyond their control. But, as I’m sure someone will find, there will be a correlation between these data and the affluence of the school districts.


This Just In: ESSA Makes No Difference in Yet Another State

September 23, 2017 Leave a comment

When ESSA was passed in Congress, it was marketed as an opportunity for states to establish their own accountability standards, thus giving them a chance to break away from the standardized test scores as the primary metric as “imposed on them” by the federal government. In state after state, though, the results have not borne out that promise. Here’s a portion of a report from Megan Raposa of the Argus Leader, a regional newspaper that’s part of the USA Today chain, that was presented in a Q and A format (my emphases added):

What’s different now? Monday’s rule change was only one piece of ESSA’s implementation. It changes the way schools are ranked by the state, and it gives schools a new way to look at how students are performing. 

What criteria will the state use to assess schools? The state will look at test scores, student attendance, graduation rates, college and career readiness, and how well English learners perform on standardized tests. 

How will this affect students? Largely, it won’t. Students will still take the same standardized tests as they did under No Child Left Behind. Also, they may see their teachers putting more emphasis on “college and career readiness,” a key theme in South Dakota’s education goals.

So while ESSA “gives schools a new way to look at how students are performing” the “new” method for measuring student performance will not make any difference to students since they will be taking the same standardized tests as they did under No Child Left Behind!

If South Dakota was an outlier, this would be no big deal… but to date I am unfamiliar with any states who are making a substantial break from their reliance on standardized tests. One the final tally is in on ESSA submissions, I’ll write a post on the “new way” states are assessing student performance… and I will be astonished if ANY of them are diminishing their reliance on tests.

Categories: Uncategorized Tags: , ,

This Just In: NJ Sees the Light! Restores Local Control in Newark. I Await Reformers’ Rebuke

September 14, 2017 Leave a comment

The headline for David Chen’s Tuesday’s NYTimes article on September 12 doesn’t acknowledge defeat for reformers, but it DOES mean a victory for democracy:

After More Than 20 Years, Newark to Regain Control of Its Schools

While the headline doesn’t acknowledge a defeat for “reformers, this part of Mr. Chen’s does:

…the decision to give authority back to the city is in many ways a recognition that state control is an idea whose time has passed. Around the country, 28 other states enacted similar policies, fueled by a desire to hold districts more accountable.

In handing the control back to the city, the State declared victory by citing the fact that the takeover ended what a judge 20 years ago identified as a situation where”... “nepotism, cronyism and the like” had precipitated “abysmal” student performances and “failure on a very large scale.”

The State ended the “…nepotism, cronyism and the like” but in doing so lined the pockets of many for profit enterprises, experienced horrific deficits, and many unsuccessful attempts to make substantial improvements to performance as measured by test scores, graduation rates, and attendance data. It wasn’t until Ras Baraka took over as mayor three years ago and forged a solid working relationship with the state appointed superintendent, Christopher Cerf, that things started to get better. Mr. Baraka used $10,000,000 of the $100,000,000 donated by Mark Zuckerberg to “finance a network of “community schools,”…to provide health care and social services beyond classroom hours”, an action that increases community engagement and ultimately made the transition away from State control possible.

As Mr. Chen noted in his article, Newark was the second city to get a release from State control. Jersey City preceded them… and their mayor had nothing good to say about the impact of the State:

But (Jersey City mayor) Mr. Fulop gives little credit to the state. “Those things converging have helped the school system gradually get better; it has nothing to do with the State of New Jersey’s policies,” Mr. Fulop said. “You wouldn’t find anybody who points to state control and says, thankfully the state was here.  

Mr. Baraka, among others, is hopeful that when the transition is complete, the city will have learned its lesson.

“Local control means that you’re in charge now — you can’t cuss out people now unless you’re cussing yourself out,” he said. “Stop thinking about us versus them, because us is the them.”

Democracy prevails over corporatism. Here’s hoping the voters in Newark make certain they elect responsive and responsible board members. If they do, they will continue to thrive.


Politico Report Finds that Metrics Matter… and US News and World Report’s Metrics for Colleges Are Increasing Inequality

September 12, 2017 Leave a comment

We are approaching the time when US News and World Report issues its annual analysis of colleges, a report that was launched in 1983 and became, in the magazine’s own words from a 2008 article , “the 800-pound gorilla” of higher education. Politico writer Benjamin Wermund concurs with that assessment, and in his story issued on Sunday asserts that the 800 pound gorilla has undercut social mobility in our country and offers evidence from a new “report card” to support his contention. He opens his essay with this:

America’s universities are getting two report cards this year. The first, from the Equality of Opportunity Project, brought the shocking revelation that many top universities, including Princeton and Yale, admit more students from the top 1 percent of earners than the bottom 60 percent combined. The second, from U.S. News and World Report, is due on Tuesday — with Princeton and Yale among the contenders for the top spot in the annual rankings.

The two are related: A POLITICO review shows that the criteria used in the U.S. News rankings — a measure so closely followed in the academic world that some colleges have built them into strategic plans — create incentives for schools to favor wealthier students over less wealthy applicants.

As I’ve written on several occasions (see here, here, and here for examples), the US News and World Report’s annual report card is too reliant on test data, has contributed to a horse race mentality among colleges, contributed to the extreme competition that infects affluent high schools, and, most insidiously, reinforces the notion that microscopic differences in algorithmic “scores” reflect qualitative differences in schools.

Mr. Wermund uses hard data from the Equality of Opportunity Project, to demonstrate that the US News and World Report does something even worse: it closes doors of opportunity to individuals in families with low incomes. Mr. Wermund notes that the US News and World Report’s metrics include several components that favor students from affluent schools:

  • student performance (i.e.evidence that their acceptance pool has “the best and brightest” as measured by standardized tests and GPAs)
  • lower acceptance rates (i.e evidence of their “competitiveness”)
  • performing well on surveys completed by guidance counselors (which favors affluent high schools with robust college counseling staff), and
  • alumni giving (which compels colleges to draw from affluent applicants)

At the same time, the US News and World Report ignores economic diversity, a measure that would encourage schools to accept more children from less affluent households. And Mr. Wermund digs deeper into the impact of the US News and World Report’s annual report card by interviewing several past and present college presidents. Their reactions were astonishing, with one, Brit Kiran of the University of Maryland, offering a particularly scathing indictment:

Kirwan cast the problem in simpler terms, saying that U.S. News creates the false impression that schools with the wealthiest students are, based on their criteria, the best.

“If some foreign power wanted to diminish higher education in America, they would have created the U.S. News and World Report rankings,” he said. “You need both more college graduates in the economy and you need many more low-income students getting the benefit of higher education — and U.S. News and World Report has metrics that work directly in opposition to accomplishing those two things that our nation so badly needs.”

Mr. Wermund suggests that this emphasis has had a major impact on politics as well:

“Elite colleges are part of the apparatus that produces Trumpism and produces working class, white resentment,” said Walter Benn Michaels, a professor at the University of Illinois at Chicago.

“It fits perfectly into Trump’s narrative … Basically, if you’re a low-income or working-class white student who works hard and you find out that what matters in admissions is who your daddy is, or what your race is, you’re completely left out,” said Richard Kahlenberg, senior fellow at the Century Foundation. “When a politician like Donald Trump comes along and says the system is rigged, you’re very likely to believe that. In this case, it is rigged — against those students.”

So… what is to be done? The disheartening news is: “Not much”. Donald Trump and the most conservative members of the Senate want to compel colleges to spend more of their endowments or possibly lose their non-profit status, which might compel them to use their endowments to underwrite scholarships for more children raised in poverty. President Obama suggested developing a Federal metric for colleges that would report on the incomes earned by graduates, an idea that would effectively reward colleges offering degrees in science and technology while penalizing colleges that offer degrees in teaching and social work. But one thing is clear, and Mr. Wermund notes at the end of his article: US News and World Report won’t be making changes any time soon:

Brian Rosenberg, president of Macalester College, said he met with U.S. News officials and raised concerns that the rankings incentivize schools to spend more money when the cost of college is already skyrocketing.

“The question I asked was, ‘Doesn’t this seem to run counter to what’s really in the public’s interest?’” he recalled.

“The answer was, ‘Yes, we know it — but we don’t care.’”

Morse, of U.S. News, denied that, saying, “We do meet regularly with college presidents and admissions deans and we’re definitely aware of what’s written about U.S. News.”

To many presidents, though, prodding U.S. News to change feels like a lost cause.

Said Rosenberg, “It feels a little bit like shaking your fist at the gods — there’s nothing I can do about it.”