Archive

Posts Tagged ‘Testing’

“For a real shot at improving #ed, don’t bind accountability to standardized test scores”. 

July 9, 2017 Leave a comment

This week’s newsletter from the Christensen Institute features a provocative article by Thomas Arnett titled “School Boards: Good For Policy, Bad For Management“. Mr. Arnett’s premise is that when it comes to managing schools, Boards should cede their control to a strong administrator. He writes:

Management through democratic consensus only works when board members largely agree on how the school system should be run. Unfortunately, this circumstance rarely exists outside of small and demographically homogeneous communities. Instead, the teachers, administrators, unions, taxpayers, parents, and politicians who back and elect board members often have divergent priorities and disagree strongly about how to improve education. Furthermore, all these constituent groups have different ideas of what will cause improvement—from more money to more computers, from better teachers to smaller class sizes, and many more.

In low-consensus circumstances, the only way for an organization to make progress beyond the status quo is to give strong leaders the authority to define a vision for change and then drive the organization in a new direction. But superintendents in districts with democratically-elected boards rarely get the power they need to make change happen. Inevitably, some constituent groups elect a few activist board members with a mandate to drive their own agendas. Once this happens, the divergent priorities of board members pull the district in multiple directions at once, making it nearly impossible for superintendents to prioritize their efforts or purposefully pursue any particular goal.

In these paragraphs, Mr. Arnett provides a good overview of the tensions that exist between Superintendents, Boards, and the various constituent groups each serves. But there is one factor that he overlooks in his analysis: the impact of standardized testing. Before the advent of NCLB, the various constituent groups not only had debates over how the district should be RUN, they also had debates over how to MEASURE how well schools are being run. When NCLB came to town, the debate about measurement disappeared: test scores were all that mattered…. and that determination had a HUGE impact on decision making up-and-down the chain of command and within districts. And the reliance on tests as the primary metric for “quality” is now having a subtle but powerful impact on the way schools are managed…. and the impact is not beneficial.

With summative standardized test scores used as the ultimate measure of “quality”, individual and collective student progress is measured against other students in an age cohort. Because TIME is the only variable available to administrators and teachers when PERFORMANCE on tests is viewed as a constant benchmark, schools serving students who fall on the low end of a statistical artifact— namely the bell curve created by standardized test writers— devote more time to teaching those items. Another consequence of test-centered metrics is that teachers in those schools spend much of their energy working with students who are on the cusp of achieving another statistical artifact, namely the “cut score”. This leads to the ultimate adverse effect of test-centered metrics: an ever widening divergence between affluent schools and schools serving children in poverty. Because test scores align with wealth, “high achieving” schools tend to serve children raised in affluence and “low achieving” schools tend to serve children raised in poverty. As noted above, teachers in “low achieving” schools need to devote more and more of their time teaching-to-the-test and those districts, consequently, devote more and more of their resources to that narrow goal. When faced with the budget difficulties that emerged after 2008, districts serving children raised in poverty shed “frills” like Art, Music, PE, and social services. Affluent districts, on the other hand, were not only more insulated from the budget hits of 2008, their teachers generally ignored the STATE standardized tests since their students scored in the higher ranges without requiring any focused instruction. Finally, and most importantly from my perspective, the use of summative standardized tests as the primary metric precludes any move toward mastery learning ,where student progress is measured based on formative tests.

As the paragraph above illustrates, with standardized tests as the ultimate metric, neither administrators nor school boards have any choice about which master to serve: they MUST ensure that test scores are high or they will lose their jobs (if they are administrators) or lose local control of their schools (if they are a Board).

Mr. Arnett’s tweet on his article read “For a real shot at improving #ed, don’t bind admins to divergent demands of multiple masters”. My tweet on the question of governance and change, noted above, reads: “For a real shot at improving #ed, don’t bind accountability to standardized test scores”.

ESSA’s Flaws Exposed as Betsy DeVos Assesses State Plans

July 8, 2017 Leave a comment

Anyone who reads this blog knows that I have long held misgivings about ESSA… and Erica Green’s article in yesterday’s NY Times flags some deficiencies that surprised advocates, deficiencies that I did not foresee in my earlier critiques.

Many conservatives believed that ESSA was going to provide more flexibility to States in terms of oversight by the federal government. But the early analyses by the USDOE under Betsy DeVos’ leadership indicates that will NOT be the case. Their response to Delaware’s state plan is exhibit one:

In one case, the acting assistant secretary for elementary and secondary education, Jason Botel, wrote to the State of Delaware that its long-term goals for student achievement were not “ambitious.”

It is mind-boggling that the department could decide that it’s going to challenge them on what’s ambitious,” said Michael J. Petrilli, the president of the conservative-leaning Thomas B. Fordham Institute, who worked in the Education Department under President George W. Bush. He called the letter “directly in opposition to the rhetoric and the promises of DeVos.”

But USDOE’s assessment of Connecticut’s plan flags a concern of progressive educators, who hoped that State’s might be able to break away from the strait-jacket of standardized testing.

The state was also criticized for its use of an alternative system for measuring academic performance instead of more standard “proficiency” measurements on state tests, as the law requires.

Such feedback signaled that the department “appears to be resorting to very traditional and narrow ways of interpreting student and school performance,” said Laura Stefon, chief of staff for the Connecticut State Department of Education.

So after being hailed as a bi-partisan bill that satisfied both sides of the aisle, why has ESSA riled up those on both sides of the education debate? One chief state officer offers a cynical explanation:

Christopher Ruszkowski, the acting secretary for the New Mexico Public Education Department, said the idea that the new law would yield total state control was merely “rhetoric from the Beltway.”

I think a lot of the euphoria over return to local control was an overpromise,” he said. “What this signals is that U.S.D.E. will continue to play the role they’ve always played in the years ahead.”

In addition to alienating policy-makers in both the “reform” and progressive camps when it comes to testing, ESSA has riled us another group: the National Science Teachers Association. Why?

Connecticut was also among a handful of states faulted for including science as a subject for measuring achievement, even though the law allows the use only of reading and math. This feedback was widely criticized by academic groups, including the National Science Teachers Association, who said the department was interpreting the law too literally.

The science teachers, like the teachers of any content outside of reading and math, experienced staffing challenges as school districts were forced to teach-to-the-test in order to meet “ambitious” goals required under NCLB and RTTT. They, like the state and local leaders of schools and the state legislators, were led to believe that ESSA would enable their states to develop accountability systems that would incorporate their topics. In the end, as Mr. Green notes in her closing paragraphs, the consensus seems to be that ESSA DID over-promise and under-deliver:

State leaders said they believed they were all but promised their plans would be approved. Instead, Chris Minnich, the executive director of the Council of Chief State School Officers, said some aspects of the Education Department’s feedback were “overzealous” and could undermine community involvement.

“It’s going to be really hard for a state to go back and say, ‘I know I told you we were doing all of this, but we’re going to change it because the federal government told us not to,’” Mr. Minnich said.

There was bi-partisan support for this bill when it passed. I fear that there will NOT be a bi-partisan acceptance for the responsibility of the bill’s deficiencies… and know that in the end the children who will suffer the most are those being raised in poverty.

AP Tests and the Ratings Game

July 4, 2017 Leave a comment

Diane Ravitch had two posts yesterday on AP tests: one questioning their validity and another questioning them in the context of Washington Post columnist Jay Matthews’ misbegotten rating system.

 

In response to one of them, I offered my thoughts on AP, from a retired Superintendent’s perspective and a parent’s perspective:

First, and most importantly, Jay Matthews’ attempts to rate schools— like ANY attempt to rate schools— reinforces the notion that parents are “consumers” who can select public schools. This might be true for affluent parents who are choosing which suburban enclave they want to live in but it clearly isn’t and never will be true for poor minority parents.

Second, the community where I last led a public school system– Hanover NH, home to Dartmouth College— did not offer AP courses when I retired in 2011. As a result, it was not as highly rated by Jay Matthews’ metric as other NH school districts. While some parents wished we offered AP courses, the school boards I worked with supported the faculty at the HS who did not see the need for such courses. (see the next paragraph)

Third, even though Hanover HS did not offer AP courses, many Hanover HS students took the AP tests anyway… and typically those who did take the tests passed. This is a “secret” about AP that is not widely publicized: any student who pays the fee can take an AP test and have their scores reported to a college that cares about them… and any student taking a well taught calculus course, English course, foreign language course, or science course will likely do as well as a student enrolled in an AP course.

Finally, I do believe that SOME public schools benefit from offering AP courses. The rural MD HS my daughter attended (I’ll call it East Phippstitch HS) offered AP courses. I believe her scores on those tests effectively validated the quality of that HS to the competitive colleges she applied to in New England. Hanover HS has a reputation among many colleges. East Phippstitch HS did not. The fact that East Phippstitch offered AP courses and the fact my daughter scored well on those tests she took in her junior year MAY have been a factor in her gaining acceptance to schools that otherwise would have placed her application in another file.

The bottom line: whether to offer AP courses should be a local determination that is made independent of any artificial rating system like Jay Matthews’ index and made in concert with the administrators and teachers in the district.

Standardized Tests and Smart Fools Redux

July 2, 2017 Leave a comment

As noted in a post earlier today, I was captivated by a Scientific American interview with eminent psychologist Robert Sternberg. After reading the article and writing my blog post, I left the following comment on her blog, a comment that was largely derived from my earlier blog post:

Great article! Thanks for sharing. My take:

Dr. Sternberg didn’t say so explicitly, but the kind of tribalism that sets todays ethical standards comes from “cultures” that celebrate “outlaws” and anti-establishment behavior. Voters knew that Donald Trump was a misogynist who cheated on his wife, a ruthless businessman who viewed cheating on his taxes as a shrewd business move, and an anti-intellectual who loved “the uneducated” and despised the “intellectual elites”. The tribal cultures that hold Mr. Trump in high esteem, the tribal evangelical culture, and the tribal gun culture ultimately elected a man who opposed the rule of law and the establishment. And Dr. Sternberg sees this tribalism as a by-product of our test culture that places a premium on teaching individual test-taking skills at the expense of “teaching good values and good ethics and good citizenship”.

I concur with Dr. Sternberg. What gets tested gets taught, and we have ignored testing for the complicated and relatively difficult to measure inter-personal and intra-personal skills that lead to “good values and good ethics and good citizenship”. Instead our schools have placed a premium on the relatively inexpensive and easy to measure analytic skills associated with reading and arithmetic. We haven’t taught the important skills and we are witnessing the by-product when those who do not possess the skills needed to thrive in our new economy band together in tribes with like-minded world views.

In a world that increasingly operates in echo chambers and a world where “choice” may result in children focusing even more on test-taking skills and attending schools with fellow tribal members, it may be difficult to encourage the kind of curriculum Dr. Sternberg espouses… a curriculum that values creative, independent thinking that “defies the crowd and defies the Zeitgeist”. We face an uphill battle in getting back to common ground where all of our citizens agree on what constitutes “good values and good ethics and good citizenship”… and “choice” will ultimately make that uphill battle even steeper!

That comment elicited some additional comments from Diane Ravitch’s readers, including this one from “NYC Public School Parent” who wrote:

This sounds good except….

This mis-use of standardized testing and teaching to the test is a more recent phenomenon. It wasn’t the youngest voters who elected Trump.

Trump’s core support was among voters who are older and whose education was before standardized testing became the measure of all.

I agree with critics of the test-taking culture but is Sternberg talking about that culture from 40 years ago or today?

After reading this and reflecting on it, I wrote the following response:

Thank you for providing me some food for thought… and while I can’t speak for Dr. Sternberg, I can share my take on the roots of the “test-taking culture”… I’m 70 years old… and in the OK and PA public schools I attended in the 50s and 60s standardized tests were one of the bases (if not the PRIMARY basis) for assigning students to homogeneously groupings… “Back in the day” they were mostly used for sorting STUDENTS (as opposed to SCHOOLS)… and many of my classmates in JHS who were relegated to the “low sections” weren’t around when I graduated… Moreover, PSATs and SATs were routinely used to determine which non-legacy students got to attend “elite” colleges… I also worked as an administrator in public education from 1975 through 2011 where I witnessed a movement to use tests to identify the “gifted and talented” students, a notion I could not support because it ultimately identified a large segment as “UN-gifted and UN-talented”

My thought: those kids shunted aside in the 1960s and 70s and the so-called “UN-gifted and UN-talented” are the “core support” for Mr. Trump… Mr. Trump’s opposition to “liberal elites” resonates with them because it was the “elites” who designed and administered the tests that made them feel like second class citizens throughout their education.

While I sincerely wish the mis-use of standardized tests was a recent phenomenon, my experience as a student and a public school administrator tell me otherwise… Here’s hoping we can free ourselves from this paradigm in the near future.

Standardized Tests Giving Us Smart Fools… But Our World Seeks Wisdom, Not Knowledge

July 2, 2017 1 comment

Diane Ravitch’s blog provided a link to a Scientific American interview with eminent psychologist and testing expert Robert Sternberg who recently received recognition from the Association for Psychological Science Association for his lifetime contributions to psychology. In the introductory paragraph to the interview, edited by Claudia Wallis, she describes Dr. Sternberg’s thinking on intelligence as follows:

Sternberg, who has studied intelligence and intelligence testing for decades, is well known for his “triarchic theory of intelligence,” which identifies three kinds of smarts: the analytic type reflected in IQ scores; practical intelligence, which is more relevant for real-life problem solving; and creativity.

In the interview, Dr. Sternberg decries public education’s emphasis on analytic knowledge, which has resulted in an increase in IQ scores of 30+ points over several decades but has not yielded the kind of knowledge required to sustain a highly functioning society. Here’s Dr. Sternberg’s synopsis:

Tests like the SATACT, the GRE—what I call the alphabet tests—are reasonably good measures of academic kinds of knowledge, plus general intelligence and related skills. They are highly correlated with IQ tests and they predict a lot of things in life: academic performance to some extent, salary, level of job you will reach to a minor extent—but they are very limited. What I suggested in my talk today is that they may actually be hurting us. Our overemphasis on narrow academic skills—the kinds that get you high grades in school—can be a bad thing for several reasons. You end up with people who are good at taking tests and fiddling with phones and computers, and those are good skills but they are not tantamount to the skills we need to make the world a better place…

What I argue is that intelligence that’s not modulated and moderated by creativity, common sense and wisdom is not such a positive thing to have. What it leads to is people who are very good at advancing themselves, often at other people’s expense. We may not just be selecting the wrong people, we may be developing an incomplete set of skills—and we need to look at things that will make the world a better place.

Of course looking at “…things that will make make the world a better place” is the easy part. Agreeing on those things, teaching those things, and measuring those things will be the challenge. Ms. Wallis probes Dr. Sternberg on “Wisdom” and got the following responses:

Wisdom is about using your abilities and knowledge not just for your own selfish ends and for people like you. It’s about using them to help achieve a common good by balancing your own interests with other people’s and with high-order interests through the infusion of positive ethical values…

You learn wisdom through role-modeling. You can start learning that when you are six or seven. But if you start learning what our schools are teaching, which is how to prepare for the next statewide mastery tests, it crowds out of the curriculum the things that used to be essential. If you look at the old McGuffey Readers, they were as much about teaching good values and good ethics and good citizenship as about teaching reading. It’s not so much about teaching what to do but how to reason ethically; to go through an ethical problem and ask: How do I arrive at the right solution?

When Ms. Wallis asked if we have less wisdom now than in the past, Dr. Sternberg had a sharp response:

Not only do we not encourage creativity, common sense and wisdom, I think a lot of us don’t even value them anymore. They’re so distant from what’s being taught in schools. Even in a lot of religious institutions we’ve seen a lot of ethical and legal problems arise. So if you’re not learning these skills in school or through religion or your parents, where are you going to learn them? We get people who view the world as being about people like themselves. We get this kind of tribalism.

Dr. Sternberg didn’t say so explicitly, but the kind of tribalism that sets todays ethical standards comes from “cultures” that celebrates “outlaws” and anti-establishment behavior. Voters knew that Donald Trump was a misogynist who cheated on his wife, a ruthless businessman who viewed cheating on his taxes as a shrewd business move, and an anti-intellectual who loved “the uneducated” and despised the “intellectual elites”. The tribal cultures that hold Mr. Trump in high esteem, the tribal evangelical culture, and the tribal gun culture ultimately elected a man who opposed the rule of law and the establishment. And Dr. Sternberg seems this tribalism as a by-product of our test culture that places a premium on teaching individual test-taking skills at the expense of “teaching good values and good ethics and good citizenship“.

I concur with Dr. Sternberg. What gets tested gets taught, and we have ignored testing for the complicated and relatively difficult to measure inter-personal and intra-personal skills that lead to “good values and good ethics and good citizenship” favoring instead the relatively inexpensive and easy to measure analytic skills associated with reading and arithmetic. We haven’t taught the important skills and we are witnessing the by-product when those who do not possess the skills needed to thrive in our new economy band together  in tribes of like-minded world views. Is there a way out of the woods? Dr. Sternberg remains optimistic:

If one could convince even a few universities and schools to try to follow a different direction, others might follow. If you start encouraging a creative attitude, to defy the crowd and to defy the zeitgeist, and if you teach people to think for themselves and how what they do affects others, I think it’s a no-lose proposition. And these things can be taught and they can be tested.

In a world that increasingly operates in echo chambers and a world where “choice” may result in children focusing even more on test-taking skills and attending schools with fellow tribal members, it may be difficult to encourage creative, independent thinking that defies the crowd and defies the Zeitgeist. We faee an uphill battle in getting back to common ground where all of our citizens agree on what constitutes “good values and good ethics and good citizenship“.

 

Article on MD’s New Rating System A Reinforces Notion that Parents are Consumers who PRESUMABLY have a Choice

July 1, 2017 Leave a comment

Liz Bowie’s Washington Post article describing the state of Maryland’s new rating system elicited a response that is a combination of resignation, dismay, and frustration.

Resignation because it is reinforces the notion that parents are “consumers” who presumably have a choice when it comes to school; dismay because that clearly is NOT the case for those parents who are living in poverty; and frustration because I don’t see either circumstance changing any time soon.

Ms. Bowie’s article describes the tension between the State Board, appointed by Republican Larry Hogan, who seek a greater emphasis on so-called “academic” measures and the largely Democratic legislature that passed a law limiting the percentage of a rating that can be based on “academic” measures (i.e. standardized test results) to 65%. But given the GOP leadership in the USDOE, the State Board’s desire to emphasize academics is likely to prevail. Why?

Basing just 65 percent of the weight on academics may be deemed too low by the U.S. Department of Education. The department has already told Delaware that its 80 percent weight for academics is too low.

And what, exactly, constitute “academic measures”? Here’s Maryland’s idea of “academics”, which currently constitute 65% of a school’s rating but, based on the USDOE’s reaction to Delaware’s submission, will need to constitute an even higher percentage:

… students’ annual academic growth and the percentage of students who pass the state English and math exams.

Meanwhile, those who celebrated the passage of ESSA might think again. Here’s how Ms. Bowie described ESSA as compared to NCLB, the law it replaced.

ESSA replaces the No Child Left Behind Act, which was in effect for more than a decade and was widely criticized for being too punitive because schools were judged almost solely on test scores. ESSA was designed to give more authority back to the states for how schools are held accountable.

The legislators who championed ESSA because it stopped the centralization of decision-making in Washington—especially the conservative GOP members who harp on this issue repeatedly— should ask themselves if that has really happened. It seems that the spirit of ESSA should allow Maryland to base 65% of it’s ratings on test scores. But the billionaires who love to demean public education and who help underwrite the campaigns of the “states-rights” Tea Party will likely look the other way on this issue.

Categories: Uncategorized Tags: , , ,

The Content of One’s Character Cannot be Measured by a Standardized Test or a Mathematical Algorithm

June 28, 2017 1 comment

NPR re-posted an article on the need for reform in admissions to “elite public schools” written for The Conversation by Faculty Director for Professional Education, BU School of Education, Boston University. The article defines “elite public schools” as those that use test scores as the predominant metric for admission, referencing schools like “New York’s StuyvesantBoston Latin or Walter Payton(in Chicago)”. Mr. Murray decries these admissions standards because they inevitably result in segregation by race due to the high correlation between race and test results. Instead of using test scores and other easily quantifiable data as the primary basis for entrance into these competitive schools, Mr. Murray suggests that “elite” public schools follow the example set by several elite colleges who are part of a Making Caring Common, described as “a project of the Harvard Graduate School of Education, these institutions are piloting new admissions policies that focus less on numbers and more on “ethical engagement.”” Mr. Murray elaborates:

In a report released in January 2016, Making Caring Common argued for elite colleges and universities to include opportunities for candidates to submit authentic demonstrations of empathy, service to others and commitment to the common good as part of their application. They contend that these important values are worth promoting to students and families. In fact, research suggests that strength of character and “grit” are key determinants of future academic and career success.

Importantly, these new metrics could weigh social and emotional attributes that students across all backgrounds could exemplify in some way.

To date, over 175 colleges and universities have signed on to this concept, seeking to diversify their classes and to offer an opportunity to attend college to a wider pool of students. Mr. Murray suggests that elite exam schools could adopt a similar method for admissions:

A school might give special consideration, for example, to candidates who worked to support their families at an early age, served as caregivers to younger siblings, organized efforts to support a needy classmate or led a food drive to help a local shelter.

Exam schools across the country could team with Making Caring Common and its growing list of higher education partners to determine how best to validly and reliably collect, evaluate and weight these types of student experiences.

Unsurprisingly, one commenter to the NPR article found this whole idea distasteful. Effectively speaking on behalf of many who value “merit-based” admissions, commenter “brian m” wrote:

Why would you want to place kids in the Advanced Placement courses that they do not test ready for and will probably fail to pass without a hand me grade. UGH. Why are we rerunning and rerunning race based numbers and using race as the determining factor! Martin Luther king said it wonderfully . Martin Luther King, Jr. I have a dream that my four little children will one day live in a nation where they will not be judged by the color of their skin, but by the content of their character. Why do we digress to 1963. The liberal mindset is to make the races equal by giving certain races things they dont earn. Take race out of theses decisions! What do you tell the kids that get passed over for placement into these schools and kept high grades that are the wrong race?

My rebuttal to this comment was:

“What do you tell the kids that get passed over for placement into these schools and kept high grades that are the wrong race?”
You tell them that the qualities needed to succeed in school and life depend on more than getting high test scores or good grades… You tell them that the content of one’s character cannot be measured by a neat and clean mathematical algorithm…

And finally, you tell them that they should be grateful they will not be profiled by police whenever they drive a nice car or shop in a good department store…

Categories: Uncategorized Tags: ,