Posts Tagged ‘Testing’

Diane Ravitch’s Critique of American Psychological Association to Speak Up Misses the Point

March 30, 2017 1 comment

In a blog post yesterday, Diane Ravitch quoted from a comment left by testing expert Fred Smith whose comments echoed these questions:

Why isn’t the American Psychological Association speaking out about the misuse of standardized testing? Where are the professors who teach about testing? Why are they silent when children as young as 8 are subjected to hours of testing? Why are they silent when children in middle school are compelled to sit through tests that last longer than college admission tests? Why are they not defending their own standards for the appropriate use of tests? Is their silence a sign of complicity or indifference?

My comment to this post was this:

The psychologists here are analogous to the economists in the lead up to the calamitous Wall Street crash and, as others have noted, the various researchers who give cover to Big Pharma…There are a few renegades who will speak out against the testing, but the corporate line is that testing and measurement are a good thing because it helps feed the paradigm that schools-are-a-business-whose-bottom-line-is-test-scores… And the best tests are those that can be done quickly and cheaply and yield a number that can be put onto a spread sheet and used to establish a rank order… As long as educators use tests in any way to sort and select, standardized tests will be with us.

In the end, we need to change the implicit paradigm of the factory school where students are batched by age cohorts and measured against their age peers and move to a completely individualized and personalized form of instruction where time is the variable and mastery is constant. Such a system would require no more personnel that we use today but would require everyone working the children to do so in a coordinated fashion. It CAN be done… but only if we shed our current framework of how to educate children effectively.

Thomas Edsall Describes the Vicious Circle of Poverty, Fails to Describe the Best Way Out

March 24, 2017 Leave a comment

In his NYTimes column yesterday, Thomas Edsall offered an insightful and thorough description of the vicious circle of poverty with graphs, research citations, raw data, and paragraphs like this that summarize his findings:

The result is a vicious circle: family disruption perpetuates disadvantage by creating barriers to the development of cognitive and noncognitive skills, which in turn sharply reduces access to college. The lack of higher education decreases life chances, including the likelihood of achieving adequate material resources and a stable family structure for the next generation.

The factors that contribute to “family disruption” are being born to a single parent, being born to a mother who lacks a high school degree, and being born into a household that is below the poverty line.  Mr. Edsall offers evidence that those factors are increasing substantially among less educated populous, noting particularly the non marital birthrate which has jumped among mothers with a high school education level or less but remained steady among college educated parents. This circumstance of birth, in turn, leads to better lives for children born into college-educated married families:

The authors of the “Diverging Patterns” paper — Shelly Lundberg and Jenna Stearns of the University of California-Santa Barbara, and Robert A. Pollak of Washington University in St. Louis – make the case that

there are good reasons to think that children are key to the socioeconomic differences in marriage behavior.

For college graduates, they argue, “marriage has become the commitment device that supports intensive joint investments in children,” a cooperative “joint project of raising economically successful children.” In contrast, they write,

the expected returns to child investments by parents with limited resources and uncertain futures may be lower than for more educated parents with greater and more secure investment capabilities.

At the conclusion of his article, Mr. Edsall draws a series of conclusions, which are summarized below:

First, the spectrum of noncognitive skills and character strengths are a major factor in American class stratification.

Second, neither religious leaders nor practicing politicians nor government employees have found the levers that actually make disadvantaged families more durable or functional.

For liberals and the Democratic Party, the continued failure of government initiatives to achieve measurable gains in the acquisition of valuable noncognitive skills by disadvantaged youngsters constitutes a major liability.

Advocates for the disadvantaged must also highlight and capitalize on the many demonstrably effective antipoverty solutions already well known to the academic, research and nonprofit communities. Without better funded and better crafted organization and advocacy on behalf of the poor, the propaganda and accusations now emanating from the right will ineluctably reshape the law of the land — and once institutionalized, such “remedies” could prove staggeringly difficult to reverse.

For public schools, these translate into the following action steps:

  • Schools need to emphasize noncognitive skills and character strengths. These have long been a part of the “hidden curriculum” that is implicit in codes of conduct and the timely submission of homework, term papers, etc.
  • Schools need to work collaboratively with religious leaders, practicing politicians, and other government employees to identify intervention strategies that have promise. This is easier said than done in the hostile environment that exists today where much of the political capital is spent on shifting the blame and most of the agencies expend much of their efforts fighting for increasingly scarce tax dollars.
  • The media need to emphasize the pointlessness of gathering data that measures “non cognitive skills”. If the public and politicians have learned anything from the “school reform” movement it should be thiscollecting data for the purpose of “measuring performance” of groups of students is pointless and will always lead to the same result. Whenever time is a fixed part of the measurement of anything (e.g. by the time a student enters “x” grade or is “y” years old), the students who have the strongest start in life— in the development of cognitive and non cognitive skills— will always do better. As noted in earlier posts, when I began my career as a public school administrator in the mid-1970s the state of Pennsylvania administered a test to all students and determined that there was a high correlation between test scores and a mother’s education and father’s occupation based on a metric that scaled work from professional careers to laborers. Mr. Edsall breathlessly reported the same findings— forty years later.
  • Intervention programs need to begin MUCH earlier: It is clear that nurturance is crucially important for both the acquisition of non cognitive and cognitive skills. It is also clear that mothers who were not raised in an environment where nurturance was present are challenged to provide that kind of environment without support.
  • Only government programs can provide those programs. That is “Government is the solution, NOT the problem”. Mr. Edsall is correct in his final point: “Advocates for the disadvantaged must also highlight and capitalize on the many demonstrably effective antipoverty solutions already well known to the academic, research and nonprofit communities.” And here’s my hunch: when those advocates highlight the successful programs they will find that the only way to bring those programs to scale is to provide money raised through taxes to make them government programs. As noted frequently in this blog, before we can restore our faith in the ability of anyone to climb out of poverty we need to restore our faith in the ability of government to provide programs for those in poverty. We need to recognize that part of being a citizen in this country is to help those in need and share the fruits of our good fortune.


An Unsettling eSchool Article Describes What Happens When You Give A Kindergartener a Chromebook

March 18, 2017 Leave a comment

I am an an advocate for using technology to individualize and personalize instruction, but I fond myself getting a know in my stomach as I read Laura Ascione’s eSchool article titled “If You Give a Kindergartener a Chromebook”. The article described the experience Jamie Morgan, a Kindergarten teacher in Wichita Falls TX, has using Chromebooks in her classroom of children, many of whom had special needs. This paragraph gave me my first knot:

Because her class from the previous year was high-achieving, no one expected this new class to achieve the same test scores. And although Morgan’s new class entered with “scary” test scores, by the end of the year, their test scores surpassed the high scores of her previous class. Much of that achievement is due to the Chromebooks, Morgan said.

My reaction to this paragraph: TEST SCORES to determine “achievement” for Kindergarten students??!!! Have we lost our collective minds?

As I read on I learned that the students in Ms. Morgans class spend hours on end in front of a computer mastering the use of various Google applications. I have five grandchildren whose ages range from 4 to 11 and I cannot imagine wanting the to spend classroom time on a computer. They enjoy engaging with each other, playing pretend games, writing “plays” to present to us, and engaging in physical activities. My children do everything possible to keep the children off screens.

After reading the article I was more convinced than ever that the last thing Kindergartners need is a course based on Chromebooks. Far better for them to use their open minds to learn another language or, better yet, learn how to ride bikes, hit a tennis ball or baseball, or enjoy walking in the woods.


Can a Mental Health App Replace Face-to-Face Therapy? No… BUT It IS Far Better than Nothing

March 11, 2017 Leave a comment

The Christiansen Institute offers thought provoking weekly articles on the potential for disruptive technology to help public education meet the demands placed on it. This week’s e-issue of their newsletter included an article by Thomas Arnett describing the potential for newly developed apps that rely on Cognitive Behavioral Therapy (CBT) to provide psychological and psychiatric support to schools. Mr. Arnett provides an overview on the use of these new apps as follows:

Untreated mental illness silently plagues a large portion of the United States population. Roughly one in five adults in America suffer from some form of mental illness in a given year, and approximately 60 percent of those cases go untreated. These statistics are similar for teenagers; and educators report that depression, anxiety, and social phobias among youth seem to be on the rise.

Fortunately, a new menu of online mental health resources start to address these unmet needs; and some pioneering options have efficacy results comparable to face-to-face therapy. Programs such as MoodGYMMyCompass, and Beating the Blues teach principles and techniques from cognitive behavioral therapy (CBT) to help people suffering from anxiety and depression. Other online solutions designed for teens, such as Bite Back and Base Education, teach students how to focus, reduce stress, handle difficult emotions, and improve social relationships.

Will online alternatives disrupt traditional face-to-face therapy in the not-too-distant future? To answer that question, consider how they measure up to the disruptive innovation litmus tests.

The “litmus test” poses six questions developed by Clayton Christiansen to determine if a new technology has the potential to be “disruptive”— that is if a new technology can result in a paradigmatic change in the way a business is operated or a service is provided:

1. Does it target nonconsumers or people who are over-served by an incumbent’s existing offering in a market? 

2. Is the offering not as good as an incumbent’s existing offering as judged by historical measures of performance? 

3. Is the innovation simpler to use, more convenient, or more affordable than the incumbent’s existing offering? 

4. Does the offering have a technology enabler that can carry its value proposition around simplicity, convenience, or affordability upmarket and allow it to improve?  

5. Is the technology paired with a business model innovation that allows it to be sustainable with its new value proposition? 

6. Are existing providers motivated to ignore the new innovation and not threatened at the outset? 

In assessing the potential for these CBT apps Mr. Arnett acknowledges that the apps fall short on the second question posed in the “litmus test”. They clearly and unarguably fall short when compared to face-to-face therapy:

Online alternatives to therapy fall short on many fronts when compared to visits with professional psychologists. Current online software cannot read and interpret patient’s verbal and nonverbal cues to diagnose mental illnesses with professional accuracy, nor can it identify patients’ needs, preferences, and life circumstances to develop custom-tailored advice. Software also cannot form relationships with patients to motivate them and hold them accountable.

But even with that clear and unequivocal deficiency, the on-line apps are clearly superior to nothing, which is what troubled teens are getting now. Moreover, with some degree of hybridization is might be possible to use apps to help the limited number of trained school personnel address mental health issues. Mr. Arnett concludes with this:

Although professional psychiatrists, psychologists, and counselors may scoff at the limitations and risks of online mental health support, online options will not threaten professionals’ livelihood any time soon. Online options may be effective for helping people with moderate and untreated anxiety, depression, and addiction, but they have a long way to go before they can match high-quality professional treatment for more debilitating conditions such as severe depression, bipolar disorder, and schizophrenia.

If online mental health solutions have the potential to disrupt the traditional model of mental health care, the unfolding of this disruption cannot come soon enough for K–12 education. School psychologists, nurses, and social workers are in short supply, and many students do not receive needed mental health treatment. Meanwhile, many teachers find themselves shouldering students’ mental health needs on their own. Unfortunately, when mental illnesses go untreated, students pay the price in lower academic achievement and overall well being.

As my colleagues Julia Freeland Fisher and Michael Horn have written, schools that aim to address student achievement challenges need to integrate across factors beyond academics that affect students’ ability to learn. Mental health is definitely one such factor, and convenient, low-cost, disruptive alternatives to traditional mental health care may prove critical for unlocking schools’ capacity to bring high-quality mental health care under their roofs.

I read Disrupting Class, Clayton Christensen and Michael Horn’s book, roughly ten years ago and was and still am convinced that their book was full of ideas with great potential for public education. They used the transistor radio as a metaphor to describe how technology might enhance education. Like the transistor radio, technology could deliver instruction (or in this case therapy) in a rapid, low fidelity but inexpensive fashion to a wider audience. The teachers’ (or in this case therapists’) role would change from being the deliverer of low fidelity content to being the “refiner” of the content: they could offer periodic assessments of whether the student was mastering the content— or in this case whether the content was having the intended impact on the student’s well being. 

Skeptics abound when it comes to using technology in education, a skepticism driven, in part, by the fear that on-line education will ultimately replace teachers (or in this case therapists) completely. But teachers— like the therapists– should not feel threatened by technology, for just as “Online options may be effective for helping people with moderate and untreated anxiety, depression, and addiction” the online options for instruction can only be effective for helping students who are self-actualized and motivated learners. Just as on-line apps for mental health will never be able to “match high-quality professional treatment for more debilitating conditions such as severe depression, bipolar disorder, and schizophrenia”, on-line instruction will never be able to motivate a student to learn and never be able to fully understand the unique needs of each student. That is where the art of teaching comes into play… an art that is being lost as we increasingly teach-to-tests at the expense of addressing each student’s potential.

The Double Edged Sword of Deregulation: States Can Ignore Equity and Civil Rights with Impunity

March 10, 2017 Leave a comment

To no one’s surprise, yesterday Congress passed a bill that will effectively eliminate the regulations that the Obama administration wrote as part of the implementation procedure for ESSA. NYTimers reporter Dana Goldstein summarized some of the elements of the legislation, focussing primarily on the issue of standardized testing:

It is customary for federal agencies to issue detailed regulations on how new laws should be put into effect, and Mr. Obama’s Department of Education did so in November. But some lawmakers from both parties saw the regulations as unusually aggressive and far-reaching, and said they could subvert ESSA’s intent of re-establishing local control over education and decreasing the emphasis on testing.

The Obama regulations pushed states to weight student achievement measures, such as test scores and graduation rates, more heavily than other factors in labeling schools as underperforming. The regulations also required schools to provide parents and the public with an annual report card detailing schoolwide student achievement data and other indicators of success.

Among the most contentious of the Obama rules was one that required schools to test at least 95 percent of their students.

“The regulations were an overreach,” said Robert Schaeffer, the public education director of FairTest, a group that fights standardized testing. But, he said, “the total repeal is also an overreach,” because it targets civil rights regulations as well as testing rules.

And therein lies the conundrum of standards and regulations: how can a government at any level, including the school board level, avoid imposing standards and regulations when housing patterns and the accompanying economic and educational well-being of children is inequitable. As Ms. Goldstein writes, there is a change in sentiment about the role of the federal government in the setting of standards and measurement of performance:

Beginning in the 1980s, moderate Democrats and moderate Republicans tended to agree that the federal government ought to hold local schools to tough standards, and monitor them closely to make sure they were shrinking achievement gaps between different groups of students. The ESSA repeal effort shows that center no longer holds. On the Senate floor Wednesday, Mr. Alexander said a regulatory repeal would “restore to states, to classroom teachers and to school boards decisions about what to do about the children.”

Giving states the authority over education sounds good… but history shows that when states are given that obligation racial and economic justice take a backseat and the education of ESL and handicapped students flags. The fact that 42 states have experienced lawsuits over funding equities should serve as sufficient evidence that some kind of federal oversight is needed and the relatively recent trend of resegregation of public schools should reinforce the need for some kind of federal intervention. But for now, economic and racial justice are taking a backseat. For the sake of poor, minority, ESL, and handicapped children, I hope the pendulum swings back with a vengeance.

Two Technology “Eduprenuers” Offer Good Advice and Scary Insights on ESSA

February 15, 2017 1 comment

School Media reporter Maris Stansbury posted a summary of a Webinar held earlier this week offering educators a description of what they need to know right now regarding the implementation of ESSA. The webinar was led by two technology “edupreneurs”, Steve Rowley and Michael Campbell, and focussed on the need for educators to focus on their state’s accountability system. They indicated that as it stands now, each state will need to develop it’s own peer-reviewed accountability plan using “the Obama accountability template”:

Under the Obama Administration’s accountability template, accountability systems (supported by tracking and data technology) involve four primary indicators: proficiency on state tests; English language proficiency; another academic factor that can be broken out by subgroup; and a “wild card” item that each state can choose within certain guidelines.

States can also set additional benchmarks; for example, these may allow for certain high school benchmarks which would not apply to elementary or middle schools. Each state must be aware of how to accurately track their benchmarks.

This is sound advice from my perspective. The state accountability metrics will inevitably determine the priorities in each school given that the high stakes consequences of failing to meet standards remain in place with ESSA.

But what Rowley and Campbell foresee as possible modifications to ESSA regulations is troubling:

Though DeVos says states should move along with their ESSA plans, the Obama administration’s accountability template part of ESSA is currently under review by DeVos and Congress.

According to DeVos, she and her team are reviewing the Obama administration’s ESSA accountability template because some measures may not be “absolutely necessary.” The new department may release a revised or completely re-written template for states by mid-March this year.

Another potential change is that DeVos’ department may also allow a state or group of states to work together to write their own template through the CCSSO.

If DeVos does allow a state or group of states to devise their own accountability template, it may prove difficult for peer reviewers to determine quality and manage expectations due to a lack of uniformity. However, DeVos and her department could also change the guidelines for peer review—something her camp has not yet mentioned.

It is not surprising that DeVos and “her team” are reviewing the “Obama accountability template” with the intention of eliminating measures that may not be “absolutely necessary.” When that statement is combined with the one indicating that “…DeVos and her department could also change the guidelines for peer review and the fact that she wants States to “fast track” the development of their plans, it wouldn’t surprise me at all if DeVos’ “team” determined that States could rely on off-the-shelf standardized tests to measure school performance. Oh… and if a State wanted to use those same test results to calculate the “value added” by teachers they would be free to do so— without any peer review or input from educators, researchers, or school leaders.

The best piece of advice Rowley gives is this:

“I can’t stress enough—find your state accountability plan and become very familiar with it,” he said. While there is no designated area for locating each state accountability plan, Rowley recommends becoming familiar with your state education agency’s website and keeping a lookout for announcements of things like requests for comment and draft plans.

Outside of state agency websites, keeping abreast of policy news out of Washington, can help you ask the right questions of your state leaders.

With all the news currently pouring out of Washington relative to Russia’s involvement with advisors to President Trump, all the Executive Orders and legislation gutting environmental and banking regulations, and all of the other news reports on things like weather events, accidents, and international clashes, it would be easy for the public to lose sight of announcements on the State Department of Education website seeking comments on draft accountability plans that are likely to be voluminous. And the reality is that there is only so much bandwidth an individual has when it comes to juggling news and the day-to-day responsibilities of work. When tracking all of the news cited above and all of the news about state accountability is added to the workloads of teachers, administrators, and Board members, it will not be surprising if some very bad regulations are put in place in some states.

Welcome to the wonderful world of ESSA….

Metrics Matter… and PISA is the Wrong Metric for What Employers Seek: Creativity, Innovation, and Collaboration

February 14, 2017 Leave a comment

One of the under-reported consequences of ESSA is the fact that STATES will be free to determine the metrics they use to hold schools accountable. As we have witnessed since the advent of NCLB, there is truth in the aphorism “what gets measured gets done”. Fearful that they will be identified as “failing” and potentially closed, public schools across the country have focused on the standardized tests used to measure “success”. In some cases, as described in this blog, schools have eliminated “frills” like art, music, libraries, and recess in order to provide more time for academics so students can succeed on the high stakes standardized tests.

But what if the standardized tests we are using are not a valid measure of success? What if they do not measure the skills employers are seeking today? What if the international standardized tests used to “prove” that our schools are “failing” as compared to other countries in the world are invalid?

According to a recent post by Valerie Strauss the “gold standard” PISA tests are NOT valid measures of national schooling, do NOT measure the skills employers seek, and do not “prove” anything about the quality of our schools. And here is the really bad news for “reformers” who use tests like the PISA as the basis for decrying our schools: there are three assessments of national performance that prove the opposite of what they assert. In fact, the US schools are doing an excellent job in developing the skills that matter to employers.

Ms. Strauss reaches this conclusion based on research done by consultant Nancy Truitt Pierce, a member of the Monroe School Board in Washington state who was appointed by Gov. Jay Inslee to his STEM Alliance Advisory Board. In monthly meetings with Seattle executives she found that they seek the following skills in hiring new staff:

What I hear from the key corporate leaders I meet monthly with is that they want candidates coming out of our public schools who are creative, innovative, collaborative problem solvers. Yes, the candidates must also have strong foundational skills of math, science and language arts but I suggest we are putting too much emphasis on the PISA math score as a key indicator of public school quality. I suggest there are other indicators that would serve us in much better ways. 

Ms. Pierce illustrates why the PISA scores are an invalid metric for a host of reasons and offers three alternatives to PISA, alternatives that measure the skills employers need in today’s workplace. She finds the United States comes out at or near the top on each of them:

• The Global Creativity Index ranks the United States second of 139 countries in the latest results, 2015.
• The 2016 Global Innovation Index ranks the United States fourth out of 128 countries.
• The 2017 Global Entrepreneurship Index ranks the United States first of 121 countries.

These findings undercut the notion that our schools are “failing” and will cause us to “fall behind” in the international competition. Moreover, when you drill down on her findings it becomes evident that our schools are “failing” on the PISA scores only because the US measures the performance of ALL students while other nations measure only the performance of children who come from the most affluent and well educated families.

Ms. Strauss concludes her post with the results from these assessments and this hope for the future from Ms. Pierce:

My hope is to get policymakers to:

1) Clarify our overarching goal to include creativity, innovation and entrepreneurship as key outcomes of our public school system.
2) Focus on the indicators above to demonstrate success.
3) Reduce the overreliance on math tests as the primary metric for success.

I have the same hope for State Departments of Education as they begin the process of designing accountability measures in the coming months. There ARE alternatives to the current metrics. We should use them.

Categories: Uncategorized Tags: , ,