In a blog post yesterday, Diane Ravitch quoted from a comment left by testing expert Fred Smith whose comments echoed these questions:
Why isn’t the American Psychological Association speaking out about the misuse of standardized testing? Where are the professors who teach about testing? Why are they silent when children as young as 8 are subjected to hours of testing? Why are they silent when children in middle school are compelled to sit through tests that last longer than college admission tests? Why are they not defending their own standards for the appropriate use of tests? Is their silence a sign of complicity or indifference?
My comment to this post was this:
The psychologists here are analogous to the economists in the lead up to the calamitous Wall Street crash and, as others have noted, the various researchers who give cover to Big Pharma…There are a few renegades who will speak out against the testing, but the corporate line is that testing and measurement are a good thing because it helps feed the paradigm that schools-are-a-business-whose-bottom-line-is-test-scores… And the best tests are those that can be done quickly and cheaply and yield a number that can be put onto a spread sheet and used to establish a rank order… As long as educators use tests in any way to sort and select, standardized tests will be with us.
In the end, we need to change the implicit paradigm of the factory school where students are batched by age cohorts and measured against their age peers and move to a completely individualized and personalized form of instruction where time is the variable and mastery is constant. Such a system would require no more personnel that we use today but would require everyone working the children to do so in a coordinated fashion. It CAN be done… but only if we shed our current framework of how to educate children effectively.
I am an an advocate for using technology to individualize and personalize instruction, but I fond myself getting a know in my stomach as I read Laura Ascione’s eSchool article titled “If You Give a Kindergartener a Chromebook”. The article described the experience Jamie Morgan, a Kindergarten teacher in Wichita Falls TX, has using Chromebooks in her classroom of children, many of whom had special needs. This paragraph gave me my first knot:
Because her class from the previous year was high-achieving, no one expected this new class to achieve the same test scores. And although Morgan’s new class entered with “scary” test scores, by the end of the year, their test scores surpassed the high scores of her previous class. Much of that achievement is due to the Chromebooks, Morgan said.
My reaction to this paragraph: TEST SCORES to determine “achievement” for Kindergarten students??!!! Have we lost our collective minds?
As I read on I learned that the students in Ms. Morgans class spend hours on end in front of a computer mastering the use of various Google applications. I have five grandchildren whose ages range from 4 to 11 and I cannot imagine wanting the to spend classroom time on a computer. They enjoy engaging with each other, playing pretend games, writing “plays” to present to us, and engaging in physical activities. My children do everything possible to keep the children off screens.
After reading the article I was more convinced than ever that the last thing Kindergartners need is a course based on Chromebooks. Far better for them to use their open minds to learn another language or, better yet, learn how to ride bikes, hit a tennis ball or baseball, or enjoy walking in the woods.
The Christiansen Institute offers thought provoking weekly articles on the potential for disruptive technology to help public education meet the demands placed on it. This week’s e-issue of their newsletter included an article by Thomas Arnett describing the potential for newly developed apps that rely on Cognitive Behavioral Therapy (CBT) to provide psychological and psychiatric support to schools. Mr. Arnett provides an overview on the use of these new apps as follows:
Untreated mental illness silently plagues a large portion of the United States population. Roughly one in five adults in America suffer from some form of mental illness in a given year, and approximately 60 percent of those cases go untreated. These statistics are similar for teenagers; and educators report that depression, anxiety, and social phobias among youth seem to be on the rise.
Fortunately, a new menu of online mental health resources start to address these unmet needs; and some pioneering options have efficacy results comparable to face-to-face therapy. Programs such as MoodGYM, MyCompass, and Beating the Blues teach principles and techniques from cognitive behavioral therapy (CBT) to help people suffering from anxiety and depression. Other online solutions designed for teens, such as Bite Back and Base Education, teach students how to focus, reduce stress, handle difficult emotions, and improve social relationships.
Will online alternatives disrupt traditional face-to-face therapy in the not-too-distant future? To answer that question, consider how they measure up to the disruptive innovation litmus tests.
The “litmus test” poses six questions developed by Clayton Christiansen to determine if a new technology has the potential to be “disruptive”— that is if a new technology can result in a paradigmatic change in the way a business is operated or a service is provided:
1. Does it target nonconsumers or people who are over-served by an incumbent’s existing offering in a market?
2. Is the offering not as good as an incumbent’s existing offering as judged by historical measures of performance?
3. Is the innovation simpler to use, more convenient, or more affordable than the incumbent’s existing offering?
4. Does the offering have a technology enabler that can carry its value proposition around simplicity, convenience, or affordability upmarket and allow it to improve?
5. Is the technology paired with a business model innovation that allows it to be sustainable with its new value proposition?
6. Are existing providers motivated to ignore the new innovation and not threatened at the outset?
In assessing the potential for these CBT apps Mr. Arnett acknowledges that the apps fall short on the second question posed in the “litmus test”. They clearly and unarguably fall short when compared to face-to-face therapy:
Online alternatives to therapy fall short on many fronts when compared to visits with professional psychologists. Current online software cannot read and interpret patient’s verbal and nonverbal cues to diagnose mental illnesses with professional accuracy, nor can it identify patients’ needs, preferences, and life circumstances to develop custom-tailored advice. Software also cannot form relationships with patients to motivate them and hold them accountable.
But even with that clear and unequivocal deficiency, the on-line apps are clearly superior to nothing, which is what troubled teens are getting now. Moreover, with some degree of hybridization is might be possible to use apps to help the limited number of trained school personnel address mental health issues. Mr. Arnett concludes with this:
Although professional psychiatrists, psychologists, and counselors may scoff at the limitations and risks of online mental health support, online options will not threaten professionals’ livelihood any time soon. Online options may be effective for helping people with moderate and untreated anxiety, depression, and addiction, but they have a long way to go before they can match high-quality professional treatment for more debilitating conditions such as severe depression, bipolar disorder, and schizophrenia.
If online mental health solutions have the potential to disrupt the traditional model of mental health care, the unfolding of this disruption cannot come soon enough for K–12 education. School psychologists, nurses, and social workers are in short supply, and many students do not receive needed mental health treatment. Meanwhile, many teachers find themselves shouldering students’ mental health needs on their own. Unfortunately, when mental illnesses go untreated, students pay the price in lower academic achievement and overall well being.
As my colleagues Julia Freeland Fisher and Michael Horn have written, schools that aim to address student achievement challenges need to integrate across factors beyond academics that affect students’ ability to learn. Mental health is definitely one such factor, and convenient, low-cost, disruptive alternatives to traditional mental health care may prove critical for unlocking schools’ capacity to bring high-quality mental health care under their roofs.
I read Disrupting Class, Clayton Christensen and Michael Horn’s book, roughly ten years ago and was and still am convinced that their book was full of ideas with great potential for public education. They used the transistor radio as a metaphor to describe how technology might enhance education. Like the transistor radio, technology could deliver instruction (or in this case therapy) in a rapid, low fidelity but inexpensive fashion to a wider audience. The teachers’ (or in this case therapists’) role would change from being the deliverer of low fidelity content to being the “refiner” of the content: they could offer periodic assessments of whether the student was mastering the content— or in this case whether the content was having the intended impact on the student’s well being.
Skeptics abound when it comes to using technology in education, a skepticism driven, in part, by the fear that on-line education will ultimately replace teachers (or in this case therapists) completely. But teachers— like the therapists– should not feel threatened by technology, for just as “Online options may be effective for helping people with moderate and untreated anxiety, depression, and addiction” the online options for instruction can only be effective for helping students who are self-actualized and motivated learners. Just as on-line apps for mental health will never be able to “match high-quality professional treatment for more debilitating conditions such as severe depression, bipolar disorder, and schizophrenia”, on-line instruction will never be able to motivate a student to learn and never be able to fully understand the unique needs of each student. That is where the art of teaching comes into play… an art that is being lost as we increasingly teach-to-tests at the expense of addressing each student’s potential.
To no one’s surprise, yesterday Congress passed a bill that will effectively eliminate the regulations that the Obama administration wrote as part of the implementation procedure for ESSA. NYTimers reporter Dana Goldstein summarized some of the elements of the legislation, focussing primarily on the issue of standardized testing:
It is customary for federal agencies to issue detailed regulations on how new laws should be put into effect, and Mr. Obama’s Department of Education did so in November. But some lawmakers from both parties saw the regulations as unusually aggressive and far-reaching, and said they could subvert ESSA’s intent of re-establishing local control over education and decreasing the emphasis on testing.
The Obama regulations pushed states to weight student achievement measures, such as test scores and graduation rates, more heavily than other factors in labeling schools as underperforming. The regulations also required schools to provide parents and the public with an annual report card detailing schoolwide student achievement data and other indicators of success.
Among the most contentious of the Obama rules was one that required schools to test at least 95 percent of their students.
“The regulations were an overreach,” said Robert Schaeffer, the public education director of FairTest, a group that fights standardized testing. But, he said, “the total repeal is also an overreach,” because it targets civil rights regulations as well as testing rules.
And therein lies the conundrum of standards and regulations: how can a government at any level, including the school board level, avoid imposing standards and regulations when housing patterns and the accompanying economic and educational well-being of children is inequitable. As Ms. Goldstein writes, there is a change in sentiment about the role of the federal government in the setting of standards and measurement of performance:
Beginning in the 1980s, moderate Democrats and moderate Republicans tended to agree that the federal government ought to hold local schools to tough standards, and monitor them closely to make sure they were shrinking achievement gaps between different groups of students. The ESSA repeal effort shows that center no longer holds. On the Senate floor Wednesday, Mr. Alexander said a regulatory repeal would “restore to states, to classroom teachers and to school boards decisions about what to do about the children.”
Giving states the authority over education sounds good… but history shows that when states are given that obligation racial and economic justice take a backseat and the education of ESL and handicapped students flags. The fact that 42 states have experienced lawsuits over funding equities should serve as sufficient evidence that some kind of federal oversight is needed and the relatively recent trend of resegregation of public schools should reinforce the need for some kind of federal intervention. But for now, economic and racial justice are taking a backseat. For the sake of poor, minority, ESL, and handicapped children, I hope the pendulum swings back with a vengeance.
School Media reporter Maris Stansbury posted a summary of a Webinar held earlier this week offering educators a description of what they need to know right now regarding the implementation of ESSA. The webinar was led by two technology “edupreneurs”, Steve Rowley and Michael Campbell, and focussed on the need for educators to focus on their state’s accountability system. They indicated that as it stands now, each state will need to develop it’s own peer-reviewed accountability plan using “the Obama accountability template”:
Under the Obama Administration’s accountability template, accountability systems (supported by tracking and data technology) involve four primary indicators: proficiency on state tests; English language proficiency; another academic factor that can be broken out by subgroup; and a “wild card” item that each state can choose within certain guidelines.
States can also set additional benchmarks; for example, these may allow for certain high school benchmarks which would not apply to elementary or middle schools. Each state must be aware of how to accurately track their benchmarks.
This is sound advice from my perspective. The state accountability metrics will inevitably determine the priorities in each school given that the high stakes consequences of failing to meet standards remain in place with ESSA.
But what Rowley and Campbell foresee as possible modifications to ESSA regulations is troubling:
Though DeVos says states should move along with their ESSA plans, the Obama administration’s accountability template part of ESSA is currently under review by DeVos and Congress.
According to DeVos, she and her team are reviewing the Obama administration’s ESSA accountability template because some measures may not be “absolutely necessary.” The new department may release a revised or completely re-written template for states by mid-March this year.
Another potential change is that DeVos’ department may also allow a state or group of states to work together to write their own template through the CCSSO.
If DeVos does allow a state or group of states to devise their own accountability template, it may prove difficult for peer reviewers to determine quality and manage expectations due to a lack of uniformity. However, DeVos and her department could also change the guidelines for peer review—something her camp has not yet mentioned.
It is not surprising that DeVos and “her team” are reviewing the “Obama accountability template” with the intention of eliminating measures that may not be “absolutely necessary.” When that statement is combined with the one indicating that “…DeVos and her department could also change the guidelines for peer review“ and the fact that she wants States to “fast track” the development of their plans, it wouldn’t surprise me at all if DeVos’ “team” determined that States could rely on off-the-shelf standardized tests to measure school performance. Oh… and if a State wanted to use those same test results to calculate the “value added” by teachers they would be free to do so— without any peer review or input from educators, researchers, or school leaders.
The best piece of advice Rowley gives is this:
“I can’t stress enough—find your state accountability plan and become very familiar with it,” he said. While there is no designated area for locating each state accountability plan, Rowley recommends becoming familiar with your state education agency’s website and keeping a lookout for announcements of things like requests for comment and draft plans.
Outside of state agency websites, keeping abreast of policy news out of Washington, can help you ask the right questions of your state leaders.
With all the news currently pouring out of Washington relative to Russia’s involvement with advisors to President Trump, all the Executive Orders and legislation gutting environmental and banking regulations, and all of the other news reports on things like weather events, accidents, and international clashes, it would be easy for the public to lose sight of announcements on the State Department of Education website seeking comments on draft accountability plans that are likely to be voluminous. And the reality is that there is only so much bandwidth an individual has when it comes to juggling news and the day-to-day responsibilities of work. When tracking all of the news cited above and all of the news about state accountability is added to the workloads of teachers, administrators, and Board members, it will not be surprising if some very bad regulations are put in place in some states.
Welcome to the wonderful world of ESSA….
Metrics Matter… and PISA is the Wrong Metric for What Employers Seek: Creativity, Innovation, and Collaboration
One of the under-reported consequences of ESSA is the fact that STATES will be free to determine the metrics they use to hold schools accountable. As we have witnessed since the advent of NCLB, there is truth in the aphorism “what gets measured gets done”. Fearful that they will be identified as “failing” and potentially closed, public schools across the country have focused on the standardized tests used to measure “success”. In some cases, as described in this blog, schools have eliminated “frills” like art, music, libraries, and recess in order to provide more time for academics so students can succeed on the high stakes standardized tests.
But what if the standardized tests we are using are not a valid measure of success? What if they do not measure the skills employers are seeking today? What if the international standardized tests used to “prove” that our schools are “failing” as compared to other countries in the world are invalid?
According to a recent post by Valerie Strauss the “gold standard” PISA tests are NOT valid measures of national schooling, do NOT measure the skills employers seek, and do not “prove” anything about the quality of our schools. And here is the really bad news for “reformers” who use tests like the PISA as the basis for decrying our schools: there are three assessments of national performance that prove the opposite of what they assert. In fact, the US schools are doing an excellent job in developing the skills that matter to employers.
Ms. Strauss reaches this conclusion based on research done by consultant Nancy Truitt Pierce, a member of the Monroe School Board in Washington state who was appointed by Gov. Jay Inslee to his STEM Alliance Advisory Board. In monthly meetings with Seattle executives she found that they seek the following skills in hiring new staff:
What I hear from the key corporate leaders I meet monthly with is that they want candidates coming out of our public schools who are creative, innovative, collaborative problem solvers. Yes, the candidates must also have strong foundational skills of math, science and language arts but I suggest we are putting too much emphasis on the PISA math score as a key indicator of public school quality. I suggest there are other indicators that would serve us in much better ways.
Ms. Pierce illustrates why the PISA scores are an invalid metric for a host of reasons and offers three alternatives to PISA, alternatives that measure the skills employers need in today’s workplace. She finds the United States comes out at or near the top on each of them:
• The Global Creativity Index ranks the United States second of 139 countries in the latest results, 2015.
• The 2016 Global Innovation Index ranks the United States fourth out of 128 countries.
• The 2017 Global Entrepreneurship Index ranks the United States first of 121 countries.
These findings undercut the notion that our schools are “failing” and will cause us to “fall behind” in the international competition. Moreover, when you drill down on her findings it becomes evident that our schools are “failing” on the PISA scores only because the US measures the performance of ALL students while other nations measure only the performance of children who come from the most affluent and well educated families.
Ms. Strauss concludes her post with the results from these assessments and this hope for the future from Ms. Pierce:
My hope is to get policymakers to:
1) Clarify our overarching goal to include creativity, innovation and entrepreneurship as key outcomes of our public school system.
2) Focus on the indicators above to demonstrate success.
3) Reduce the overreliance on math tests as the primary metric for success.
I have the same hope for State Departments of Education as they begin the process of designing accountability measures in the coming months. There ARE alternatives to the current metrics. We should use them.