Posts Tagged ‘Measurement’

Ohio Governor Kasich Proposes Teachers Learn About the Local Economy… How Will That Work Where No Local Economy Exists?

February 20, 2017 Leave a comment

Diane Ravitch’s post this weekend included a link to an article by Doug Livingston, Akron Beacon-Journal staff writer on Governor John Katich’s proposed mandate that teachers “…see what it’s like to work outside the classroom so they can better match their students to the needs of local employers.” How will this be accomplished?

“It could be as simple as teachers touring local business and having those conversations … to just get a better sense of what those in-demand jobs are,” said Ryan Burgess, director of the Governor’s Office of Workforce Transformation, which put together the group that developed the “on-site work experience” externships and about 20 other proposals in Kasich’s budget.

Asked how kindergarten teachers might benefit from touring a local business, Burgess said it’s never too young to explore a career.

For the next generation of firefighters, he said, teachers who have toured fire stations can work the experience into lessons. “As the governor would say, how do we capture the imagination of young people?”

One of the rebuttals was that schools should require that the business leaders be required to spend a day in schools so that they could understand the public school teachers’ perspective. We did such a thing in the mid 1990s in the MD district I led at the time, whereby some businessperson spent a day in the classroom and the teachers spent a day at the businessperson’s worksite. Unsurprisingly the businessmen came away with an appreciation for how difficult teaching is! You have to be at work by 7:00 AM??? You have over 100 kids a day at the HS??? You don’t have “at-will” bathroom breaks at the elementary school??? You’re on your feet five+ hours a day??? There is so much information to keep track of!!! Egads! Oh, and this was the reaction even with a teacher with them in the classroom who had prepared a skeleton lesson plan for the visiting businessperson to follow! Some of the teachers couldn’t resist pointing out that they needed to work part-time after school to help cover mortgage payments or set aside money for their kids’ educations.

In fairness to Mr. Kasich’s proposal, the exchange worked well the other way. Teachers DID see how the workplace had changed from what they either recalled (from summer temp jobs in college) or read about… But it was purely voluntary and, consequently, rewarding for both parties.

But there is one reality to pulling this off at the state level: the paperwork is DAUNTING! Worksites will require the signing of waivers (many businesses DO have non-governmental workplace regulations to follow!), some sort of structured activities for the visiting teachers to follow, and someone at some level will have to make certain that the teachers comply with the externship regulation. What concerns Becky Higgins, president of the Ohio Education Association and members more than anything is “…the apparent devaluing and extra mandates placed on teachers“. As Ms. Higgins asked:

“Are there any other licensed professionals who have to do an externship outside of their area of expertise to get their licenses approved?”

Will this idea work in Ohio… I have my doubts. I wonder how businesses will feel about finding time and space for thousands of teachers to spend time visiting? More importantly, how will districts with no industry or local businesses deal with this? Will teachers spend time observing in local convenience stores? Or shadowing a local contractor? Or will they need to trave to the nearest town that has a Walmart? Or what if the only local enterprise is a coal mine? Or a military base? Or another government agency? And lastly, I wonder how some employers will feel about inviting a union member to work in their midst?

Two Technology “Eduprenuers” Offer Good Advice and Scary Insights on ESSA

February 15, 2017 1 comment

School Media reporter Maris Stansbury posted a summary of a Webinar held earlier this week offering educators a description of what they need to know right now regarding the implementation of ESSA. The webinar was led by two technology “edupreneurs”, Steve Rowley and Michael Campbell, and focussed on the need for educators to focus on their state’s accountability system. They indicated that as it stands now, each state will need to develop it’s own peer-reviewed accountability plan using “the Obama accountability template”:

Under the Obama Administration’s accountability template, accountability systems (supported by tracking and data technology) involve four primary indicators: proficiency on state tests; English language proficiency; another academic factor that can be broken out by subgroup; and a “wild card” item that each state can choose within certain guidelines.

States can also set additional benchmarks; for example, these may allow for certain high school benchmarks which would not apply to elementary or middle schools. Each state must be aware of how to accurately track their benchmarks.

This is sound advice from my perspective. The state accountability metrics will inevitably determine the priorities in each school given that the high stakes consequences of failing to meet standards remain in place with ESSA.

But what Rowley and Campbell foresee as possible modifications to ESSA regulations is troubling:

Though DeVos says states should move along with their ESSA plans, the Obama administration’s accountability template part of ESSA is currently under review by DeVos and Congress.

According to DeVos, she and her team are reviewing the Obama administration’s ESSA accountability template because some measures may not be “absolutely necessary.” The new department may release a revised or completely re-written template for states by mid-March this year.

Another potential change is that DeVos’ department may also allow a state or group of states to work together to write their own template through the CCSSO.

If DeVos does allow a state or group of states to devise their own accountability template, it may prove difficult for peer reviewers to determine quality and manage expectations due to a lack of uniformity. However, DeVos and her department could also change the guidelines for peer review—something her camp has not yet mentioned.

It is not surprising that DeVos and “her team” are reviewing the “Obama accountability template” with the intention of eliminating measures that may not be “absolutely necessary.” When that statement is combined with the one indicating that “…DeVos and her department could also change the guidelines for peer review and the fact that she wants States to “fast track” the development of their plans, it wouldn’t surprise me at all if DeVos’ “team” determined that States could rely on off-the-shelf standardized tests to measure school performance. Oh… and if a State wanted to use those same test results to calculate the “value added” by teachers they would be free to do so— without any peer review or input from educators, researchers, or school leaders.

The best piece of advice Rowley gives is this:

“I can’t stress enough—find your state accountability plan and become very familiar with it,” he said. While there is no designated area for locating each state accountability plan, Rowley recommends becoming familiar with your state education agency’s website and keeping a lookout for announcements of things like requests for comment and draft plans.

Outside of state agency websites, keeping abreast of policy news out of Washington, can help you ask the right questions of your state leaders.

With all the news currently pouring out of Washington relative to Russia’s involvement with advisors to President Trump, all the Executive Orders and legislation gutting environmental and banking regulations, and all of the other news reports on things like weather events, accidents, and international clashes, it would be easy for the public to lose sight of announcements on the State Department of Education website seeking comments on draft accountability plans that are likely to be voluminous. And the reality is that there is only so much bandwidth an individual has when it comes to juggling news and the day-to-day responsibilities of work. When tracking all of the news cited above and all of the news about state accountability is added to the workloads of teachers, administrators, and Board members, it will not be surprising if some very bad regulations are put in place in some states.

Welcome to the wonderful world of ESSA….

Metrics Matter… and PISA is the Wrong Metric for What Employers Seek: Creativity, Innovation, and Collaboration

February 14, 2017 Leave a comment

One of the under-reported consequences of ESSA is the fact that STATES will be free to determine the metrics they use to hold schools accountable. As we have witnessed since the advent of NCLB, there is truth in the aphorism “what gets measured gets done”. Fearful that they will be identified as “failing” and potentially closed, public schools across the country have focused on the standardized tests used to measure “success”. In some cases, as described in this blog, schools have eliminated “frills” like art, music, libraries, and recess in order to provide more time for academics so students can succeed on the high stakes standardized tests.

But what if the standardized tests we are using are not a valid measure of success? What if they do not measure the skills employers are seeking today? What if the international standardized tests used to “prove” that our schools are “failing” as compared to other countries in the world are invalid?

According to a recent post by Valerie Strauss the “gold standard” PISA tests are NOT valid measures of national schooling, do NOT measure the skills employers seek, and do not “prove” anything about the quality of our schools. And here is the really bad news for “reformers” who use tests like the PISA as the basis for decrying our schools: there are three assessments of national performance that prove the opposite of what they assert. In fact, the US schools are doing an excellent job in developing the skills that matter to employers.

Ms. Strauss reaches this conclusion based on research done by consultant Nancy Truitt Pierce, a member of the Monroe School Board in Washington state who was appointed by Gov. Jay Inslee to his STEM Alliance Advisory Board. In monthly meetings with Seattle executives she found that they seek the following skills in hiring new staff:

What I hear from the key corporate leaders I meet monthly with is that they want candidates coming out of our public schools who are creative, innovative, collaborative problem solvers. Yes, the candidates must also have strong foundational skills of math, science and language arts but I suggest we are putting too much emphasis on the PISA math score as a key indicator of public school quality. I suggest there are other indicators that would serve us in much better ways. 

Ms. Pierce illustrates why the PISA scores are an invalid metric for a host of reasons and offers three alternatives to PISA, alternatives that measure the skills employers need in today’s workplace. She finds the United States comes out at or near the top on each of them:

• The Global Creativity Index ranks the United States second of 139 countries in the latest results, 2015.
• The 2016 Global Innovation Index ranks the United States fourth out of 128 countries.
• The 2017 Global Entrepreneurship Index ranks the United States first of 121 countries.

These findings undercut the notion that our schools are “failing” and will cause us to “fall behind” in the international competition. Moreover, when you drill down on her findings it becomes evident that our schools are “failing” on the PISA scores only because the US measures the performance of ALL students while other nations measure only the performance of children who come from the most affluent and well educated families.

Ms. Strauss concludes her post with the results from these assessments and this hope for the future from Ms. Pierce:

My hope is to get policymakers to:

1) Clarify our overarching goal to include creativity, innovation and entrepreneurship as key outcomes of our public school system.
2) Focus on the indicators above to demonstrate success.
3) Reduce the overreliance on math tests as the primary metric for success.

I have the same hope for State Departments of Education as they begin the process of designing accountability measures in the coming months. There ARE alternatives to the current metrics. We should use them.

Categories: Uncategorized Tags: , ,

What Values ARE We Teaching in Schools… and What Values SHOULD We Be Teaching in Schools

January 29, 2017 Leave a comment

As noted in this blog and in almost every publication I read, there is a tremendous pushback against Betsy DeVos, President Trump’s nominee for Secretary of education. This pushback has led opponents to dig deeply into the myriad donations Ms. DeVos and her husband have made to anti-public school organizations and, in doing so, they have uncovered some alarming speeches and reports. In a post yesterday, Diane Ravitch flagged a Huffington Post article by Rebecca Klein that reported on one such speech:

A conservative Christian group with ties to Donald Trump education pick Betsy DeVos once released a promotional video that proudly featured a speaker describing how Adolf Hitler and others indoctrinated children.

The undated video by the Student Statesmanship Institute ― which trains Christian kids to bring their “biblical worldview” to politics and media careers ― opens with a speech by Michigan leader Richard Posthumus, who describes how Hitler, Joseph Stalin and Vladimir Lenin used children to spread their message.

“Sometimes, it seems like it’s very hard to change the world, the world is so big,” Posthumus says in the video, over a dramatic soundtrack. “But what SSI has done is equip young people with the tools to go out and be ready for the competing ideas that’s in this world.”

The video immediately cuts to another comment.

“Joseph Stalin, Adolf Hitler, Vladimir Lenin: They knew one thing, that to change a culture, to change a country, they had to reach the children,” he says.

This statement reflects the thinking of many that our public schools are NOT “equipping young people with the tools to go out and be ready for the competing ideas of in this world”. Based on this statement, I sense that SSI is seeking to have the “Christian kids” infuse their work with a “biblical worldview”, an effort that is antithetical to the values I believe are needed to provide students with the skills students need to succeed in the world they are about to enter. And what are those skills? A recent article by Penny Loretto in The Balance provides a good list that mirrors virtually every list I scanned in a quick on line search:

The ability to assess a situation is important in all career fields. Being able to gather information and understand multiple perspectives is critical to moving up in your career.

No matter what the job most of them require a minimum understanding of computers. As a recent college graduate you will have a heads up due to your knowledge of word processing, spreadsheets, presentation software, and email.

The ability to manage multiple priorities by being adaptable and flexible will make you a successful employee in almost any field. Things are changing so rapidly in today’s job market that anyone who cannot change course in mid-stream is bound to be left behind.

The ability to solve problems through creativity and a logical thought process will make you a very valuable member of the team. From handling customer complaints to managing a small or large group of people, these skills are a must.

Since so many jobs require people to effectively work teams in order to get work done, one of the things that employers look for when hiring on new people is previous examples of working on teams either in the classroom or previous internships or jobs. 

The ability to plan, organize, and set realistic goals to get the work done in a realistic timeframe, is most important for anyone starting a new job.

Since many company decisions are based on gathering and analyzing data, it’s imperative that a company’s employees know how to take the raw data and translate it into something meaningful and concrete.

The ability to determine the best course of action based on evaluating all options on logic and fact, directly results in creating intelligent solutions to any problem.

The ability to direct and motivate others is a skill that employers extremely value in the workplace. Employees that move up quickly in an organization usually possess this valuable skill. It is also a skill that can be learned through specific training and experience.

The SSIs predominant goal of training “…Christian kids to bring their “biblical worldview” to politics and media careers” is antithetical to the boldfaced items above. Someone trying to impose their views on other individuals will not be able to “gather information and understand multiple perspectives”, “be adaptable or flexible”, “solve problems through creativity and a logical thought process”, “take the raw data and translate it into something meaningful and concrete”, “(evaluate) all options on logic and fact”, or “direct and motivate others”. These inter-related skills all require the ability to be empathetic: to strive to understand the other person’s viewpoint and find a middle ground between their viewpoint and yours or generate a new way of thinking about things that draws on both backgrounds. If one is unable to understand their own mental formations they will be incapable of understanding another person’s.

While I do not believe SSI’s values will help provide students with the skills listed above, I am not certain that our public schools are based on values that will help develop those skills. Our measurement systems place no value on the instruction of “soft skills” and our schools implicitly overvalue competition and undervalue cooperation. The values our schools need to inculcate will have to wait for a future blogpost… but any insights readers might provide are welcome.

Money for Nothing: $3,500,000,0000 in SIG Grants Yield No Change in Test Scores

January 20, 2017 Leave a comment

If the “reform” movement was interested in evidence based decision making, they have now learned that spending $3,500,000,000 on their key ideas yielded no changes in student performance based on test scores and survey results. Mathematica, a non-partisan research group, recently concluded“…multiyear evaluation of School Improvement Grants (SIG) for the Department of Education’s Institute of Education Sciences. It examine(d) the practices used by schools that received grants and schools that did not, examine(d) whether SIG had an impact on student achievement, and examine(d) whether student achievement improved more with some school intervention models than with others.” The four SIG interventions are illustrated below:


As you can see, these intervention models reflect the ideas of reformers, who see the school personnel as the primary cause of “failure” as measured by test results and who also see more-of-the-same (i.e. a longer school day) as a means of improving a school. After implementing these intervention models, though, Mathematic found that nothing happened. Here are the findings in summary form:

  • Schools implementing a SIG-funded model used more SIG-promoted practices than other schools (23 versus 20, out of the 35 practices examined), but there was no evidence that SIG caused schools to use more practices. 

  • Implementing a SIG-funded model had no impact on math or reading test scores, high school graduation, or college enrollment. 

  • Elementary schools had similar improvements in math and reading test scores regardless of which SIG model they implemented.

  • Secondary schools implementing the turnaround model had larger improvements in math test scores than those implementing the transformation model. In contrast, reading improvements were similar for all models. The differences in math improvements across models might be due to factors other than the model implemented, such as existing differences between schools before they received grants.

So… lots of money was spent and no improvements occurred. How might that $3,500,000,000 have been spent more effectively? It seems to me that using a RTTT model to establish wraparound services in “failing schools” would have been better than using the RTTT money to “blow up” traditional schools and replace them with ones using the existing model for schooling…. but that would require the “reform” crowd to acknowledge that exogenous factors (i.e. poverty, re-segregation, homelessness, absent or overworked single parents, etc.) play a role in the attainment of test scores that teachers cannot mitigate by themselves. By focussing everything on the school the RTTT grants overlooked the need for schools to link with parents, with the community, and with the array of social agencies designed to help children and families. It reinforced the silo mentality that separates agencies from each other instead of designing a means of having this agencies work together… and it also played into the notion that privatization would be superior to retaining the current governance model whereby school boards make decisions for children in their locale.

As much as I regret seeing President Obama leave office, I regret even more the opportunity squandered by RTTT. There was a moment in time when an injection of funds could have moved the needle toward interagency cooperation. Instead, we spent billions to show that “reform” doesn’t work… and now we have vouchers to reduce us.

Sorry, FairTest! ESSA is NOT Going to Save the Day… and ALL Teachers are NOT Ready to Administer Well Conceived Assessments.

January 7, 2017 Leave a comment

I generally agree with FairTest’s perspective on the overuse of standardized testing and periodically pore through and greatly appreciate their carefully archived articles describing the flaws of those tests. But I find myself at odds with Mr. Neill’s optimism regarding ESSA, his faith in the ability of all teachers to develop and thoughtfully use assessments, and his unwillingness to accept any form of computerized testing. All of this was prompted by an article of his published in Valerie Strauss’ Answer Sheet titled “How Testing Practices Have to Change in US Public Schools”. The following is an elaboration on the comment I left on-line in response to the article:

Mr. Neill overlooks some sobering political realities in his rosy assessment of the status of standardized testing:

First, 35 states are under the control of Republican governors and/or legislatures and many of those states are of a mind to “run schools like a business”. In doing so they will likely continue their use of standardized tests as the primary metric for “school quality”.
Second, the ESSA rules are likely to be undone by the incoming administration in a fashion that might effectively encourage (or mandate) the restoration of standardized tests. Mr. Neill undoubtedly recalls that the VATs included in RTTT were not a legislated mandate; they were a de facto administrative mandate foisted on public schools and states by the USODE.

Third, there is an implicit belief that the end of standardized testing will result in the simultaneous advent of well conceived teacher developed tests. Having led public school districts from 1981 through 2011 I can attest to the fact that testing practices vary wildly from classroom to classroom and that most teachers never had training in the development of effective assessments. And despite their uneven quality and inconsistency, those teacher developed tests were always “high stakes assessments” from the student’s perspective since they were used to determine if a student passed or failed a course. Moreover, since the advent of NCLB the teacher’s ability to develop assessments has eroded. Teachers in all but the most affluent school districts are primarily focussed on improving standardized test scores. The bottom line on teacher developed assessments: if Mr. Neill hopes to rely the their use for accountability purposes it will require a massive staff development initiative.

Fourth, the call to avoid the use of computerized formative assessments is misguided. Teachers routinely give pencil-and-paper assessments and— yes even in this day and age— worksheets that are presumably designed to determine if a student has mastered the content the teacher presented. Administering those routine assignments via computer frees the time teachers use for grading those quizzes and worksheets enables them to use that time to individualize instruction. The use of well-crafted computerized formative assessments would be a huge step forward if it displaced the quizzes and worksheets that to this day are used as “seat work” in schools.

Mr. Neill’s cause is a righteous one, and I believe we ARE making progress in the way we use assessments at the national level. But I also believe we need to be clear-eyed about the ability for public schools to move in a different direction when it comes to accountability.

Categories: Uncategorized Tags: , , ,

Dog Bites Man: Education Week Report Card Shows that Money Matters

January 6, 2017 Leave a comment

I was going to write yet another blog post describing how Education Week’s “Report Card” metrics prove the obvious: State’s that spend well on education and make an earnest effort to equalize spending do better than state’s that skimp on spending. But fellow blogger Mike Klonsky wrote a brief and eloquent post that does an effective job of explaining this fact. Read it here.