Posts Tagged ‘Measurement’

Personalized INSTRUCTION vs. Personalized LEARNING

November 26, 2014 Leave a comment

A post from Diane Ravitch yesterday provided a link to a report by UCLA professor Noel Enyedy titled “Personalized Instruction: New Interest, Old Rhetoric, Limited Results, and the Need for a New Direction For Computer Mediated Instruction”.

At the outset of the report, Enyedy offers his definition for Personalized Instruction and differentiates it from Personalized Learning:

It is critical to note that “Personalized Instruction” is not the same as “personalized learning,” even though promoters and vendors of technological systems often use the terms interchangeably. Personalized instruction focuses on tailoring the pace, order, location, and content of a lesson uniquely for each student—as when a software program introduces a quiz at some point during instruction and then, based on the student’s score, either presents the student with new material or with a review of material not yet mastered. It is a rebranding of the idea of individualized instruction first promoted in the 1970s, before the widespread availability of personal computers.

Personalized learning, on the other hand, places the emphasis on the process of learning as opposed to attending exclusively to the delivery of content. Personalized learning refers to the ways teachers or learning environments can vary the resources, activities, and teaching techniques to effectively engage as many students as possible—as when, for example, students with a stronger intuitive understanding of the topic are assigned to small groups and given a challenging task to independently extend their understanding while the teacher concurrently works directly with a small group of students who have less prior knowledge of the topic. This interpretation of “personal” does not imply that each student receives a unique educational experience, but instead that students are provided with multiple entry points and multiple trajectories through a lesson.

Enyedy, after emphasizing that the scope of this study is limited to personalized instruction, does an admirable job of outlining the rationale for expanding the use of technology supported “Personalized Instruction”. He describes and analyzes the shortcomings of the factory school model, noting its inability to provide students with the “critical thinking and independent agency” needed to function in a democracy.

In his description of on-line and blended personalized instruction, Enydey identifies one major problem with its implementation to date: inequity.

Research has found that schools in less affluent areas are more likely to use the technology for remedial instruction and for drill and practice, whereas affluent schools are more likely to use technology in ways that advance problem solving and conceptual understanding. These choices, often left up to individual teachers, have serious implications for equity within the classroom and across schools and districts.

Enydey then attempted to perform a meta-analysis of personalized instruction models, an analysis that he acknowledged was limited because there were not a sufficient number of K-12 systems in place. This meant the lion’s share of the studies he analyzed were at the college level where student agency was arguably higher. But the meta-analysis also incorporated one other flaw, which this paragraph flags:

The study examined the standardized test scores for the same three blended learning schools compared with three other schools in the district to see if the gap between high and low achievers was closed by using blended instruction for one year. The study showed that neither blended learning nor face-to-face instruction in this district was particularly successful at improving the performance of lower achieving students. The gap closed 3% in the blended learning schools compared with the 2% improvement in the comparison schools that used conventional teaching methods.

The flaw is that Enydey, like most policy makers, cannot shake the age-based grade-level paradigm that is the basis of the factory school! If we are to abandon the factory model, we have to also abandon the notion that time is constant and learning is variable…. and therefore abandon the use of our current standardized tests to measure “student learning”. That is, we should not measure how much a student has learned in one year, but devise a means of measuring the extent to which a student is making progress in learning-how-to-learn. To date, we have no means of measuring that and so we continue to measure what it EASY to measure instead of what is IMPORTANT to measure, relying on a factory metric instead of a more holistic metric.

Another flaw in the study is the failure to acknowledge and advocate for more access to technology in schools and, more importantly, in the homes of students nd teachers. This paragraph touches on that topic:

In one RAND study,40 based on the actual expenditures of schools that transitioned to an Intelligent Tutoring System for Algebra 1, the cost increased an average of $120 per student for the one course. This increase was reduced to $70 per student per class in schools with a good existing technological infrastructure. However, as many as half the schools in implementation studies undertaken by SRI Education41 and RAND42 were found to need a substantial investment in their technological infrastructure before they could take advantage of Personalized Instruction.

Presiden Obama’s support for a new surtax on phone services to raise $3 billion for schools is a step in the right direction if we ever hope to address the inequities among schools… but in order to provide each and every student with the same opportunities to learn, as emphasized repeatedly in this blog, we need to provide each and every student and teacher with high speed internet at their doorsteps. Until every child can access the power of the internet in their home and every teacher can access the comprehensive data packages outside of school we will be stuck with the models for teaching and learning we have today.

Enydey does note near the end of his paper that the current models in place: on-line instruction and personalized instruction, may be replaced with something different in the future:

The type of computer technology that many believe will lead to transformational change will be technologies built around the process of learning and that attempt to enhance human-to-human interaction, not supplant it: technologies that spark conversations and inquiry; technologies that support these conversations with tools for visualization, simulation, analysis and communication; technologies that allow the students to create physical or computational objects; and technologies that allow students to share their ideas and solutions with their peers and larger social networks for feedback and refinement. There are many promising new models for how computers should be used to support learning.

These promising new models are predicated on two major changes: one a change of thinking on our part and the other an investment in technology. We need to change our thinking by abandoning the factory school model, which will lead to the abandonment of age-based student cohorts and the abandonment of standardized tests as the measure of “learning”. And, we need to make a he investment in our nation’s technology infrastructure by ensuring that each school and home has the means of providing personalized instruction AND learning to students.

Brain Scans Show What We Want to See

November 25, 2014 Leave a comment

An article by Anna North in today’s NYTimes, “Can Brain Science Be Dangerous”, suggests that some of the widely publicized research on brain science, and particularly the brightly colored brain scans that often accompany the articles on this subject, are drawing people towards conclusions that are not true or accurate. My take on the article was that the brain science findings are akin to Rorschach tests: they invite the viewer to see whatever they want to see and project it as reality. This is most troubling when scientific findings about the brain are applied to broad social issues like poverty. Some research suggests that poverty creates stresses in the lives of children that irreversibly limit brain growth while other research suggests that if interventions are applied early and effectively those same stresses can be overcome. This can enable politicians to use “science” to support whatever their views are. With this as a backdrop, one American sociologist, Susan Sered, offered this observation:

…used incorrectly, neuroscience might spread the view that poor people are lesser than others, that they are irrevocably debilitated by their experiences with poverty — or, conversely, that if they fail to respond to programs that science says will help them, it must be their own fault.

North sums up the concerns of social and neurological scientists in her concluding paragraph:

People… seem to find neuroscience extremely persuasive, even when it’s wrong. And this may be part of what critics fear — that images and facts about the brain are so powerful, they can make us believe things we really shouldn’t.

I would take it a step further and suggest that “images and facts about the brain are so powerful, they can support our wrong beliefs and close our minds to contradictory and new findings.” 


Categories: Uncategorized Tags: ,

Charters, Tracking, Real Estate, and Re-Segregation

November 24, 2014 Leave a comment

Two recent articles illustrate how charter schools and tracking amplify the trend of re-segregation in public schools, a trend that is tied inextricably to zip codes.

Last week Atlantic blogger Sonali Kohli posted an article titled “Modern Day Segregation in Public Schools” that described how the longstanding practice of tracking exacerbates the emerging trend of resegregation using a recent NJ State Board decision to illustrate her point. The problem local districts face is that their use of standardized test scores, GPAs, and teacher recommendations to determine who qualifies for advanced courses or “gifted and talented” courses results in a disproportionate number of white students being placed in classrooms. The USDOE, which requires States to use standardized testing to identify “failing schools”, is intervening and recommending some sort of de facto quota system whereby the districts who using tracking of any kind must ensure that the student populations in advanced levels and/or gifted and talented programs mirror the racial patterns of the district as a whole.

Today’s NJSpotlight features an insightful blog post by Laura Waters titled “Self Selection of Public Schools, New Jersey’s Double Standard” which uses a recent quote from Newark’s CEO Cami Anderson as the jumping off point:

In response to a question regarding a four-point drop in test scores among Newark students enrolled in traditional elementary schools, Anderson acknowledged that the city’s growing sector of public charter schools serves children who are less poor and less likely to be classified as eligible for special-education services.

“I’m not saying they [the charter schools] are out there intentionally skimming,”said Anderson, “but all of these things are leading to a higher concentration of the neediest kids in fewer [district] schools.”

Charter advocates winced and went on the defensive. Charter detractors grinned and high-fived. Both reactions miss the point.

The point Ms. Waters makes is one that I’ve made in this blog on several occasions:  parents in affluent zip codes get a wider range of choices than parents in urban areas serving children raised in poverty:

Given a choice between Newark and Millburn, motivated parents of any means would most likely choose to send their kids to school in the latter, as long as they could afford the freight of the median house cost of $665,000 and an average annual tax bill of $20,439. This sort of self-selection — skimming, if you will — is regarded as a cause for applause, an emblem of good values and good parenting. In New Jersey we embrace school skimming. With our ZIP code-driven district-assignment system, town choice is school choice. If you can afford granite countertops then you can afford great public schools.

Ms. Waters chides those who criticize charters in urban systems for skimming of the best and brightest children of engaged parents without challenging the de facto segregation that results from real estate choices.

But in Newark, a system that allows families to choose more successful, albeit nontraditional, public schools is suddenly suspect. A proud N.J. tradition is transformed into a scourge, simply because we’re talking about poor parents and not rich ones.

Waters describes offers a lukewarm proposal for solving the problem but closes with a statement that poses a conundrum for those of us who do not want to see the expansion of for profit charter schools:

So what’s the answer? Paul Tractenberg half-heartedly suggests county-wide school districts, although he concedes that such a conversion is a “quintessential political third rail” due to New Jersey’s addiction to local control. Whatever that answer is or, indeed, whether it exists, let’s agree that parents should be able to make school choices for their children, and that their right to do so shouldn’t rest on their ability to afford granite countertops.

Here’s the conundrum from my perspective: politics is the art of the determining what is possible as opposed to what is ideal… and what is possible NOW under the system we have in place is offering choices within districts and ensuring that those choices within districts do not result in segregation by race. Charter schools, be they for-profit or not-for-profit, are the most politically viable solution we have NOW. The political reality as I see it is that any solutions that approach the “ideal” (e.g. choice among all schools, equitable funding across the board, early intervention programs for children raised in poverty, county districts that enable the re-drawing of boundaries to achieve racial and socio-economic balance, etc.)  are, to echo Mr. Tractenberg, a “quintessential third rail”.

In earlier posts I’ve proposed a third rail solution with relatively voltage: the redirection of ALL federal funds to less affluent districts in states, like NJ, where civil rights violations are found and/or courts have determined that existing funding mechanisms are unconstitutional. It’s been 60 years since Brown vs. Board of Education and in those years the Federal funding for public schooling has increased substantially and enrollment patterns driven by zip codes have increased segregation across the country. One look at the data cited in each of these articles should persuade anyone that segregation continues within school districts and between school districts and the current tools in play, “strongly worded directives” and de facto quota systems are unsatisfactory and simplistic solutions to deep and complicated problems. Redistribution of federal funding to address issues decided in State and/or federal courts would promote local solutions to these problems.

From my perspective, the ideal solution to all equity issues would be the institution of constitutionally equitable per pupil allocations, the abandonment of age-based cohorts, and the implementation of individualized instruction programs. Such a reformatting of school would help schools develop self-actualized learners who have the interpersonal skills to thrive in the multi-cultural world our students are living in. If we want a fully engaged electorate of well-informed voters we cannot continue operating public schools that segregate students based on their learning rates, the knowledge they bring into schools when they enter, and whether their parents can afford granite counter tops. Such a system only reinforces what we have in place today and the direction we are heading.

Promiscuous Promotions Start Early!

November 23, 2014 Leave a comment

Frank Bruni’s column in today’s NYTimes, “Promiscuous College Come-Ons” made me want to laugh and cry at the same time. Bruni has consistently trumpeted the market based “reforms” of Bloomberg et al and after reading this column it is evident that he cannot see the ultimate consequences of subjecting all schools to the marketplace…. which led me to make this comment:

How do you reconcile your criticism for colleges shamelessly marketing themselves with your avid support for the “market based” school reform? In the world of school reformers EVERY school will need to spend money on recruitment and there will be little or no regulation on how the schools advertise themselves. The only good result of “market based” charter schools will be that by the time a student graduates from high school they will be inured to the “promiscuous promotions” presented by colleges.

The other ironic criticism was his implication that Swarthmore was lowering the rigor of its application process in order to game the statistics in the US News and World Report that give schools a higher ranking if they are “more selective”. Again, there is much data to support the fact that for-profit charter schools do the same thing to demonstrate the “demand” for their openings…. and there is Campbell’s Law which Wikipedia defines as follows:

“The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

To paraphrase: institutions— even respected institutions— will do everything possible to game the system in order to improve their standing in the eyes of “consumers”. If we want to measure effectiveness we should make sure the data we use to do so cannot be easily gamed and the algorithms we use emphasize the most important qualities we are seeking.

If we want to institute market based schools, we should be prepared for more “promiscuous promotion” at earlier and earlier ages… and maybe add b.s. detection to the Common Core.

Parents Push Back on Tests

November 10, 2014 Leave a comment

The New York Times is finally noticing that parents are pushing back against standardized testing… and with some coaching might begin to recognize that the whole standardized testing movement is based on simplistic and wrongheaded thinking.

Today’s paper features an article by Lizette Alvarez describing the parent pushback against the wide array of mandated standardized tests in FLA, testing that expanded greatly as a result of Jeb Bush’s initiatives a decade ago that was compounded by RTTT. I left a comment that was just under the 1500 character limit that made the following points:

  • Standardized tests do not measure the quality of education,
  • The new test results are lower because of the way they are scaled
  • Using those tests to measure teacher performance is invalid and simplistic.
  • Using test as the primary measure for “quality” will increase the focus on testing in the classroom.
  • Politicians love standardized tests!

Standardized tests do not measure the quality of education: States have administered standardized tests for decades and the results are always the same: affluent districts serving the children of well educated parents always outscore the financially strapped districts serving children raised in poverty. The new test will be no different EXCEPT that there will be more failing students and schools.

The new test results are lower because of the way they are scaled: The new Common Core tests expand the number of “failing” schools because they are scaled to an artificial and idealized standard that assumes all students will graduate from high school ready for college instead of being scaled to the mean scores of an age cohort as they have been in the past. As a result, more students are “failing”, more schools are “falling”, and more districts are “failing”. Whether this is a bug or a feature depends on the extent to which you believe that politicians are in cahoots with squillionaires who are investing in for-profit charter schools and technology companies. For now, I’m on the fence. I think some politicians listen to investors but I also believe some politicians are naively convinced that schools CAN be measured based on test scores and test scores CAN improve if kids and teachers work harder. They believe this in large measure because considering the alternative might require them to raise taxes to provide more support for children raised in poverty.

Using those tests to measure teacher performance is invalid and simplistic: The value-added methodology that uses test scores to measure “growth” of students, teachers, and schools is a statistical artifact. There are reams of scholarly articles that undercut the validity of this approach. I’ve written about this frequently on this blog… enough said.

Using test as the primary measure for “quality” will increase the focus on testing in the classroom: When test results are used to evaluate teachers, to determine if schools will be closed, and to determine if entire districts will be taken over by the state or turned over to for-profit entrepreneurs, it is not surprising that they become the focal point in every classroom…. and as noted in a post earlier today, when those districts are strapped for money they cut everything BUT test preparation activities.

Politicians love standardized tests! They love the tests because they yield precise data that is inexpensive to collect and prove that schools are failing because of “bad teachers” and if the TEACHERS are the problem the fix for “failing schools” is inexpensive and fast: replace the “dead wood” teachers with new (and less expensive) teachers. Voila!

It is heartening to see that the Times is reporting on this nascent movement among parents… but somewhat distressing to see them reporting on this a month after the dust-ups in FLA and a week after a close election in that state and in several other states where “reform minded” governors got elected. Maybe after a spring of rebellion on tests some Presidential candidate will stand up against the test-and-punish approach and begin supporting the importance of public education and the effects poverty has on learning.

NYTimes Continues diBlasio Bashing

November 7, 2014 Leave a comment

The NYTimes, the allegedly “liberal” newspaper of record, continues its relentless bashing of Bill diBlasio’s efforts to provide enhanced services to community schools and lionizing of Michael Bloomberg’s “reform” approach of closing schools and replacing them with charters despite the lack of evidence that Bloomberg’s 12 years in office made any difference whatsoever in the overall performance of NYC schools.  In an editorial in today’s paper, titled “New York Needs a Stronger School Plan, the Times editor’s open by acknowledging that diBlasio’s plan “…needs to be fleshed out in greater detail before it can be fully appraised” but then proceeds to appraise it by asserting it:

…might not be sufficient to remake the city’s lowest-achieving, most-dysfunctional schools. The plan could easily delay action on schools that are in desperate straits and should be reorganized or closed in fairly short order.

They repeated their claim from earlier this week that the kind of integrated service approach was unsuccessful in SOME schools in Cincinnati based on THEIR analysis of results. In the same article they effectively acknowledge that in SOME cases Bloomberg’s ideas did not succeed, but no matter, diBlasio should continue that course of action. The section of the editorial that I found especially maddening was this:

Schools that do not meet performance benchmarks face “consequences,” the city says, including changes in leadership or reorganization. Much, of course, will depend on the benchmarks. If they are weak, mediocre or failing schools will continue to operate, instead of being reconstituted or shut down.

Bloomberg’s benchmark tests during the first two terms of office, the ones he used to tout the success of his test-and-punish approach, turned out to be bogus— which is worse than “weak” or “mediocre”. When the tests were statistically adjusted it seemed that the success rate of his reconstituted schools wasn’t nearly so good. And, as readers of this and other progressive blogs know, the charter schools that have remarkable achievement rates often had remarkably low retention rates, remarkably high student suspension rates, and de facto selectivity. So far diBlasio has not tried to juke his statistics or make wild claims about fast turnarounds. Schools are human enterprises not factories, and changing human behavior is much harder and more time consuming that changing the procedures on an assembly line in a factory. The Times should know that and help the readers understand it. Instead, they are reinforcing the idea that there is a quick, cheap, and direct way to fix a complicated problem that will require additional resources.


A Sigh of Relief in CA… but an Omen as Well

November 5, 2014 1 comment

The non-partisan race for CA State Superintendent of Public Instruction pitted two Democrats against each other: the incumbent Tom Torklason who advocates continuation of the current model for non-profit public schools and neo-liberal challenger Marshall Tuck who advocates for market-based reform that would provide increasing amounts of public funding to for-profit charter schools. There were two other bright line distinctions between the candidates: Torklason opposed the Vergera decision that eliminated tenure for teachers in CA and opposes the use of test scores to measure teacher effectiveness; Tuck supported the Vergera decision and the use of VAM. As Mokota Rich of the NYTimes wrote, the CA race drew a lot of outside attention and money because it was viewed “…as a proxy for the national debate over teacher tenure rules, charter schools and other education issues that have divided Democrats.”

An hour ago Alexei Koseff of the Sacramento Bee reported the results, and as the title of this blog post intimates Torklason won. But the relatively narrow margin of victory (53-47) is an indication that the education reformers are making a substantial dent in public opinion. As Koseff noted, the Vergera case was a focal point of Tuck’s campaign:

Tuck built his campaign on the case, galvanizing supporters after a judge declared the policies unconstitutional in June. He wielded the ruling against Torlakson like a bludgeon, spending most of his public appearances urging California to reject the “status quo” and get behind the decision.

Over the past year, progressive bloggers lamented the attention the Vergera case generated while grudgingly acknowledging it was politically shrewd, particularly in its timing. Whether the case won or lost, it shone a light on “tenure” which is perceived by many voters as “protecting bad teachers”. This notion, in turn, plays into the hands of “reformers” who assert that our nation’s low performance on international tests can be easily fixed by ridding classrooms of “bad teachers who are protected by tenure”. This is akin to the logic that the budget can be balanced if we eliminate “waste, fraud, and abuse” but is a logic that resonates more than Torklason’s reaction to the Vergera case, as reported by Koseff:

Torklason said the case was an attack on teachers, who should not be blamed for the failings of the education system. He pushed for more school funding and was the rare politician to speak out in favor of extending tax hikes voters approved in Proposition 30 when they begin to expire in two years.

Given the choice of “reform” which can be accomplished by “eliminating bad teachers” and introducing competition into education with a reduction in taxes or the “status quo” which requires more funds to help existing public schools without a reduction in taxes makes “reform” appealing.

Here’s my optimistic take-away from 3000 miles away:

  • The CA voters did not support the “status quo”, they rejected Tuck. Given the framing of these two articles from opposite sides of the country, it is evident that Tuck successfully framed Torklason as a defender of the “status quo”, particularly a defender of “the unions”. This framing accounted for the narrowness of the victory.
  • CA voters are not prepared to adopt privatization as a solution, even if it results in tax savings. While the two media reports did not see the race as a referendum on spendingTuck’s campaign implicitly rejected the argument that more spending was needed on schools and that “introducing competition” would preclude the need for a continuation of tax hikes in the future.
  • CA voters see through the fallacy of defining teacher quality based on standardized test results. Both writers mentioned VAM as a political issue and Tuck and Torklason were on opposite sides of the fence on this issue. To the extent that voters rejected Tuck’s proposal for change they also rejected the notion that VAM is worthwhile.

But here’s why I see the results as ominous:

  • The “reformers” framed the debate: By making the debate “reform” vs. “status quo” they simultaneously framed the debate as “taxpayers” vs. “unions” and “accountability” vs. “laissez-faire”. This, in turn, placed those of us who see the need for a holistic approach, one that would require more resources for the children raised in poverty, on the defensive.
  • The anti-union sentiment is strong: When 47% support an explicitly anti-union candidate it is a sign that reliance on union voters is shaky. In a future post I will outline my thoughts on why the support for unions is eroding, but it is increasingly evident that mainstream voters are put off by candidates who get large sums of money from organized labor while ignoring the money streaming into candidates from the .1%.
  • The failing schools meme is not going away: While Torklason did advocate for more funding for schools, he not use the campaign to dispute the fact that schools are failing and emphasize that schools serving children raised in poverty need more resources if Californians hope to see an increase in aggregate test scores.
  • Dark outside money is moving into school politics: The outside money invested in Tuck’s campaign necessitated a proportional response from teachers’ unions and that resulted in this outcome:

The contest drew more than $20 million in outside spending, more than for any other elected office in California this fall. Billionaire philanthropists looking to overhaul California’s low-ranking public schools squared off against powerful teacher unions defending their job protections, with both sides spending heavily on television attack ads and nasty mailers.

To paraphrase Al Shanker, when unions and school boards call each other names, the public believe them both. When school campaigns result in adults taking sides, and both sides spend heavily on “attack ads and nasty mailers“, the public believes what both sides are claiming and school children lose in the end. THAT is the MOST ominous result of the CA election.