Posts Tagged ‘Measurement’

Trickle Down Testing: NOLA Kindergartners Spend 95+ Hours Taking Computer-Based Standardized Tests

March 27, 2015 Leave a comment

A recent Slate essay, “Welcome to Kindergarten. Take This Test… And This One”, describes the testing gauntlet imposed on NOLA students in the name of accountability. Alexandria Neason writes about the experiences of third year Kindergarten teacher Molly Mansel’s challenges in administering computerized tests to her entering kindergartners. The first challenge was teaching them to use a mouse when most of them were used to swiping screens on phones and pads. Then came the test itself:

Mansel’s students started taking tests just three weeks into the 2014–15 school year. They began with a state-required early childhood exam in August, which covered everything from basic math to letter identification. Mansel estimates that it took between four and five weeks for the teachers to test all 58 kindergarten students—and that was with the help of the prekindergarten team. The test requires an adult to sit individually with each student, reading questions and asking them to perform various tasks. The test is 11 pages long and “it’s very time-consuming,” according to Mansel, who is 24 and in her third year of teaching (her first in kindergarten).

The rest of the demanding testing schedule involves repeated administrations of two different school-mandated tests. The first, Measures of Academic Progress, or MAP, is used to measure how students are doing compared with their peers nationally—and to evaluate teachers’ performance. The students take the test in both reading and math three times a year. They have about an hour to complete the test, and slower test takers are pulled from class to finish.

The second test, called Strategic Teaching and Evaluation of Progress, or STEP, is a literacy assessment that measures and ranks children’s progress as they learn letters, words, sentences, and, eventually, how to read. Mansel gives the test individually to students four times throughout the year. It takes several days to administer as Mansel progresses through a series of tasks: asking the students to write their names, to point to uppercase and lowercase versions of letters, and to identify words that rhyme, for example.

These are pre-tests… and over the course of the year Mansel’s students will spend 95 hours taking these tests… and if Ms. Mansel’s performance rating is based on “growth” you can be certain they will spend many more hours in front of screens instead of playing with blocks or engaging in social play with classmates. All of this is being done in the name of maintaing international competitiveness with other countries. But when do other developed countries introduce reading and what does research tell us about this issue? David Elkind’s EducationNext article in 2012 addressed this question:

Evidence attesting to the importance of developmentally appropriate education in the early years comes from cross-cultural studies. Jerome Bruner reports that in French-speaking parts of Switzerland, where reading instruction is begun at the preschool level, a large percentage of children have reading problems. In German-speaking parts of Switzerland, where reading is not taught until age six or seven, there are few reading problems. In Denmark, where reading is taught late, there is almost no illiteracy. Likewise in Russia, where the literacy rate is quite high, reading is not taught until the age of six or seven.

So if research shows that premature instruction in reading increases the probability of reading difficulties, why are we introducing “academics” early? The short answer is that scientific evidence is immaterial in the politicized environment of American schooling today. The consequences on children are adverse whether or not they learn how to read earlier, for the 95+ hours they spend in front of screens are 95+ hours that could have been spent engaged in activities that would help them develop interpersonal skills and self-regulation.

Because Badges ARE Better Than Degrees, MOOCs Will Eventually Prevail

March 25, 2015 Leave a comment

Earlier this month, Kevin Carey wrote an Upshot article that, if anything, understated the value of “badges” or “verified certificates” as opposed to degrees. As noted in several earlier posts and described in Carey’s article, “badges” are earned by the completion of a series of courses or activities embedded in a course, and when these “badges” are recognized as bona fide credentials the MOOC movement will gain irreversible traction:

Free online courses won’t revolutionize education until there is a parallel system of free or low-fee credentials, not controlled by traditional colleges, that leads to jobs. Now technological innovators are working on that, too.

The Mozilla Foundation, which brought the world the Firefox web browser, has spent the last few years creating what it calls the Open Badges project. Badges are electronic credentials that any organization, collegiate or otherwise, can issue. Badges indicate specific skills and knowledge, backed by links to electronic evidence of how and why, exactly, the badge was earned.

Some of the commenters criticized Carey’s naiveté or his desire to turn higher education into a utilitarian enterprise that turns out “cogs in the machine”. From where I sit, “badges” have tremendous promise for students— especially those students who are NOT engaged in formal education past high school or those directionless students who enroll in college because it is what their parents expect. Moreover, from my perspective as a former employer and a current consumer I can think of several places where “badges” are already in place:

  • Technology repairs
  • Auto repairs
  • Accounting
  • Medical providers
  • Real Estate

The list could be extended endlessly because we are obsessed with credentials, many of which, as Carey notes, are meaningless at worst and obtuse at best:

… H.R. departments know what a bachelor’s degree is. “Verified certificates” are something new. But employers have a powerful incentive to move in this direction: Traditional college degrees are deeply inadequate tools for communicating information.

The standard diploma has roughly the same amount of information that prisoners of war are required to divulge under the Geneva Conventions. College transcripts are a nightmare of departmental abbreviations, course numbers of indeterminate meaning, and grades whose value has been steadily eroded by their inflation.

Instead of the diploma being the coin of the realm for HR staff, a detailed summary of the skills learned at college would take it’s place… in effect a portfolio of the work completed in college would replace the numeric GPA and single sheet of course listings. Once that takes place, HR staff members will likely place a diploma bearing applicant on equal footing with a non-degrees applicant who has superior job-specific skills as evidenced by a certificate. This happens already in technology-related areas where an applicant with a specific product certification is deemed superior to someone with a generic computer technology degree when they are applying. In our school district which used Apple computers, for example, we sought “Apple Certifications” in all applicants and valued experience in a school environment over a generic technology degree. I imagine auto dealers seek the same kind of product-specific training in their applicants and trust that the phlebotomist at my doctor’s office has certification in that area.

As Carey reports, the details on “badges” are being worked out in an organic fashion… and once they are worked out and in place the MOOC revolution will happen rapidly and education at all levels will need to adapt just as quickly.


Is “Personalized Learning” a New Efficiency or a New Form of Schooling?

March 19, 2015 Leave a comment

Audrey Watters is always thought provoking, and her brief essay with the dating title “The History of ‘Personalization” and Teaching Machines” that I just came across teased out the question that is the title of this post. In the essay Watters suggests that personalized learning may not be a liberating force that enables students to learn what they wantwhen they want to. Instead it is a means of feeding students what they need at a pace that enables them to master the skills as defined by those “in privilege and power”.

The distinction is an important one. If “personalized learning” is defined as allowing a student to progress through the Common Core curriculum at a rate of speed that matches their capability to learn, then personalized learning is an efficiency to the factory model, akin to Skinner’s theories. If it is defined as allowing a student to learn what they wantwhen they want to it is more akin to Ivan Illich’s experiential De-schooling model.

From my perspective schooling should be designed to provide students with the tools— the foundational skills— needed to learn-how-to-learn and to connect them with those who can help them expand their knowledge once they have those foundational skills. The foundational skills can be delivered using a behaviorist approach: math and reading fundamentals, fundamental writing skills, and basic analytic skills are based on hierarchical frameworks that lend themselves to various forms of asynchronous on-line instruction. A teacher in these classrooms would intervene when a student is struggling with a particular concept but would not engage in “broadcast” instruction to a group of students. Otherwise, the teacher would serve as a facilitator, helping the student gain self-awareness and self-understanding through dialogue with the teacher, with peers, and with other mentors.

We have the technological capabilities to make the mastery of foundational skills more efficient and effective than it is today and the human resources available to provide each student with a mentor to help them find answers to the questions that they want to find answers for. “Personalized learning plans” should be able to achieve both ends.

High Tech, High Stakes Testing Company “Spies” to Protect Itself Against High Tech Opportunities to Cheat

March 15, 2015 Leave a comment

Years ago when I was in college and contemplating joining a fraternity, one of the benefits touted by some of the Greek organizations was their comprehensive files of final examinations. This trove of old examinations served as a study guide and, in some cases would give you the actual examination itself if a teacher gave the same test year-after-year. Oh… and (wink, wink) in some cases one of your resourceful fraternity brothers might even provide you with questions given earlier in the day.

Seven years ago when I was superintendent of schools in NH a group of juniors and seniors entered the school after hours, broke into a teacher’s office, took final examinations on the eve of the examination, and circulated them among their friends. Because the event happened at the close of the school year, and because the event was not brought to the attention of the principal until after the school year concluded, and because we determined that the pilfering of the examinations required breaking into locked workspaces, we involved the police in our investigation. The arrests and trials that occurred the following school year resulted in national coverage (in part because it coincided with a debate in our community as part of the NH primary election), divided the community and school board over the issue of police involvement in the case, and ultimately led the staff, parents, community, and school board to engage in a dialogue on the ethos of the school.

These two personal experiences came to mind when I read Diane Ravtich’s recent posts on the steps Pearson is taking to prevent cheating on it’s high stakes high tech tests through the use of social media…. and the whole issue raises several questions about the consequences of administering high stakes tests of any kind.

As readers of this blog realize, I am an opponent of high stakes standardized testing. But my opposition to such testing includes opposition to heavily weighted final examinations like those that drove college students in the 60s to compile filing cabinets full of tests and high school students in 2008 to break into teachers’ offices on the eve of examinations. Unfortunately our entire educational system is built on the premise that such tests are a valid measure of learning. Why? Because they are basis for measuring student performance in virtually all colleges. To make matters worse, the AP Tests reinforce this mentality as do longstanding state tests like the NY Regents and now the plethora of new exit examinations that are part of the “reform” movement. Because of this reality, high school teachers administer analogous high-stakes tests to “prepare students for college” or to “get them ready for the State tests”. In short, public education is premised on the need to prepare students for summative examinations that ultimately determine whether they pass or fail a course. When viewed through that lens, is it any wonder that students might do whatever it takes to succeed on a such a high stakes summative examination?

The advent of cell phone technology combined with the desire to do whatever it takes to pass an examination inevitably results in memos from test designers like those issued by Pearson. But those protesting Pearson’s directives should ask this question:

  1. Does your school district administer teacher developed high-stakes final examinations?
  2. Does your school district allow these tests to be open-book tests?
  3. Does your school district allow students to bring cell phones (or handheld devices that access social media) into class when a high-stakes examination is being administered?

If the answer to the first two questions is “yes” then having a cell phone could arguably be acceptable since it would provide access to the trove of information available on the internet… But… what if a student, instead of using the phone to access “Google” uses it to seek an answer from a classmate?  What steps can a teacher take to prevent that from happening?

The easiest workaround to this dilemma from Pearson’s perspective might be to declare that all students not be allowed to bring cell phones into testing venue. If Pearson issued such a directive the howls would be equally loud and equally justified because a third party vendor would be dictating a school policy that may or may not match the ethos of a school. So instead of mandating the collection of cell phones before entering a test area, Pearson issued an excruciatingly detailed process schools can follow to determine if cheating has taken place ex post facto… and justifiably howls of protest are being evoked. So… what IS the solution?

The optimal workaround to this problem would be to completely abandon the use of high stakes summative examinations. Some progressive colleges have figured out ways of assessing performance that does not require letter grades and, consequently, does not rely on summative test scores. So here’s an idea: Instead of using 21st century high tech “spying” to make sure that 19th century assessments are not being breached why not adopt the assessment methods used in progressive colleges and universities? If high schools adopted the “grading” structures of Bennington, Hampshire, and Evergreen instead of those used in traditional colleges we wouldn’t rely on high stakes tests: we’d rely on professional insights of teachers and each students emerging self-awareness.

“Teach to One” in Brooklyn Elicits a Flashback to Shaw JHS in 1971

March 14, 2015 Leave a comment

I am slowly but surely shedding boxes of papers from the past and in doing so have reviewed journals I wrote in college, papers I wrote in graduate school, newspaper articles I wrote as superintendent of schools… and lesson plans from my two years of teaching middle school mathematics at Shaw Junior High School from 1970-72. As described in earlier posts, Shaw Junior High was a rough-and-tumble urban school with 3000 students on a split shift the first year I taught there and a 1600+/- school on a single shift the second year. During the first year, I found that the grade-level materials the district provided were inappropriate for my eight grade students, most of whom had not mastered the basic skills. Like most of my first-year colleagues, I encountered many discipline problems— most of which were brought on as a result of the difficulties I faced getting students engaged with the materials.

I was taking a graduate course on “Curriculum” and to complete an assignment for that course AND help me with my classroom management, I decided to write my own material for one of the sections I taught. I used some of the funds allocated to me to mimeograph a 30+ page set of materials that student could go through at their own pace. My wife, who was an artist, illustrated some of the pages with cartoon caricatures of me exhorting the class to “Do Your Math!”. With over 30 kids in the class, implementing this individualized learning was a challenge, especially since the notion of proceeding at their own pace was alien to the students. After a couple of weeks the students got the knack of it and settled into work on the material. The brightest kids in the class completed the packet quickly, but I found I could assign those same students supplementary problems and they worked on them without disrupting the class. Unsurprisingly, the most disruptive students in the class struggled the most with the work, but they were getting my personal attention to help them. I was observed in the class and while the assistant principal noted I was “not following the prescribed curriculum” he acknowledged that the class was orderly and on task… and my classroom management skills had improved.

This experience flashed before me when I read  Tina Rosenburg’s Fixes column, “Reaching Students One By One” in yesterday’s NYTimes. The “Fix” Rosenburg describes is “Teach to One” a computer-based individualized program that can deliver exactly what I was attempting to deliver 44 years ago… and with Khan Academy, a wealth of web resources, and all kinds of tracking software teachers in PS 29 in Brooklyn are capable of accomplishing the goal of matching lessons to students far more effectively than I could. Rosenburg concludes her essay with this paragraph:

Critics ask a good question: Why should a school try an expensive, disruptive high-tech platform that’s still unproven?   The answer is: in order to prove it. School of One takes comprehensive advantage of technology in ways that let teachers concentrate on teaching. That’s worth getting right. There may be ways to make it cheaper and more effective, but only through further experimentation. As for being disruptive, does anyone defend the current system? “We’re not aspiring to create the least disruptive program,” said Rose. “Our goal is a model that works.”

Taken to its ultimate conclusion programs like “Teach to One” could compel schools to engage in the ultimate disruption: the replacement of age-based grade level cohorts with individualized tracking. Here’s hoping that the standardized testing protocols, with their implicit assumption that all children learn at the same rate, don’t marginalize programs like “Teach to One” that help each and every student experience success.

Grassroots Parent Movement Disrupting Education In a GOOD Way: They are Opting Out of Tests

March 13, 2015 Leave a comment

What happens if a standardized test is given and not enough students are present to make its results valid? Will the testing movement be brought to a halt? Will the parents who keep their children home be fined or penalized? Will the children who stay home be held back? Will the administrators and/or political leaders make examples of the opt-out movement?

Based on reports from schools across NYC and NYS it appears that we might get the answer to the question soon! Based on reports my daughter is feeding me from Facebook posts from schools in Brooklyn and countless articles that are arriving daily in RSS feeds, it appears that a REAL grassroots movement is occurring among public school parents who are dismayed over the time their children are losing to tests and their sense that schools will become test preparation factories instead of centers of inquiry. One of the best articles I’ve read on this topic was Jake Dobkins’, “Public School Kids to Cuomo: Don’t Destroy Our Schools” which appeared in today’s Gothamist. In the article Dobkins describes the rally in his neighborhood school in Brooklyn, PS 10, a rally that featured AFT leaders, local politicians, and a small cadre of media. The article stood out, though, because Dobkins captured the essence of everything that is wrong with the testing in a paragraph full of tough questions that “reformers” are not addressing:

Who would want to work a job where half your yearly evaluation was based on something you had very little control over? What would happen if we fired all the teachers with low-scoring classes, since most of those teachers work in schools in the poorest neighborhoods? How would you replace all those teachers? What would New York look like if all the schools were charters, free to curate their classes with high-performing kids? Where would all the other kids go?

He then describes HIS experience as a teacher, how the test-centric curriculum is affecting his twin sister who teaches in a NYC elementary school. He gives readers a behind the scenes picture of how difficult and challenging it is to be a teacher.

My concern about Dobkins coverage and that of the mainstream media is the emphasis on the union’s participation in these protests. I fear that every time the media coverage includes the name of the union president or of the teachers union’s building representative it will lead readers to the wrongheaded conclusion that these protests are being orchestrated by the unions. The coverage will also lend credence to Cuomo’s allies who will inevitably wave the press coverage and claim the union is behind this movement.

Based on my daughter’s experience as a member of the parent’s organization’s political action team at her son’s school in Brooklyn the unions are NOT directing this movement! Indeed, based on many email exchanges I’ve had with her on the issue I sense that the movement to oppose over-testing is organic to a fault. The parents in her school write their own public relations releases, compose their own letters to city, state, and federal officials, and develop strategies for changing the direction they see their school headed if Governor Cuomo’s package passes. And while the parent group’s written materials include links to information provided on teachers union websites and/or to organizations that the unions sponsor, most of their links are to other parent organizations and to articles and written material they’ve prepared reporting the economic and educational impact to their school should Governor Cuomo’s “reforms” pass. After working for 37+ years in public schools, 35+ as an administrator, I know the difference between a union led movement and a parent-led movement and what is happening in NYC and— from what I’ve read— across the state is being driven by parents and it could be formidable and viral if it succeeds…. and it gives me hope…. and as Dobkins writes in his article’s concluding paragraphs, if hope is combined with action it is possible to stop Cuomo’s “reforms” in their tracks:

The only real hope is that if enough people come out for protests like these, perhaps his political calculus will change slightly- although that seems like a long shot. As a parent with a kid in public school, though, and another one starting soon, that’s what I’m hoping for- the alternative, which is a dystopian future where all the public school kids are packed into classes with 40 kids while their buildings are given away to charter schools and their teachers flee for less stressful careers, is just too depressing to contemplate.

If you agree, consider sending the Governor a polite email, asking him to reconsider this position. The budget isn’t done yet, and there’s still time for him to change his mind.

Getting Governor Cuomo to change his mind about his “reform” initiative is an extremely long shot… but it IS possible to get parents across the state to understand the impact of Cuomo’s “reform” package on their child’s school and if parents across the state continue communicating their opposition with their Assemblymen and State Senators there is a better than 50-50 chance their voices will be heard. There is hope if action is taken now.

The New Yorker’s Take on Cuomo’s Testing Initiative Understates VAM’s Statistical Flaws

March 12, 2015 Leave a comment

The New Yorker provides its readers with a “Daily Comment” feed that provides a column on a timely issue and, it being mid-March, the timely issue is the annual ritual of standardized testing in New York Schools. While I was heartened to see that the New Yorker was covering the emerging grassroots protests in opposition to the testing, I felt that Rebecca Mead’s article, “When a Teacher’s Job Depends on a Child’s Test”, neglected to emphasize the invalidity of the tests that are being used and failed to mention how the Common Core and standardized testing that accompanies the Common Core are being used to undercut the public’s faith in public education and thereby opening the door to privatization. A few phrases from the article will illustrate some of my concerns. To frame the issue of testing and teacher evaluation, Ms. Mead writes:

That teachers should be evaluated is an assertion with which no reasonable person involved with education—from a policy-maker to a parent—is likely to disagree. But how teachers might best be evaluated remains a contested science.

If a reader takes the time to click on the link, they will find that linking teacher evaluations to “growth” as measured by successive standardized tests is NOT “contested” any more than climate change is “contested” for the American Statistical Association has issued a statement indicating that the formulas used by states to evaluate teachers “can’t actually do this with sufficient reliability and validity” and, therefore, should not be used for any personnel decisions.  There is no mention of this anywhere in the article, which is an injustice to those of us who are opposed to the emphasis on standardized testing in public schools.

Further along in the article, Mead writes: “Cuomo’s faith in the results of state tests as the best measure of the abilities of both students and teachers is not universally shared.” At this juncture she emphasizes the political debates on testing at the expense of the educational and statistical ones. While she provides a thorough recounting the thoughtful testimony offered in Albany by Mayor de Blasio and Chancellor Farina, she fails to mention the opposition of Superintendents groups, Principals associations, and school boards across the state, all of whom have expressed reservations about Cuomo’s plans based on the lack of educational value of the tests.

Ms. Mead is also remiss in allowing Arne Duncan to have it both ways on the testing issue, writing:

Even Arne Duncan, the Secretary of Education, whose department ties school funding to test results, has warned that “too much testing can rob school buildings of joy, and cause unnecessary stress,” noting that testing should only be one measure of progress. “In too many places, testing itself has become a distraction from the work it is meant to support,” Duncan wrote last fall.

She either doesn’t know that Mr. Duncan is the one who introduced the linkage of student tests to teacher evaluations with his misbegotten Race to the Top initiative or is giving him a free pass when he makes this disingenuous “warning”.

Ms. Mead provides a good overview of the nascent opt-out movement and notes that New York City is moving away from its reliance on tests as a means of determining placement in magnet schools. And, in the end, Ms. Mead DOES view Governor Cuomo faith in testing as a questionable political move:

In the light of such widespread skepticism about over-reliance on test results—and such widespread consensus about the detrimental effects engendered by teaching to the test—the governor’s doubling down on state test results to assess teachers’ effectiveness seems a questionable calculation.

From my perspective, though, in framing the opposition to testing as “widespread skepticism” Ms. Mead overlooks the settled science on value-added measures in the same way legislators in oil-rich states overlook climate change and, in so doing, perpetuates the public’s belief that standardized tests can be used to measure teacher performance.

Finally, and most importantly, Ms. Mead doesn’t challenge the notion that poor performance on tests is the result of poor teaching and bad schools. The tacit acceptance of Cuomo’s assertion that low test scores can be raised by ridding the schools of ineffective teacher effectively reinforces this meme and consequently reinforces the notion that schools can be “fixed” by firing bad teachers and replacing bad  government monopoly schools with good free-market schools. More than anything, THAT toxic assumption needs to be challenged, for THAT assumption is diminishing the ability to attract people into public school teaching.