Archive

Posts Tagged ‘Art of teaching’

Robots Cannot Replace Humans When it Comes to Grading Essays— But No Matter! They are Cheap, Fast, and Unbiased.

July 17, 2018 Leave a comment

NPR recently ran a story by Tovia Smith on the use of robots (or more precisely computers) to grade student essays and found to no educators’ surprise, that they were not not up to the task. The story opens with this quiz:

Multiple-choice tests are useful because:

A: They’re cheap to score.

B: They can be scored quickly.

C: They score without human bias.

D: All of the above.

It would take a computer about a nano-second to mark “D” as the correct answer. That’s easy.

But now, machines are also grading students’ essays. Computers are scoring long form answers on anything from the fall of the Roman Empire, to the pros and cons of government regulations.

From this point forward in the story, Ms. Smith provides an account of the expansion of the use of computer-graded essays, work done by MIT research affiliate, Les Perelman, who has developed algorithms that generate nonsense responses to machine-graded essays that yield high scores, a rebuttal to Mr. Perelman’s work by ETS, who argue, in effect, that if someone is smart enough to game to the essays they deserve the high grade, and the “cat-and-mouse” game underway to catch students who use algorithms in states that have adopted computerized grading.

At the end of the report, Ms. Smith hits on the real problem with computerized grading: it compels teachers to teach students formulaic writing.

Indeed, being a good writer is not the same thing as being a “higher-scoring GRE essay writer,” says Orion Taraban, executive director of Stellar GRE, a tutoring company in San Francisco.

“Students really need to appreciate that they’re writing for a machine … [and when students] agonize over crafting beautiful, wonderfully logically coherent and empirically validated paragraphs, it’s like pearls before swine. The computer can’t appreciate what this person has done and they don’t get the score that they deserve.”

Instead, Taraban tutors students to give the computer what it wants. “I train them in fabricating evidence and fabricating fake studies, which is a lot of fun,” he says, quickly adding, “but I also tell them not to do this in real life.”

For example, when writing a persuasive essay, Taraban advises students to use a basic formula and get creative. It goes something like this:

A [pick any year] study by Professor [fill in any old name] at the [insert your favorite university] in which the authors analyze [summarize the crux of the debate here], researchers discovered that [insert compelling data here] … and that [offer more invented, persuasive evidence here.] This demonstrates that [go to town boosting your thesis here!]”

It results in a kind of mad-lib writing that is anything but artful, thoughtful, or pleasing to read. But it is cheap, easy, unbiased, and unbiased…. and the ultimate triumph of efficiency over excellence.

 

Advertisements

DC Miracle Story Evidence of Traction of “Fake News”

June 30, 2018 Comments off

A few days ago AP writer Ashram Kahlil wrote an article titled “DC’S Public Schools Go from Success Story to Cautionary Tale“, a story that was picked up by NPR and some other mainstream news outlets. But alas, Time magazine is unlikely to run a cover story with Michelle Rhee sitting on a dunce stool or holding a broken broom.

In 2008, both Time and Newsweek offered overs depicting then rising star Michelle Rhee, the no-nonsense DC Superintendent who pledged to clean up the public schools in that city by implemented a test-and-punish policy that garnered support among those who thought schools needed to be operated using a no nonsense “business” approach and negative attention from anyone who actually worked in schools and realized that instead of a clean sweep their schools needed new floors, new lighting, and more money.

Since 2008, funding for schools has diminished, in some cases in real dollars and in all cases in terms of actual funding… and the consequences of test-and-punish has not been the improvement of test scores but rather the expansion of corruption in the administration of those high stakes tests. And DC has had its eyes blackened badly. As Mr. Kahlil reports:

As recently as a year ago, the public school system in the nation’s capital was being hailed as a shining example of successful urban education reform and a template for districts across the country.

Now the situation in the District of Columbia could not be more different. After a series of rapid-fire scandals, including one about rigged graduation rates, Washington’s school system has gone from a point of pride to perhaps the largest public embarrassment of Mayor Muriel Bowser’s tenure.

This stunning reversal has left school administrators and city officials scrambling for answers and pledging to regain the public’s trust.

A decade after a restructuring that stripped the decision-making powers of the board of education and placed the system under mayoral control, city schools in 2017 were boasting rising test scores and a record graduation rate for high schools of 73 percent, compared with 53 percent in 2011. Glowing news articles cited examples such as Ballou High School, a campus in a low-income neighborhood where the entire 2017 graduating class applied for college.

Then everything unraveled.

An investigation by WAMU, the local NPR station, revealed that about half of those Ballou graduates had missed more than three months of school and should not have graduated due to chronic truancy. A subsequent inquiry revealed a systemwide culture that pressured teachers to favor graduation rates over all else — with salaries and job security tied to specific metrics.

The internal investigation concluded that more than one-third of the 2017 graduating class should not have received diplomas due to truancy or improper steps taken by teachers or administrators to cover the absences. In one egregious example, investigators found that attendance records at Dunbar High School had been altered 4,000 times to mark absent students as present. The school system is now being investigated by both the FBI and the U.S. Education Department, while the D.C. Council has repeatedly called for answers and accountability.

It takes a long time to inculcate a culture of support, but a culture of fear can be implemented rapidly… and once that culture is in place it is hard to change. And that culture is especially hard to change when “salaries and job security tied to specific metrics” and those metrics can be manipulated by those who will be damaged the most: the administrators and politicians who based their careers and campaigns on their ability “…to improve public education.”

And who implemented this culture that resulted from salaries and job security tied to specific metrics?

As Mr. Kahlil reports in his closing paragraphs… it was none other than Michelle Rhee:

Critics view the problems, particularly the attendance issue, as an indictment of the entire data-driven evaluation system instituted a more than a decade ago when then-Mayor Adrian Fenty took over the school system and appointed Michelle Rhee as the first chancellor. Rhee’s ambitious plan to clear out dead wood and focus on accountability for teachers and administrators landed her on the cover of Time magazine holding a broom. But now analysts question whether Rhee’s emphasis on performance metrics has created a monster.  

Readers of this blog know the answer to that question: there is no doubt that the test-and-punish methods supported by Ms. Rhee and her follow reformers created a monster… but it’s serving their purposes: it is creating the impression that public schools are not only “failing” based on those test scores, but they are now “corrupt” because of the actions of a handful of administrators whose continued employment required them to boost them.

And here’s one fact that remains the same today as it was in 2008: the teachers who work in poverty stricken urban and rural districts like DC are giving their hearts and should to the jobs and the administrators in those same schools are being over backwards to support them. But a cover article lionizing public school teachers and principals is not nearly as compelling as one showing that an inexpensive one-size-fits-all solution is the best way to fix schools.

 

 

 

 

 

 

Failing Students Based on Standardized Test Scores is Irrational… but Persists Because of Our Mental

May 16, 2018 Comments off

Diane Ravitch wrote a post drawing from an op ed piece written by Michigan teacher Nancy Flanagan decrying the Michigan’s third grade “mandatory retention legislation” that requires schools to fail any third grader who scores below a certain level on the standardized tests used to determine “proficiency”. Ms. Flanagan writes:

What to do about children who are not confident readers in third grade? We could begin by taking the resources it will cost to retain them for a year (minimally, $10K per child) and spending it on supplemental instruction: in-school tutoring, libraries filled with easy, engaging books, after-school programs, summer reading clubs and books for children to take home.

We could offer smaller instructional groupings. We could stop the merry-go-round of silver-bullet ‘solutions,’ from emergency managers to charter schools to one-size-fits-all scripted curricula.

We could genuinely invest in our children, believing in their capacity to master not only the skill of reading, but to become an informed, productive citizen.

Reading this brought to mind my favorite Peter Senge quote: “Structures of which we are unaware hold us prisoner”…

Politicians, parents, and pundits view time as a constant and learning as a variable instead of the other way around because we group and asses children based on their age. Standardized testing reinforces this structure and when standardized testing is linked to “promote” students from one grade level to the next by politicians it creates a group of “failures” who in many cases just need time to mature. It would be preposterous to “fail” a child whose physical maturation rate was different from his age peers, but somehow it is “rational” to “fail” a child who can’t learn reading and math skills at the same rate as his or her age peers… especially if that learning is measured by a seemingly precise tool like a standardized test!

It is possible to tailor instruction to meet the unique needs of each child by matching instruction to their rate of learning… but our current structure reinforces the practice of grouping children by age and comparing children to each other, which holds us prisoner to the current factory paradigm.

Ms. Flanagan essentially urges us to change the dominant paradigm by changing the one-size-fits-all scripted curricula. I wold take it a step further to suggest we need to stop the one-growth-rate-fits-all structure we impose when we group and assess children by age.

Truthdig Article Reminds Us that the FIRST Amendment is Imperiled

May 12, 2018 Comments off

I just finished reading Truthdig columnist Amy Goodman’s inspiring account of speech given by the 2018 Teacher of the Year, a speech that received little press coverage because the press was not invited to hear it.   As Ms. Goodman reports, Mandy Manning, who teaches math and English to refugee and immigrant students at the Newcomer Center at Joel E. Ferris High School in Spokane, Washington, offered these words to the President and the Cabinet officers in attendance:

“I am honored and humbled to be the vehicle through which my students may tell their stories,” Manning said in the historic East Room, as billionaire Education Secretary Betsy DeVos and Labor Secretary Alex Acosta looked on. “I am here for David, a future IT specialist who hopes to one day be able to attend university. I am here for Tamara, who is currently studying pre-med at Eastern Washington University. I am here for Safa and Tara, both future elementary school teachers. I am here for Solomon and Gafishi, who believe that the United States is the place where they have found the center of their lives, where they can have dreams and hopes to be someone.”

Ms. Manning gave her address to a small gathering that did not include any members of the press and was not recorded save a cell phone recording. Why? As Ms. Goodman notes, no explanation was offered:

We don’t know why the press wasn’t allowed to be in the room. Perhaps they didn’t want reporters to see the six buttons Mandy Manning was prominently wearing on her dress. Her buttons included artwork from the 2017 Women’s March, a rainbow flag and the slogan “Trans Equality Now!” One displayed the quote “Life begins at the end of your comfort zone.”

While Mr. Trump assures gun owners that the Second Amendment is sacrosanct, he effectively conveys to teachers and students across our country that the First Amendment may be imperiled. Thanks to Democracy Now and Truthdig, media outlets that will likely be removed from social media, some Americans will be able to read what the Teacher of the Year stands for.

 

Categories: Uncategorized Tags:

“You’re the One Creating the ‘Bad’ Schools”: Nation’s Top Teachers Denounce Devos During Closed-Door Session

May 2, 2018 Comments off

Huzzah to the Teachers of the Year for speaking truth to power. Too bad more of them didn’t do the same thing when Arne Duncan was advocating Race to the Top and Margaret Spellings was promoting NCLB. Here’s hoping their voices will resonate in districts across the country!

Source: “You’re the One Creating the ‘Bad’ Schools”: Nation’s Top Teachers Denounce Devos During Closed-Door Session

Categories: Uncategorized Tags: ,

My Modest Proposal: Test for Student Understanding Instead of School and Teacher Accountability

May 2, 2018 Comments off

Yesterday Diane Ravitch posted a “Modest Proposal” on testing and asked for feedback. She began by posing this question: “Why do our policymakers at the federal, state, and local levels continue to require and enforce annual testing, despite the non-existent benefits?” Her proposal to counter this testing mania was this:

Why not give the tests in the first week of school and use only a test whose results may be returned within a month? Let machines score the standardized questions, and let teachers score the constructed responses. The testing vendor would know that they would be chosen only if they could report the results in a month, in a format that informs teachers what students do and do not know. That way, the teacher can find out where students are as they begin the year and tailor instruction to address the needs of the students.

That way, tests would no longer be high-stakes. They would be expressly designed for diagnostic purposes, to help teachers help students. The results would come too early to misuse the tests to stigmatize students, punish teachers, and close schools. There would be no punishments attached to the tests, but plenty of valuable information to help teachers.

My reaction to her question about why policymakers at the federal, state, and local levels continue to require and enforce annual testing and my own “modest proposal” follow:

Why do legislators and those who elect them want to use standardized tests to measure schools? Because they are relatively cheap, relatively easy to administer, and provide seemingly precise data that can be used to sort and select students and schools in a fashion that is easy to understand. And so schools are using tests designed for accountability of adults instead of tests designed to measure student’s understanding.

It strikes me that teachers could crowd-source formative assessments using social media, formative assessments that would enable them to accomplish what Duane Swacker suggests: “… teacher-made classroom testing and assessments designed to help the students learn where they are in their own learning.” Such tests would be untethered from “grade levels”. These crowd sourced formative assessments would not only promote self-actualization on the part of students but also provide classroom teachers with valuable feedback on how the approaches they are using are work for the specific children in their classroom. Assuming someone with technological expertise would be willing to provide the expertise needed to design this kind of “testing network” without making an unseemly profit, these crowd-sourced tests would be very inexpensive to design and relatively easy to administer. Indeed, these formative assessments might replace the “unit tests” teachers use to measure student performance. The only downside of these assessments— or any formative assessments— is that they could not be used to rate schools.

I believe we have the technological ability to design specially tailored FORMATIVE assessments that would enable students to progress at their own rate in subjects where there are clear hierarchical skills to be mastered. Instead of using SUMMATIVE assessments to hold SCHOOLS and TEACHERS accountable for students achieving specific outcomes based on the artificial construct we call “grade levels” we should use FORMATIVE assessments to “…help students learn where they are in their own learning.” We should let time be the variable and learning be constant instead of the other way around.

John Tierney’s Atlantic Article Misses One KEY Point: APs Are Being Taken to Game a Bogus Rating System

March 18, 2018 Comments off

Wayne Ridenour, a current Facebook friend who taught my daughter’s AP History Course over two decades ago, posted an article from the Atlantic by John Tierney titled “AP Classes Are a Scam” and left the comment “Sorry but this is all too true”. Both Mr. Tierney’s article and Mr. Ridenour’s comments are valid, but Mr. Tierney’s article overlooks a key factor that is driving the expansion of AP courses and Both Mr. Tierney’s article and Mr. Ridenour’s comment overlook one key positive factor about AP courses.

John Tierney’s analysis of why AP courses are a scam hits all the flaws of the test:

  • AP courses are NOT equivalent to college courses
  • Because fewer and fewer colleges recognize AP courses for credit, the monetary savings that once existed are no longer possible
  • High schools are no longer screening admissions into AP courses (more on that below)
  • Minority students are under-represented in AP enrollments despite the expanded pool of this taking the courses
  • Small, economically challenged schools divert resources to AP courses which has the effect of limiting non-AP courses
  • And worst of all, AP courses are prescribed, robbing the best and brightest teachers of the opportunity to offer their own creative courses that might challenge and engage the best and brightest students in a school.

But Mr. Tierney fails to mention one factor that has driven increases in AP enrollments: the many rating systems that use some form of AP enrollments as a proxy for “quality”. It all began when Washington Post education writer Jay Matthews began ranking schools in his region using the percentage of students enrolled in AP courses as primary factor. While he acknowledged the limitations of such a ranking system, his use of them had a national impact. The result: an explosion of AP course offerings, an expansion of the pool of students who enrolled in AP courses, and the consequent forcing out of “honors” courses with teacher-driven courses of study with AP courses whose course of study was determined by ETS.

But Mr. Matthews use of AP enrollments as a metric DID recognize one practical reality: absent some kind of national standard there is no ready means of determining if a student who received a high grade in an “honors class” at a small or underfunded school met the same standards as a student who earned high grades in an affluent school district. The high school my daughter attended in the early 1990s did not send many students to competitive colleges and so the caliber of its courses was an unknown. I believe that both her SAT scores and her AP scores helped validate the balance of her transcript and provided evidence that she might succeed in the classrooms of those schools, two of which she was accepted to. This reality— that competitive colleges use APs as a validation for transcripts— is why Jay Matthews included AP as a proxy for “quality”. Whether the expansion of AP enrollments that followed is a virtuous circle or a vicious one is open to question. Having led five different school districts, I observe that the more affluent a district is the less it is concerned with proxies: if a district has a well established “brand” in the admissions offices of elite colleges it has no need for AP course and the teachers at those schools eschew AP courses… and that, in my judgement, is a virtuous circle.