Home > Uncategorized > Campbell’s Law CAN Be Repealed… but Only By Introducing Multiple and Soft Measures

Campbell’s Law CAN Be Repealed… but Only By Introducing Multiple and Soft Measures

October 28, 2018

Decades ago, in 1985 to be precise, I gave a presentation to teachers at the beginning of the school year that featured a slide that read “What Gets Measured Gets Done”. At the time, NH was about to launch some form of standardized test that was intended to be the end all for accountability and, at the time, I was advocating that our district devise multiple measures for accountability in order to avoid being held accountable based on a single, flawed measure. At the same time, my Assistant Superintendent and I conferred with the consultants from the standardized test company to see if there was some way we could use their norm-referenced test as a criterion referenced test (short answer: it was possible but only through convoluted calculations) and we persuaded the school boards to look at the test results through a criterion-referenced lens as opposed to the norm-referenced lens that we felt mis-represented the effectiveness of our schools since what was tested did not match what we were teaching. I don’t think we were familiar with Campbell’s Law at that time, but without being aware of it we were determined not to fall prey to it. What is Campbell’s Law? Fred Hess offers this definition in a recent Medium post:

Formulated in 1976 by social psychologist Donald Campbell, it reads, “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” Put simply: “When a measure becomes a target, it ceases to be a good measure.”

Without knowing of Campbell’s Law at the time, my Assistant and I both understood the aphorism “What Gets Measured Gets Done” and we both believed that the national norm referenced standardized tests being used in New Hampshire did NOT measure what we were teaching and, therefore, should NOT be used as a valid accountability measure. Fortunately, the district we worked in was led by school board members who understood this and, therefore, did not pay that much attention to the test results that were reported in the local media. As I witnessed throughout my career, one reason the school boards did not pay that much attention to the test results was that our schools invariably scored very high on the tests: parents were generally well educated, engaged with the schools, and pushed their children to succeed.

Throughout my career as a Superintendent I tried to get the school boards to develop multiple measures for the schools, because if the only measure of success is a nationally normed test score the only thing students will be exposed to in class is a curriculum that is focussed on what is tested… and standardized tests cannot measure what is really important: the development of a joy for continuous learning thought one’s life.

I still believe “What Gets Measured Gets Done” and still believe that norm-referenced standardized tests have a place in measuring a school’s effectiveness… but given the high correlation between parent income and education and test scores it seems foolish to equate high test scores with quality. And given the very tenuous correlation between high test scores and college success and/or earnings, it seems even more preposterous to use those scores as the sole metric for measuring quality. But we love to rank and compare in our country, and norm-referenced tests give us an easy way to do so…. and so the beat goes on….

Advertisements
Categories: Uncategorized Tags: ,
%d bloggers like this: