Archive

Archive for the ‘White Papers’ Category

NCLB Waivers: A Chance to Measure What’s Important

November 14, 2011 1 comment

This White Paper was submitted to New Hampshire Commissioner of Education, Dr. Virginia Barry, on November 14, 2011

The management aphorism “What Gets Measured Gets Done” aptly describes the effects of No Child Left Behind. For the past decade, NCLB required State Departments of Education to use standardized proficiency tests as the basis for rating schools. In the minds of the politicians and the public, these tests were the most effective way to hold schools accountable for student learning. The rationale: standardized tests are easy to administer, relatively cheap, and provide an objective, numeric basis for rating schools. The seemingly precise data these tests generate also provided the media with the means to publish tables showing how schools compared with each other and how their performance varied from year-to-year.

Once school boards, administrators and teachers realized that the pass rates on these standardized tests and the data reported in the newspapers would be the basis for defining the success or failure of their schools, the focus of schooling narrowed. Elementary and middle schools concentrated on reading and mathematics, the subjects measured by the tests, in some cases leading to the elimination or reduction of other subjects like art, music and physical education. Elective programs in middle and high schools came under closer scrutiny as schools devoted more time to subjects assessed by standardized tests. Finally, because the federal law provided no incentives for the attainment of high test scores, schools paid less attention to those students whose performance met or exceeded the minimum standards, focusing instead on students who fell short of the standards, particularly those closest to the standard. What got measured got done.

The Obama administration’s decision to allow states to seek waivers to the current No Child Left Behind guidelines provides states with an opportunity to change the way they hold schools accountable for student learning.

To seize this opportunity, states will need to meet a set of requirements set forth by Secretary of Education Arne Duncan; they must: adopt college-and-career ready standards and develop tests to assure students meet those standards; adopt a means of identifying the lowest performing schools in the state and develop a way to support them; develop and implement rigorous teacher and Principal evaluation systems that use student performance as a factor; and, while instituting these changes, reduce “duplicative and burdensome paperwork”. Each of these requirements involves measurement and, as we have witnessed over the past decade, what gets measured has consequences. In seeking waivers, then, it is crucial that states get the metrics right. This White Paper proposes a set of metrics that are defensible, relatively inexpensive, and potentially transformative.

College and Career Ready Standards

There is no need for any state to spend any money developing an assessment to measure “college readiness”. That assessment already exists. Upon acceptance to public universities and community colleges, most students are required to take placement tests to determine if they require remedial courses. Instead of taking these placement tests when they enroll in a post-secondary school, students who aspire to higher education could take them in October of their sophomore year. If the student passes the tests, they would be eligible for dual-enrollment in college courses during their junior and senior years in high school. If the student does not pass the placement test, they would have two years to schedule courses to help meet the “college readiness” standard defined by the tests. Ideally, the results from the placement tests would help high schools identify the competencies each student needs to attain to avoid being assigned to remedial courses when they enroll in a post-secondary school. Teachers would then be able to use this information to design courses and/or class assignments that would help each student master those competencies. The school’s “college readiness” metric would be the percentage of students who enroll in post secondary schools without requiring remedial coursework.

The development of a “career ready” metric requires the establishment of alliances between schools and employers, alliances that would help schools understand the specific competencies a student needs to enter the work force. New Hampshire, where competencies are the basis for the attainment of credits, could be a national leader in developing “career readiness” standards. Here’s how: Sophomores who do not aspire to college but instead seek work in specific career areas would confer with high school ELO Coordinators. The ELOS Coordinators who would identify nearby employers in those career areas and work with them to arrange provide work-study opportunities for high school students in their region. The ELO Coordinators would then work with the employers to define a set of competencies that constitute their “entry standards” and develop Extended Learning Opportunities that would meet those standards. School personnel would then develop courses or learning modules that, when combined with the “entry standard” ELO, would assure the student fulfills the competencies needed to meet the school’s graduation requirements. Ideally, the career bound student could find a work-study assignment that would help them enter the work force upon graduation. The “career ready” metric would be the percentage of students who meet the “entry standard” ELO agreed upon at the outset of the student’s work experience.

Ultimately, schools would be expected to have 100% of their graduates meeting either the college entry standard or the workforce entry standards set by employers. The adoption of these two metrics would require changes in high schools. Students would need to determine their interest in pursuing post-secondary education at the beginning of their sophomore year in order to provide opportunities for remediation as undergraduates. Junior and Senior courses for students aspiring to post-secondary education would have college level content for those students who pass the placement examination or targeted remediation for those student who did not pass. Schools would expand the use of ELOs in the junior and senior year for career bound students, ELOs that would guarantee that graduates have both the competencies needed for graduation and the entry-level skills sought by local and regional employers. The net result for career bound students would be an emphasis on the attainment of competencies and a corresponding de-emphasis on passing courses. Freed from teaching subject matter courses to career bound students, teachers currently assigned to those classes could work as ELO Coordinators.

Identifying and Improving Low Performing Schools

For decades results on standardized tests of all kinds have been consistent: student scores on standardized tests correlate with their parents’ level of education and household income. When NCLB required the use of standardized proficiency tests to identify “low performing schools”, it was no surprise that the great majority of those schools served students coming from low income homes. Children raised in poverty often begin Kindergarten with limited vocabularies, short attention spans, and poor social skills. This lack of school readiness results in an achievement gap that is evident from the student’s first day in school, a gap that widens and persists as students progress through the grade levels. To level the playing field in the assessment of student performance, teachers and administrators in schools with high numbers of children in poverty advocated the use of a “growth model” to measure school performance. They believed that this method of assessment, which holds schools accountable for each student’s growth over the course of a school year, would be a fairer measure of school performance. After being used in several states for as many as ten years, however, it the growth model ultimately classifies schools the same way as the traditional standardized test model[1]. For example, in Colorado, whose accountability model is currently offered as an exemplar for New Hampshire, eighteen of the nineteen schools identified as requiring a “dramatic turnaround” have free and reduced lunch counts of 60% or more with the other school in the group having over 45% of students in that category[2]. The conclusion: even if New Hampshire adopts a growth model it is highly probable that the lowest performing schools will be those that house a high percentage of free and reduced lunch students.

When the results of these “new, improved” state assessments are made public and high poverty schools are identified as “failing”, advocates of No Child Left Behind and critics of public education will contend that schools are “again using poverty as an excuse for low-test scores”. They will suggest that teachers and principals in schools serving low-income students need to work harder, focus their instruction more, and set higher expectations for their students. But the reality is that schools cannot address the inherent performance gap that results from poverty. By the time most students in New Hampshire take their first “new improved” high stakes assessment in third grade, they will have spent 95% of their life outside of public school. Given that reality, it is difficult to expect schools to increase student achievement all by themselves. There is, however, compelling evidence that when schools serving economically disadvantaged students assume a broader role in the lives of children they can get impressive results, especially when they intervene early and coordinate their efforts with other agencies. Since the households of children born in poverty often receive welfare, Medicaid, and housing support, their parents deal with a host of social service agencies. Unfortunately, the agencies providing these services do not work collaboratively with schools. Early education programs like Head Start and public pre-Kindergarten that provide economically disadvantaged students with instruction in fundamental skills are not universally available in New Hampshire and, like the other social service agencies, may not have close ties to schools.

To help all public schools demonstrate they are taking action to counteract the effects of poverty and to reinforce the message that schools cannot address those problems in isolation, the State Department of Education should require schools to collaborate with social services agencies that serve their students and require evidence that schools are striving to engage parents in their child’s education. The State Department could, for example, require schools to establish cooperative agreements with health and social services agencies and articulation agreements with pre-schools and child care centers located within their attendance zone. They could also require schools to administer surveys designed to measure the number and nature of parent outreach activities each school sponsors. Poverty is not an acceptable excuse for poor school or student performance. If economically disadvantaged students are falling short of the mark, schools need to demonstrate they are doing everything possible to overcome the obstacles to their learning.

Improving Teacher and Principal Effectiveness

States seeking NCLB waivers will need to develop guidelines to “improve teacher and principal effectiveness”, guidelines that require “frequent” evaluations and include “student growth” measures as a major factor. These waiver requirements effectively reinforce two popular myths about teacher evaluation:

  • Districts have the capacity to conduct frequent evaluations of all teachers
  • There are standardized tests that measure student growth and the results of those tests can be used to measure the effectiveness of individual teachers.

While neither of these assumptions is true, it is difficult to argue that schools cannot improve the way they evaluate principals and teachers, but the public needs to realize that the evaluation improvements required for NCLB waivers will require additional spending. Increasing the frequency of evaluations using the traditional methodology requires additional administrative staff. Designing and implementing a “value-added assessment” that measures student growth requires additional funds as well, especially if the assessments are required in all content areas at all grade levels. Given the need to fulfill these new requirements at minimal cost to the State and school districts in order to qualify for a NCLB waiver, I propose the following metrics to assess staff performance.

More frequent evaluations: Districts could increase the frequency of classroom observations at a relatively low cost by re-employing retired teachers and administrators on a contract basis. In the private sector, businesses often employ selected retirees on a part-time or full time basis as contractors. When this practice is done in combination with early retirement incentives, businesses have an opportunity to reconfigure their staffing patterns and to recruit young talent. Retirees could provide districts with a cadre of evaluators and teaching coaches making it possible to increase the frequency of observations without adding staff. Alternatively, districts could use video technology to increase the frequency of classroom observations, inviting teachers to share videos of exemplary lessons with their immediate supervisor, the Principal, or with peer evaluators. This would effectively serve the same purpose as an announced observation but would allow the “observation” to occur at a time convenient for the evaluators outside the school day. Ideally, as part of the statewide evaluation model (see next section), evaluators could use electronic tools like those recommended as part of the Marzano Causal Teacher Evaluation Model.

Measuring student growth: It is fortunate that New Hampshire s abandoning the NECAPs since virtually all researchers agree that proficiency tests like NECAPs cannot be used to measure an individual teacher’s effect on a child’s performance over time. Furthermore, research does not support the use of existing value added assessment models as the primary basis for evaluating teachers. Instead, researchers and education policy makers advocate the use of a wide array of measures to determine the effect a teacher is having on students.

Therefore, instead of using scarce resources to develop a value added measurement as the primary means of assessing teacher and administrator performance, New Hampshire should consider adopting a statewide multi-dimensional evaluation model. Such an evaluation model might include:

  • A formative computer adaptive assessment like NWEA as the measure of a teacher’s effect on an individual student’s academic achievement[3]
  • Parent and/or student questionnaires
  • Parent and/or student interviews
  • Student portfolios or other exhibitions, and
  • Peer evaluations.

When these sources of information are combined with the frequent observations cited in the previous section, administrators would have a more comprehensive picture of a teacher’s performance. An important caveat: Whatever evaluation model the state develops, it should be done in collaboration with the professional organizations in the State and should strive to recognize that good teaching is much more than achieving higher test scores.

With the implementation of the changes outlined in this extended essay in place, the measurement of Principal effectiveness would be straightforward: Did the school forge community alliances? Did the Principal collect and analyze the information provided by the multi-dimensional evaluation system? Based on surveys conducted by the Superintendent did the Principal have an effective working relationship with parents, students, and teachers? By using data beyond standardized test scores the Superintendent would have a broader perspective on the Principal’s performance.

Reducing Duplicative and Burdensome Paperwork

The metrics recommended in this essay could be implemented with a minimum amount of additional paperwork with one exception: the “entry standard” ELOs envisioned in the “career ready” metric. Paperwork associated with classroom observations is ultimately a local determination, based on contracts in each district. If computer adapted assessments are used instead of a new standardized assessment, paperwork associated with the administration of school-wide testing would also be reduced. The surveys envisioned throughout the essay can be completed on-line and made available to evaluators in that format, eliminating additional paperwork. The additional paperwork associated with the “entry level” ELO initiative could be mitigated if a grant could be obtained to provide funding for the staff and/or additional staff time required to develop a database of competency based employer ELOs. The paperwork could also be minimized id regional collaboratives like SERESC and NCES coordinate the district efforts to obtain “entry level” ELOs.

One critical caveat: the USDOE cannot expect State Departments to reduce paperwork if they require them to design and implement evaluation systems designed to sort schools and teachers into categories based on their “performance”. Recent reports on the impact of Race To The Top indicate that the teacher evaluation plans increased paperwork for administrators and effectively diminished the time they spent on informal visits to classrooms.[4] This is not surprising. Fair labor practices would require substantial documentation to dismiss or reduce the compensation of an employee who is issued a contract. Reducing paperwork, then, will especially challenging if the NCLB waivers require the State to link evaluation with compensation in any way shape or form.

Summary: The Right Metrics 

The metrics recommended in this essay would change the way schools operate at the secondary level, change the scope of school responsibilities at the elementary level, and broaden the scope of information administrators gather on classroom teachers. With the exception of the additional costs incurred to improve the current evaluation systems, the other changes outlined could be accomplished by redirecting resources. The coordination of ELOs could be assigned to teachers freed from classroom assignments. If NWEAs or another computer adaptive assessment replace a “new, improved” state assessment as a means of measuring student performance the funds required for the “new, improved” assessment for could be re-directed. Indeed, if NWEAs were adopted at the state level, districts currently funding them in their operating budgets would save money. Much of the new data needed to assess teacher and Principal effectiveness can be gathered through the use of inexpensive online surveys and by redirecting the time spent by Principals and Superintendents.

What gets measured gets done. If we want to continue narrowing the content of what is taught in public schools, narrowing the focus of public schooling, and using standardized tests to measure school performance, we should stay the course: spend money on a “new and improved” state assessment and spend time teaching to that assessment. If we want something different from schools, we should consider new metrics.

 


[1] Betebenner, Linn, Growth in Student Achievement: Issues of Measurement, Longitudinal Data Analysis, and Accountability

[2] School Turnarounds in Colorado, monograph by Julie Kowal and Joel Abeldinger for Donnell-Kay Foundation

3 The Kingsbury Center, NWEA’s research arm, in a 2010 memo to the Charleston County School District, cited limitations to NWEA’s use as a value added measure, expressing particular concern about that school district’s plan to base 60% of their evaluation on NWEA results. These limitations would need to be considered in determining the role NWEA results would play.

[4] Michael Winerip, “In Tennessee, Following the Rules for Evaluations Off a Cliff”, New York Times, November 6, 2011.

Reformatting New England Schools

March 30, 2011 Leave a comment

This was widely circulated in Vermont in Spring 2011 in response to HB 153, the school consolidation bill. 

The best way to change the existing reality is to create a new reality

that makes the old one obsolete … Buckminster Fuller

 Executive Summary

This extended essay recommends replacing the current governance, administrative and organizational structure of small rural, graded schools based on the Factory Model with “Network Education Centers” (NECs) based on a Network Model. These NECs would provide individualized instructional support for students. The students would use distance-learning resources to pursue most of their studies independently and attend periodic seminars to reinforce skills taught online and to develop crucial communication skills. Teachers would staff the NECs, where they would provide individualized instructional support to students and also help coordinate wrap-around services. Those wraparound services could include: morning and afternoon childcare; social services; medical and dental services; mental health services; and other services children need or a community wishes to provide. These NECs would be administered and governed by Regional Collaborative Centers (RCCs). The RCCs would supersede SUs and their governing boards would supersede local school boards. The RCCs would provide administrative services and technical support to the NECs. The establishment of RCCs would accomplish Vermont’s goal of consolidating school districts without necessitating the closure of schools in small communities. This essay has five sections:

  • The Factory School Model describes the current organizational and governance framework for schooling and the deficiencies with that structure.
  • The Network School Model describes a new framework for schooling that takes full advantage of the technology available today. It provides an overview of the Network Education Center (NEC), which is the primary organizational unit of the Network School model.
  • The Administration of Network Schools elaborates on how the Network Education Centers (NECs) and Regional Collaborative Centers (RCCs) would be managed.
  • The Governance of Network Schools outlines how the Network Education Centers (NECs) and Regional Collaborative Centers (RCCs) would be governed.
  • The Urgency for Change describes the political and economic factors that make the need to abandon the factory model urgent.

The Factory School Model

The governance and organizational structure of public schools has not changed significantly since 1900. Elected or appointed school boards develop policy for their town or region, adopt budgets for the public to consider, and appoint superintendents who oversee the day-to-day operations of schools. During the era when our country changed from an agrarian to industrial economy the continuation of local school boards and the hierarchical organizational structure of school districts made sense. The continuation of local school boards made sense because public schools focused on the need to prepare students in their community for either higher education or for jobs in the local and regional workplace. The organizational structure of schools, modeled after the factories and mills that served as the backbone of local economies in the 20th century, provided the most cost-effective means of sorting students for group instruction at the elementary level and for sorting students at the secondary level to determine those who would qualify for higher education and those who would enter the workforce.

In the global information economy, however, the governance and hierarchical organizational structure of public schools no longer makes sense. The mission of public education today is to ensure that all high school graduates have the skills, motivation, and self-discipline needed to become lifelong learners. To fulfill this mission, public schools can no longer cling to the factory model of schools where students are sorted into batches based on age cohorts and expected to master skills in a prescribed time frame. Today, every child must be given the time and customized instruction required for them to learn-how-to-learn, to become flexible, creative, and adaptable thinkers.

Students from economically disadvantaged homes and homes where education is not valued pose an especially difficult challenge for public schools in their efforts to provide all students with the skills needed to survive in today’s economy. When these students attend schools that lack resources, the challenge is even greater. The social services in place today to support disadvantaged students are uncoordinated, often duplicative, and, therefore, inefficient. In many instances, this lack of coordination among service agencies adds another level of complication to teachers. To achieve the ambitious public expectations for schools, teachers will need find a way to coordinate their efforts with the service providers who support students.

The Network Schools Model

The optimal way to teach someone is by tutoring. Tutors can fully engage the student by matching their instruction to the student’s ability level, unique interests, and unique way of learning. Before Skype, YouTube, and the vast array of free online video instruction that emerged in the past five years, the notion of providing a tutor for each student was inconceivable. With this abundance of free instructional resources available online, there is an opportunity to abandon the existing model for schooling and replace it with one that provides instruction in a time frame that corresponds to each student’s readiness to learn and to his or her way of learning. The Network School model assumes that each student will develop the skills needed to become a self-actualized learner by progressing through a carefully conceived sequence of learning experiences. The grid on the last page contrasts the Network School and the Factory School in a number of dimensions. The broad goals of the Network School are provided below:

  • Pre-K through Grade 8: Provide individualized skill instruction (i.e. reading; writing; fundamental mathematics; factual content in science and social studies; etc.); provide small group instruction on oral communication, social skills, team building, and, where deemed most efficient, in content skill instruction; provide coordinated wraparound services needed for students and families. The ultimate goal of Pre-K to Grade 8 instruction is to ensure students have the skills needed to design independent studies and participate in seminar courses at the secondary level.
  • Secondary: Provide a means for students to complete self-designed independent study courses using a combination of online resources and seminar sessions in lieu of traditional direct classroom instruction. Also, during this phase of their schooling students could be released from studies to complete community service projects to help them see how they will be able to contribute to their communities’ well being.
  • Overarching Goal: The hybrid instruction offered by network schools will focus on the skills needed for each student to learn-how-to-learn, to be an independent thinker, to be a compassionate community member, and to determine the path he or she intends to follow upon graduation.

In the network school model, a Network Education Center or NEC replaces the school building. The mission of the NEC is to provide coordinated support services for each child in the community. The NEC could serve as the yearlong and before-and-after school child-care facility for the community. It could provide office space for social workers, medical providers, and other non-profit agencies that serve youth. The NEC could also serve as the technology resource center for those students or community members who do not as yet have ready access to the internet, or it could serve as the town library. Each community would be free to define the scope of services provided in the NEC. By determining the services at the local level, it will be easier to coordinate those services and eliminate duplicate or overlapping services.

Administration of Network Schools

Individual Network Education Centers (NECs) would require minimal oversight to function. The NEC administrator would be responsible for coordinating the services provided in the facility, including the oversight of instruction provided to students. In small rural NECs, the administrator might also serve as a teacher or case manager. The NECs would be managed by a Regional Collaborative Center that would provide them with technological and administrative support.

The RCC’s technological support could take the form of managing and administering the technology infrastructure required to provide online learning opportunities to all students. That technology infrastructure would include the management of data warehouses and the development of reports drawn from instructional software.

The RCC’s administrative support could include:

  • Budgeting (e.g., budget development, purchasing, payroll, issuing reports, auditing)
  • Personnel management (e.g., hiring staff for NECs, negotiating and administering RCC-wide contracts)
  • Compliance monitoring for state and federal guidelines (e.g., completing required state reports, ensuring all compliance with all standards), and
  • Logistics (building maintenance; transportation; bulk purchases, supply storage, etc.)

Executive Directors who would perform the tasks currently handled by Superintendents would lead the RCCs. They would report to an RCC Board (see Governance section, below) and serve as liaison to the State Departments that provide education, health and social services to students.

Governance of Network Schools

The governance of public schools in Vermont and New Hampshire is archaic, inefficient, and ineffective. Two factors complicate efforts to reform the current structure. The first, as discussed in earlier sections of this paper, is the notion that schools operate like factories with “graduates” being the product and standardized tests administered to age-based cohorts the basis for quality control. The second complicating factor is the view that the town is the primary unit of governance. This town-centric perspective when combined with the factory school model contributes to inefficiencies such as the duplication of administrative services, low pupil teacher ratios and the resulting high per pupil costs. The town-centric perspective also results in small secondary schools that cannot offer a wide array of courses to their students, disparities in the tax bases between districts, complicated transfers of payment among districts, and wildly disparate learning opportunities for students.

The implementation of the network school model would necessitate a change from the current governance structures, moving away from the town-centric perspective toward a regional perspective. As noted in the previous section, Network Education Centers (NECs), which take the place of “schools,” are managed by Regional Collaborative Centers (RCCs). Instead of having multiple elected boards governing independent school districts, elected board members drawn from the catchment area would govern the RCCs. Instead of being governed by elected boards, the NECs would develop advisory councils to help determine the scope of services to be provided in each town.

The RCC catchment areas could be defined voluntarily. They might defined by existing vocational center boundaries, by existing regional collaborative boundaries, by existing social agency boundaries, or by county boundaries. Large single town or city districts who are not involved in formal tuition agreements with neighboring towns would replace their elected schools boards with elected RCC boards, thereby facilitating coordination of services among various agencies serving children.

Urgency for Change

The network school model is evolving in adult learning and among home school parents who have withdrawn their children from public schools. In job-site training, webinars are replacing seminars and remote links are replacing large group meetings. Many graduate schools offer asynchronous online courses in place of the traditional courses offered only on campus in the past. Home school parents are using inexpensive online lessons that engage their children. They are also accessing free instructional materials in every content area at all levels. Through social media home school parents can arrange ad hoc opportunities for their children to connect with their peers to work and play together.

While this evolution toward network schooling is taking place outside public education, we who work in public schools spend countless hours debating over how many desks to have in each room, which textbooks to buy, and how much to pay our employees. Our students have cell phones with apps and we have blackboards and worksheets. Given the pervasiveness of online learning, the gutting of programs in public schools, and the limited revenues on the horizon, it is easy to envision a future where homeschoolers abound and the cost per student gets even higher.

Electronic learning is here now. We need to begin talking about how we can make it a reality in our schools.

Factory School

Networked School

Purpose of Schooling Finding an appropriate vocation: “…develop in each individual the knowledge, interests, ideals, habits, and powers whereby he will find his place and use that place to shape both himself and society toward ever nobler ends” Becoming a self-actualized learner, independent thinker, and compassionate member of society

Student Grouping

Age cohorts By mastery cohorts, if at all
School Organization By grade levels By mastery cohorts

Rate of instruction

Mean rate of cohort Individualized: Based on Student Learning Profile (SLP) developed by parents, instructor(s), and, if applicable, case manager
Method of instruction Approach that meets the needs of majority of cohort Individualized: Based on SLP developed as above
Delivery of instruction Large Group based on common text(s) Individualized: web-based, tutorial, and small group

Motivational Theory

School-wide system of rewards and punishment Individualized: Based on SLP developed as above

Definition of Intelligence

Velocity: The rate of learning Capacity: The amount a student learns

Basis for Student Assessment

Norm Referenced: Comparative ranking based on rate of learning as measured by tests 

Academic: All student performance measures based solely on academic

Criterion Referenced: Mastery of learning objectives defined in SLP developed as aboveHolistic: Student SLPs would incorporate some means of measuring “self-actualization, independent thinking, and compassion”

Basis for School Assessment

Norm Referenced:Comparative ranking based on percentage of students scoring above minimum cut score Criterion Referenced:Description of services, standardized report on parent, student satisfaction

Daily Schedule

Six hours for all Customized, flexible

School calendar

Agrarian for all Customized, flexible

Provision of Social Services to Students

Fragmented – Each agency gathers student information independently, treats student in isolation Integrated – Teacher/case manager serves as single point of contact, teams with other providers to share information, provide integrated services

Implicit Values

Compliance, conformity, competition, consumerism Independence, initiative, interdependence, introspection

Race to the Top: NO!

October 31, 2009 Leave a comment

This White Paper was sent to NH Commissioner of Education Virginia Barry in October, 2009. She subsequently circulated it to her staff and the superintendents in New Hampshire in advance of our meeting later that month. The section on merit pay was re-worked into an article that was published in Education Week in March 2010 and the section on alternative compensation methods was reworked and used as part of an ongoing study in SAU 70 where I served as Superintendent.  

As noted in the attached letter, there are several reasons I believe it is a bad idea to link student assessments to teacher performance. At the same time, I believe it is necessary to reform the current compensation system so that performance is rewarded over experience and the accumulation of coursework. This extended memo has two parts: a section elaborating on my reasons for opposing a link between student assessments and teacher pay; and a section offering an alternative to the existing unitary pay schedule, an alternative that could incorporate several elements of “merit pay”.

The Case Against Linking Teacher Pay to Student Assessments

There is no research that unequivocally supports the use of any examinations to measure “value added” by an individual teacher or school

To qualify for Race to the Top funds, States have been asked to eliminate any legislation that precludes the use of “data regarding student achievement from being tied to teachers for the purposes of evaluation”. The section of the Federal Register describing the rationale for this requirement states:

Research indicates that teacher quality is a critical contributor to student learning and that there is dramatic variation in teacher quality.\2\ Yet it is difficult to predict teacher quality based on the qualifications that teachers bring to the job. Indeed, measures such as certification, master’s degrees, and years of teaching experience have limited predictive power on this point.\3\ Therefore, one of the most effective ways to accurately assess teacher quality is to measure the growth in achievement of a teacher’s students;\4\ \5\ and by aggregating the performance of students across teachers within a school, to assess principal quality…. (emphasis added)

The assertion that “…one of the most effective ways to accurately assess teacher quality is to measure the growth in achievement of a teacher’s students…” is not substantiated in research findings. Indeed, The Board on Testing and Assessment, an arm of the National Research Council, issued a letter to Secretary Duncan on October 5, 2009, expressing serious reservations about the use of value added tests as the basis for evaluating teachers. After listing a series of technical and practical concerns regarding value added testing, their 19-page letter includes the following caution regarding value added assessments:

The use of test data for teacher and educator evaluation requires the same types of cautions that are stressed when test data are used to evaluate students: “Tests are one objective and efficient way to measure what people know and can do, and they can help make comparisons across large groups of people. However, test scores are not perfect measures: they should be considered with other sources of information when making important decisions about individuals” (Lessons Learned, p. 15). This caution is even more important when applied to complex statistics—like value-added analyses—derived from tests.

Put another way, test data should be a formative element in evaluation, not a summative one. Using test data in even a formative fashion, however, is complicated by the factors described in the next three sections.

New Hampshire schools would be required to substantially alter their grouping practices and/or test protocols in order to use to use “value added” assessments in a fashion that conforms with research models

Almost all the value added research has taken place in urban schools or county school districts where there are large grade-level cohorts, a common curriculum, common instructional practices, and comparable demographics. In these studies, researchers carefully controlled the grouping of students, the way tests were administered, and the nature of the tests. In order to replicate these research conditions in New Hampshire, elementary schools would need to make certain that teachers are assigned to comparable cohorts of students over a three year period (i.e. the same grade level, the same blend of regular and special education students, and same ability level IF ability level is the practice), that the student cohorts remain constant, that student assessments used to measure teacher performance are designed specifically for that purpose (see next bullet), and that the assessments are administered in a pre-test/post test fashion instead of once annually. At the secondary level, where students typically have 4-7 teachers per day, it is difficult to imagine how any value-added measure could be used without dramatically expanding the tests administered at each grade level.

The performance of a large number of teachers cannot be measured using existing assessments

Teachers at all grade levels who do not teach specific content that is not systematically assessed at the State level (i.e. Art, Music, PE, Guidance, Special Ed, Technology Education, etc) and secondary teachers whose content is not systematically assessed at the state level (currently all departments except English Mathematics and Science) would be exempt from any of this testing. Moreover, given that our current State assessments are given at only one grade level in high schools, it is hard to envision how longitudinal information on student performance will be gathered using assessments. In summary, since state level tests are not in place for what is arguably a majority of teachers, any system linking student performance with teacher compensation is inherently inequitable.  

New Hampshire’s NECAP assessments are not designed to generate “value added” measures

As written and designed, NECAP tests cannot yield individual student data with the kind of detail needed to measure improvement in individual students over time. This is particularly true in high-performing districts where the “headroom” is insufficient, making it impossible to measure “gains” of any kind. If New Hampshire wanted to pursue any kind of value added measure it would require the State to develop new assessments, field test those assessments, and implement them for multiple years before receiving the results needed to make any meaningful decisions on teacher performance. All of this assumes it is possible to design such an assessment for New Hampshire with its small rural schools and diverse demographics among schools and assumes assessments can be designed and implemented for secondary teachers and K-12 teachers in specialized subjects.

Performance Pay is a Flawed Idea

Adlai Stevenson one quipped: “Americans are suckers for good news. Given a choice between disagreeable fact and agreeable fantasy they will choose the fantasy every time”. For decades the American public has chosen to believe in an agreeable fantasy that merit pay for teachers will cure the ills of our “failing public schools”, particularly urban and high poverty schools. This agreeable fantasy ignores three disagreeable facts: we already have merit pay in public education; there is no link between the performance of public schools and the revenues that fund public schools; and teachers don’t want merit pay.

FACT ONE: We Already Have Merit Pay

Our current method of school funding, which is based primarily on State and local taxes, creates a de facto merit pay system, a system that works against the urgent goal of providing quality instruction in districts with the highest poverty levels. Teachers working in affluent school districts earn significantly more than their counterparts in other districts, and they have far superior working conditions. As a result, those wealthy districts attract and retain the best teachers while other districts struggle to fill positions and often lose their most promising teachers to wealthier districts within commuting range. In addition to offering more pay, the affluent districts offer superior professional growth opportunities, fully staffed and equipped media centers; a wide range student services; and manageable class sizes and course loads. Most importantly, in affluent districts parents understand and appreciate the value of education, and the community supports the schools by consistently passing budgets.

FACT 2: Performance is not linked to revenue in public education

Because public schools rely on State and local taxes, there is no connection between school performance and school district revenue. In the private sector, if a company experiences success its profits increase and management can use those additional profits to reward those employees whose hard work caused the bottom line to grow. In school districts, where pay increases depend on tax revenues, there is no connection between “profits” and performance. Tax revenues fluctuate due to variables beyond the control of school districts. When a district’s test scores soar during a year when the tax base declines due to erosion in the local property tax base, reduction in State aid, or a nationwide recession, it is impossible to reward the improved performance of an individual teacher let alone a school. If “merit pay” funds were included in my operating budget for the coming year, for example, those funds would be pitted against layoffs, increased class size, the elimination of “non-essential” programs, deferring maintenance, or higher local property taxes.

FACT 3: Teachers do not want merit pay

Teachers are not drawn to their profession because they want to make a million dollars; they are drawn to public education because they want to make a difference. Given the choice, teachers will accept decent pay and good working conditions over extraordinary pay and a stressful workplace.  Teachers want to work where they are respected and valued in the community, and where they can earn enough to live comfortably in the community where they work. Even in high paying affluent districts, that modest goal is often unattainable.

Calls for merit pay at the national and State level deflect the spotlight from the systemic problems with funding for schools and imply that a redistribution of existing funds to reward “the best” teachers will solve the problems of public education. But here’s the most disagreeable truth: a sizeable and sustained infusion of money is needed to offset the existing pay and workplace disparities that make a mockery of the ideal of equal opportunity in public schools and make teaching an unattractive career choice. The hard-working teachers in low paying districts need decent wages. The forlorn schools in those districts need to be upgraded. Students in all schools should have the staffing levels, rich curriculum offerings, and access to technology that are “givens” in affluent districts.  And, if we want the best and the brightest to consider teaching, all districts need to provide basic compensation packages that will lure them into the profession.

In conclusion, if the pursuit of ARRA funds is contingent on the State developing and implementing some means of linking student performance to teacher and principal evaluations, we should give serious consideration to taking a bye. Given that there is no research to support the linkage of student assessments to teacher performance, given the extraordinary technical challenges associated with developing value added assessments that would be available to all teachers, and given the underfunding of most public schools in New Hampshire, spending scarce dollars, scarce time, and increasingly scarce political capital on developing and implementing a “Merit pay” plan is unwise. This is especially true if the funds seek are to be used to initiate changes that will ultimately increase baseline budgets in school districts and the State Department of Education.

The Case For a Rational Reward System for Teachers

While research does NOT support the use of assessments as a basis for setting teacher compensation, research DOES support the Race to the Top’s contention that “…measures such as certification, master’s degrees, and years of teaching experience have limited predictive power on (teacher quality).

The existing system

For the past several years, reformers have advocated changes to the unitary salary schedule (“step-and-track”) that has been in place in public education for at least the past five decades. A system advocated by teacher organizations as a means to assure pay equity among teachers, the unitary salary schedule effectively rewards teachers for successive years on the job (step advancement) and for completion of relevant course work (track advancement). Typically, the step advancement stops after 15 years and the track advancements conclude with either a doctoral degree or the equivalent number of credits.

The existing reward pattern

Implicit in this pay method is the belief that teachers improve in their performance in a linear fashion during each of their first 15 years on the job after which their experience does not warrant any pay differentiation. The system also implies that the completion of blocks of relevant coursework results in improved performance that, in turn, warrants higher compensation. Those who work in education know— and education researchers can substantiate— there is no absolute link between years of teaching experience and teaching ability nor is there any substantive difference in teaching performance that results from course work beyond the masters degree. Our current system, then, effectively undervalues performance and overvalues experience and course work.

A more rational reward system

Research and practice indicate that teachers fall into four broad categories: those beginning their career (i.e. probationary teachers); those who warrant tenure but are not fully developed in the profession (i.e. “Continuing Contract Teachers”); a group whose experience, professionalism and teaching skill warrants a higher pay grade than an emerging professional (i.e. “Career Teachers”); and a group who are universally recognized by their peers, administrators, parents, and students as exceptional (i.e. “Master Teachers”). These broad categories, or pay grades, are analogous to the system in place in many colleges and universities and in some respects similar to the compensation system used by the federal government.

Pay grade compensation differentials

Under a system like this, advancement from one pay grade to another would result in a substantial pay increase, but all teachers within a given pay grade would have the same earnings based on the assumption that all teachers within a pay grade have comparable skills. Thus, a “Career Teacher” with ten years experience would receive the same salary as a teacher in that category with twenty years experience. The four pay grades would be adjusted for cost-of-living based on an index that would be negotiated periodically in the same fashion pay scales are now negotiated.

Progression through pay grades

Progression through pay grades would be based on a combination of cumulative evaluations and the attainment of a Masters Degree or its equivalent. The evaluation process would be holistic, analogous to the process currently used to determine if a beginning teacher warrants a continuing contract. The focus of evaluation would change from one of identifying teachers who fail to meet a minimum standard to one of identifying teachers who have attained the competencies needed to advance from one pay grade to another.

Consequences of this proposal

Changing from the current method of compensation to this new model will result in many changes in the oversight of instruction at the State and local level, changes that would focus more on teacher performance and less on “clock hours” and the accumulation of credits. Would this be difficult? Absolutely. But unlike the “value added” measures effectively mandated by Race to the Top it is uncomplicated, has an analogue in academia and other professions, and would require a more rational reallocation of funds going forward as opposed to an infusion of additional funds.

How this plan might address the ARRA requirement

“Student assessments” could be used in the holistic evaluation process that is the basis for determining progression from one pay grade to another. They would be a specific element in the evaluation process but NOT an exclusive element. The kinds of “student assessments” used to make this determination would vary from district to district since some districts already have developed databases for tracking longitudinal performance over time.  Districts that do not have such databases in place (which would include mine) would have an incentive to put one in place.

Bottom Line: From my perspective, if the creation of an alternative compensation schedule along the lines outlined above meets the spirit of the federal mandate, we should seek ARRA funds. If the ARRA funds are contingent on the development of a wholly new assessment design that requires districts to assign a heavy weight to student assessments, we should let other states compete and continue working on the transformation initiatives we already have in place. We are making progress on these initiatives without federal funds and will continue to do so in the face of the fiscal challenges in front of us.

Categories: White Papers Tags: , ,