Report offers framework for evaluating D.C. school reform efforts, along with first impressions

WASHINGTON — A new report from the National Research Council offers a framework for evaluating the effects of a 2007 reform law on the District of Columbia’s public schools. The evaluation program must include systematic yearly public reporting of key data as well as in-depth studies of high-priority issues, such as teacher recruitment and retention, the report says. As part of the evaluation program, the Mayor’s Office should produce an annual report to the city on the status of the public schools, including an analysis of trends and all the underlying data.

The evaluation program should be structured to provide ongoing, independent monitoring and feedback on a number of key aspects of D.C.’s public education system, which are described by the committee that wrote the report. “Although the immediate goal for the District is to evaluate the changes enacted in 2007, we also see an opportunity to build a continuing program of analyses that will be useful regardless of future political or personnel changes,” said Christopher Edley, committee co-chair and dean of the School of Law at the University of California, Berkeley.

In 2007 the Public Education Reform Amendment Act (PERAA) shifted control of D.C. Public Schools (DCPS) from an elected school board to the mayor and created the position of chancellor, among other changes. The act also mandated an independent five-year evaluation to determine whether enough progress has been achieved to warrant continuation of the new structure. The D.C. City Council asked the Research Council to provide guidance on how to structure the evaluation and to offer initial feedback about implementation of PERAA, to the extent possible.

Report Offers First Impressions, With Cautions

The report offers some first impressions of school reform under PERAA and notes that the city and DCPS have made a good-faith effort to implement the required changes, but the committee cautions that it is premature to draw general conclusions about the reforms’ effectiveness at this time.

Data suggest that a modest improvement in student test scores has continued since PERAA was enacted. However, the report stresses that scores on their own cannot identify the reasons for that improvement. Further examination is necessary to determine whether they can be linked to the reforms or are the result of other factors, such as demographic changes in D.C.’s highly mobile student population. Also, because the rising scores are averages, further analysis and monitoring are needed to determine how various subgroups of students — such as particular grade levels or students in particular areas — are faring; some could be making gains while others are not.

The report emphasizes that student test scores should not be the only factor used to judge improvement in the school system. A comprehensive evaluation should address the full range of district responsibilities: quality of teachers, principals, and other personnel; quality of classroom teaching and learning; capacity to serve vulnerable children and youth; promotion of family and community engagement; and quality and equity of operations, management, and facilities.

Although the committee found that the administrative and governance changes required by PERAA have generally been carried out, the report notes that the Office of the Ombudsman was defunded three years after its creation. Since the office was intended to be the primary channel for parents to communicate with school officials and seek redress for complaints, its absence is significant, said the committee.

Evaluation Program Should Be Long-Term, Independent

The report offers a framework for evaluating key elements of reform under PERAA that affect student outcomes — including management structures and roles and conditions for learning — as well as the student outcomes themselves.

A set of specific, measurable indicators should be established as soon as possible, building on suitable indicators already collected in the District, the report says. These would track how well the city’s public schools are doing, how well the key strategies the city has chosen to improve education are working, the conditions for student learning, and the capacity of the system to produce valued outcomes.

In addition, in-depth studies should be designed to provide deeper analysis of specific questions about high-priority issues, including both the targets of PERAA and long-standing challenges in District schools, such as special education. These studies can answer questions about the causes of trends seen in indicator data, explore potential reasons for disappointing outcomes, provide information to help alter existing strategies, or help explain why some strategies appear to be working better in some schools than in others. They are key, the report notes, because they are the means by which evaluators can answer policymakers’ questions about the effects of particular policies and practices.

To develop and sustain an infrastructure for ongoing research and evaluation of its public schools, D.C. will need to engage local universities, philanthropic organizations, and other institutions, the report says. The program must be independent of school and city leaders and responsive to the needs of all stakeholders, and its research should meet the highest standards for technical quality. There is no one model for the city to follow, but examples of research and evaluation programs that have been created in other cities — such as Chicago, New York City, and Baltimore — could be useful as D.C. develops its own program, the report notes.

“An independent evaluation program that can be an ongoing source of objective information and analysis will be an invaluable resource for the city,” said committee co-chair Robert Hauser, executive director of the National Research Council’s Division of Behavioral and Social Sciences and Education. “There are no quick answers. Education reform itself is a long-term process, and the evaluation of its outcomes also has to be seen in the long term.”

The committee will discuss the report at a public briefing to be held at 2 p.m. on Sunday, March 6, at the National Academies’ Keck Center, 500 Fifth St., N.W., Washington, D.C. A live audio webcast of the briefing will be available at http://national-academies.org.

The study was sponsored by the government of the District of Columbia, the National Science Foundation, National Research Council, Spencer Foundation, Diane and Norman Bernstein Foundation, World Bank Community Outreach Program, Philip Graham Fund, City Bridge Foundation, and the Kimsey Foundation. The National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council make up the National Academies. They are private, nonprofit institutions that provide science, technology, and health policy advice under a congressional charter. The Research Council is the principal operating agency of the National Academy of Sciences and the National Academy of Engineering. For more information, visit http://national-academies.org. A committee roster follows.


Sara Frueh, Media Relations Officer

Luwam Yeibio, Media Relations Assistant

Office of News and Public Information

202-334-2138; e-mail [email protected]

Pre-publication copies of A Plan for Evaluating the District of Columbia’s Public Schools: From Impressions to Evidence are available from the National Academies Press; tel. 202-334-3313 or 1-800-624-6242 or on the Internet at http://www.nap.edu. Reporters may obtain a copy from the Office of News and Public Information (contacts listed above).


Division of Behavioral and Social Sciences and Education

Center for Education

Committee on Independent Evaluation of D.C. Public Schools

Christopher Edley Jr. (co-chair)

Professor of Law and Dean

School of Law

University of California


Robert M. Hauser 1 (co-chair)

Executive Director

Divison of Behavioral and Social Sciences and Education

National Research Council, and

Vilas Research Professor of Sociology

Center for Demography of Health and Aging

University of Wisconsin


Beatrice F. Birman

Managing Research Scientist

Education, Human Development, and Workforce Program

American Institutes for Research

Washington, D.C.

Carl A. Cohn

Research Practitioner

School of Educational Studies

Claremont Graduate University

Claremont, Calif.

Leslie T. Fenwick


School of Education

Howard University

Washington, D.C.

Michael J. Feuer


Graduate School of Education and Human Development

George Washington University

Washington, D.C.

Jon Fullerton

Executive Director

Project for Policy Innovation in Education

Harvard University

Cambridge, Mass.

Fernando A. Guerra 2

Director of Health

San Antonio Metropolitan Health District

San Antonio

Jonathan Gueverra

Chief Executive Officer

Community College of D.C.

Washington, D.C.

Jonathan Guryan

Associate Professor of Human Development and Social Policy, and

Faculty Fellow

Institute for Policy Research

Northwestern University

Evanston, Ill.

Lorraine M. McDonnell

Professor and Chair

Department of Political Science

University of California

Santa Barbara

C. Kent McGuire

President and CEO

Southern Education Foundation


Maxine F. Singer 1, 2

President Emeritus

Carnegie Institution of Washington

Washington, D.C.

William F. Tate IV

Chair and Professor of Education

Department of Arts and Sciences

Washington University

St. Louis


Patricia Morison

Study Director

1 Member, National Academy of Sciences

2 Member, Institute of Medicine

The material in this press release comes from the originating research organization. Content may be edited for style and length. Want more? Sign up for our daily email.