Harvard Educational Review
  1. Using Data/Getting Results

    A Practical Guide for School Improvement in Mathematics and Science

    By Nancy Love

    Norwood, MA: Christopher-Gordon, 2002. 550 pp., with CD-ROM, $69.95.

    With the recent authorization of the No Child Left Behind Act, district- and school-level data have taken on new importance. Educators will need to disaggregate achievement data, track progress of all students over time, and examine assessment results to identify problems and plan appropriate instructional interventions. Noticeably absent from the calls to explore, examine, and use data to inform instruction is a roadmap laying out how to engage in this process at the district, school, and classroom levels. Nancy Love’s aptly named Using Data/Getting Results: A Practical Guide for School Improvement in Mathematics and Science ably fills this void with a straightforward guide and handbook for practitioners new to the data process and those seeking to create schoolwide efforts to use data to inform instructional improvement and support educational reform.

    Love opens the book by laying out the “Guiding Principles for Our Work,” focusing on collaborative inquiry and the use of data to “fuel reform.” Two key points are made in her discussion of the inquiry cycle. First, that inquiry and reform must be aimed at all times at improving instruction, and second, that equity be placed “center stage” in the inquiry and improvement process. While the focus of Love’s work is on examining data, she cautions early on that “nothing changes if classrooms don’t change; school reform must impact teaching and learning” (p. 7). Love’s data-inquiry process is not about data for academics or for accountability’s sake; instead, it is geared toward improving instruction. Love sees data inquiry as a collaborative and iterative cycle (see “Collaborative Inquiry into Student Learning”). A cycle calls for planners, including administrators and teachers, to commit to student learning, collect and analyze data, formulate learner-centered problems, set measurable learning goals, develop a learner-centered action plan, take action, monitor results, and start again.

    Love’s focus on equity is heartening for teachers and school leaders attempting to disaggregate data by student groups, such as low-income children and limited English speakers. Love’s vision goes farther than these traditional data analysis groups. She writes, “By equity, we mean the right of every student to achieve at high levels” (p. 11). Love believes that this principle should drive data analysis and leads to questions such as, “To what extent do performance gaps in mathematics and science exist among racial, class, cultural, or gender groups in our school?” and “How prevalent are beliefs that block equity in our school? How do they manifest in school and classroom practices?” These questions reflect an interest in analyzing data and in improving instruction for all students.

    The heart of the data process comes in chapters two through six. Chapter two explores the practical issues of using data to inform school reform and instructional improvement. It describes types of data that school staffs might explore, including assessment results, examples of students’ work, teacher and student surveys, master schedules, demographic breakdowns of student enrollment in particular courses and extracurricular activities, interviews, and classroom observations. In this chapter, Love also introduces “The Phases of Inquiry: Framing the question, collecting data, analyzing data, organizing data-driven dialogue, drawing conclusions and taking action, and monitoring results” (p. 32). Ideally, these phases are also iterative, leading back to new questions and starting the cycle anew. Recognizing that there are technological, structural, and capacity issues embedded in the data process, Love provides guidelines for designing databases to house data, offers suggestions on how to create common planning time necessary for teachers to look at data collaboratively, and includes advice on how to help teachers and school staffs engage in what can be hard and possibly threatening conversations around student performance and instruction. Chapter two, like the others that follow, ends with a lengthy catalog of additional resources that readers might find helpful. This is not simply a reference list; rather, Love provides brief summaries of each additional source, enabling readers to identify easily the readings that would be more (or less) helpful to pursue.

    Chapter three explores the intricacies of data analysis, exploring the types of data that may be available (e.g., standardized test results, state and performance assessment results, and examples of students’ work), and describing ways in which the data can be analyzed and displayed. She does not limit her discussion to test scores; she also presents samples of rubrics with student work. In addition to examining how to summarize and analyze student achievement data, the chapter delves into how one might assess students’ “opportunities to learn” (p. 95). Here, Love introduces new types of data to include in the analysis: master schedules, teacher qualifications, tracking, quality of curriculum, and access to technology, among others. This chapter also provides numerous tools for identifying questions to answer with data. Finally, Love reminds readers that data analysis is only the beginning; once challenges are identified and causes are hypothesized, staffs must develop action plans and ways to assess the effectiveness of these plans.

    In chapter four, Love discusses the curriculum and assessment decisions that need to be addressed if data-driven instructional reform is to take place. To this end, the chapter follows the journey of a fictitious science reform team engaged in reforming a school district’s curriculum, assessment system, and instruction. The chapter discusses methods to assess whether the new curriculum and instructional reforms are taking root. Through teacher and student interviews and surveys, classroom artifacts (such as lesson plans), and classroom observations, school staffs have an idea of the extent to which reforms are being implemented at the classroom level. It is notable that this chapter explores creating and analyzing student achievement data (the learning result of the new curriculum and assessment system) last. This step must come after the curriculum is developed, teachers have received support in implementing the curriculum, and students and teachers are active participants in its implementation. Love warns readers that they must ask how well — and why — instructional changes are being implemented before assessing their impact.

    Chapter five will no doubt provide great assistance to school staffs seeking to comply with the most basic reporting requirements of No Child Left Behind. In this chapter, Love describes ways to disaggregate data by race, class, gender, and English-language proficiency. It goes further than the act, however, urging staffs to investigate potential structural hurdles to equity at the school level, such as tracking practices and placement in advanced and special education classes. The chapter also presents staffs with ways to examine their own practices for actions that might indicate prejudice and discrimination. It is not enough, Love says, to identify winners and losers in a tracking system.

    In chapter six, Love provides proof that this data-inquiry process can work in real schools and real districts. She presents “reform in action” accounts of five districts where staffs are using data to reform instruction, increase minority student enrollment in high-level math classes, implement standards-based reform, curriculum development, and to create and maintain a “performance-based learning and assessment system.” By describing these districts — their successes as well as the challenges they faced — Love provides an encouraging glimpse into the role that teachers collaborating around data can have in transforming schools.

    Lengthy appendices constitute the remainder of the book, and it is these appendices that school data teams might find most useful. Love includes samples of surveys, self assessments, tools for examining state and district assessment results, interview protocols for interviews with students, protocols for rating text books, and a host of other tools. By presenting so many different tools and approaches to the inquiry of data, Love makes clear that there is no correct way to explore data and to improve instruction, nor is there any “correct” data to analyze. At the end of chapters three through five, Love provides “planning tools.” These tools include samples of team planning question guides that schools can use as they begin to plan, and data collection plan forms to help guide data inquiry (the tools also are included in modifiable format on a CD-ROM that accompanies the book). In compiling so many user-friendly tools for teachers, Love surely creates “a practical guide for school improvement.” However, the notion that this is a guide only for improvement in math and science is not quite accurate; while many of the examples in the book are geared toward math and science, they also could be adapted to meet the data needs of other disciplines. In short, Love’s guide and handbook provide the missing roadmap to using data to inform instruction and reform practices.

    N.S.S.
  2. Share

    Abstracts

    Multiple Pathways to Early Academic Achievement
    NICHD Early Child Care Research Network
    The Educational Science and Scientifically Based Instruction We Need
    Lessons from Reading Research and Policymaking
    Michael Pressley, Nell Duke, and Erica Boling

    Book Notes

    Challenges of Conflicting School Reforms
    By Mark Berends, JoAn Chun, Gina Schuyler, Sue Stockly, and R. J. Briggs

    A Student’s Guide to Methodology
    By Peter Clough and Cathy Nutbrown

    In the Deep Heart’s Core
    By Michael Johnston

    Using Data/Getting Results
    By Nancy Love

    Pregnant Bodies, Fertile Minds
    By Wendy Luttrell

    Lessons to Learn
    By Molly Ness

    Case Study Research
    By Robert K. Yin