Harvard Educational Review
  1. Data and Teaching

    Moving Beyond Magical Thinking to Effective Practice

    Joseph P. McDonald, Nora M. Isacoff, and Dana Karin

    New York: Teachers College Press, 2018. 168 pp. $29.95 (paper), $75.00 (cloth).

    Since the advent of American public schooling, teachers’ work has been vulnerable to innovations in the education field. With the more recent proliferation of data collection, accessibility, and policies commanding its use, this vulnerability has only intensified. While some schools are well positioned to take on this expansive innovation, the seemingly relentless gathering and propagation of data can be simply exhausting, even divisive, for others. Data can become yet another buzzword for an ill-defined innovation done to teachers rather than with them. As teachers used to say in the big, struggling school where I most recently taught, “Data for data’s sake.”

    Given the impact data increasingly poses for teachers’ work, the field needs more scholarship that showcases how schools are making sense of this far-reaching innovation and, in particular, more cases of schools that are effectively integrating it into practices. With Data and Teaching: Moving Beyond Magical Thinking to Effective Practice, Joseph P. McDonald, Nora M. Isacoff, and Dana Karin do just that by bringing readers directly into classrooms and team meetings in a sample of high-poverty New York City schools engaging with data. While the nine schools in the sample were selected because of their reputations for actively using data to improve outcomes, the authors find considerable variation among the schools. Broadly, they illustrate two types of schools. They examine cohesive professional communities that have, despite the challenges it presents, rallied around the promise of data use in teaching and cooperatively fostered “action spaces” for data uptake, “arena[s] for collective improvement...marked by a commitment to mobilize capacity for the work from multiple sources” (p. 111). They also look at schools where data’s grip may be visibly and/or rhetorically evident but where administrators’ and teachers’ dependence on it amounts to “magical thinking,” a simplistic view that data’s mere availability, especially data gleaned from standardized tests, will dramatically raise student achievement. These schools’ view of data as “cut and dried” (p. 3) impedes authentic inquiry, and when they already have what one principal called “sucky trust issues” (p. 67), implementation becomes still more arduous, sometimes downright factious. In contrast to action space schools, the authors characterize these schools as “vortexes” to capture the relative disarray that results “when a challenging innovation like data use in teaching is dropped in a cocksure way into an ill-prepared school or district” (p. 77).

    Throughout Data and Teaching, McDonald and colleagues describe data use in schools as a “disruptive” innovation, emphasizing both their belief in its promise and their wariness of its unintended consequences. Echoing the notion of disruption in the tech industry, the designation stresses the potential data hold for shifting entrenched teaching behaviors that have long bedeviled the profession and also warns of the danger an innovation as ubiquitous and broad as data poses for the delicate social worlds of schools. This nuanced characterization of data is refreshing, considering some enthusiasts’ naïve, can-do embrace of the innovation and pretense that data “is simpler, that it is installable without customization, that it is magic” (p. 127). “Implicitly,” the authors argue, these enthusiasts “deny the complexities of schooling itself, and in the process they threaten the impact of what they champion” (p. 127).

    In response to the magical thinking that frequently accompanies cutting-edge proposals for change in schools, Data and Teaching argues that neither data nor teaching can occur without the other and consistently yield desired results. Accordingly, the three opening chapters carefully parse data and teaching and put them into meaningful dialogue. Framing these chapter titles as questions seems additionally deliberate. Though their answers may appear self-explanatory, the authors demonstrate that there is far more to each question than many data champions typically venture. Chapter 1 (“What Is Data?”) helpfully unpacks the hackneyed term, distinguishing between big-test data and intimate data, and surfaces schools’ frequent and detrimental practice of “bas- ing decisions about what to teach in a given year on individual items from the previous year’s test” (p. 5). Too often, they maintain, schools equate the word data with high-stakes test results. McDonald and colleagues helpfully remind readers that nearly everything students produce in schools, from formative assessments to state exams, remains an important piece of the data puzzle. Chapter 2 (“What Is Teaching?”) illuminates teaching’s many complexities, which are too often minimized in plugs for increased data use but which are necessary to understand to shift instruction in any substantive way. Chapter 3 (“What Is Data Use in Teaching?”) traces the surge of data use in American education policy in general and in New York City in particular. The chapter begins in a “mid-20th-century Congressional hearing room” (p. 50) and concludes in an individual teacher’s classroom a half-century later, with the teacher wrestling with the innovation, and highlights “the role of intuition in teaching as a partner to data use” (p. 47). Such careful toggling between intimate data and big-test data is an approach that the school portraits in later chapters encourage.

    All of these introductory chapters are valuable, but it is the second that is most striking, perhaps because teaching’s realities remain conspicuously absent in many discussions of data use in schools. Citing household names in this literature—David K. Cohen, Magdalene Lampert, Deborah Loewenberg Ball, Sharon Feiman-Nemser, and first author Joseph P. McDonald—the chapter calls attention to the myriad and competing dilemmas teachers confront in their work. In foregrounding such a candid, unheroic picture of teaching, McDonald and colleagues suggest that no exploration of data and its application can be understood apart from this “bed of churning dynamics” (p. 13). Against this backdrop, they outline key instructional behaviors and snapshots of teachers employing each in context: pressing, pulling, asking, and walking away. While this review of teaching dynamics and behaviors may at times feel secondary to the discussion of data use in schools, the authors emphasize them because these practices position students’ ever-shifting thinking at the very center of teachers’ considerations. At its core, this is the stuff of data use in teaching—teachers’ purposeful interrogation of the intimate data unfolding in the moment and over time in classrooms. Without careful consideration of such data, big-test data becomes largely obsolete. How teachers navigate intimate data and put them into conversation with big-test data is the difference between the magical and effective schools featured in the book.

    The rest of the chapters offer vivid “deep dives” into four New York City schools from the sample. Readers observe the work of schools’ data teams. Some schools demonstrated keen toggling between the data types, action spaces characterized by “common goal[s] of ownership” (p. 77). As a teacher in one such school reasoned, “I feel that we’re all in the same conversation...It’s teachers and administrators that are included in this conversation...We’re all realizing that this has to happen, and...we’re all on the same page of what do we do now” (p. 76). The authors underscore this “we-ness” as a critical feature separating schools that use data effectively from vortex schools. The data use meetings in the vortex schools, they show, often devolved into quibbling too “remote from...children” (p. 96) to manifest the ambitious changes their administrators urged. With teachers lacking ownership in the innovation and perceiving its push an imposition to their autonomy, these faculties appeared fragmented, bereft of the “we-ness” so critical for actualizing the disruptive innovation they were expected to implement.

    The school portraits also take readers into teachers’ classrooms. Those in action space schools demonstrated an authentic interest in probing students’ thinking and amassing more comprehensive pictures of students through discerning and consistent data use. These classrooms, and the in-house professional development that helped sustain them, the authors argue, undertook the essential practice of reframing. These teachers did not simply plug in some innovation from afar but, rather, rethought it “to suit the school context that it enters” and readjusted it—“in a mix of fidelity and invention unique to each school—to actual capacities, needs, and conditions” (p. 107). To be sure, this requires that schools have fine-tuned infrastructures that, unlike the typical cellular organization of schools, work to unify school actors to engage in purposeful, professional learning together. The classroom visits in vortex schools deliver a different story. Isacoff describes one as “blanketed with student work and performance data charts, but in a fashion that turned information into meaningless shapes and colors” (p. 97). This scene conveying an “illusion of precision” (p. 94) becomes emblematic for a ravenous but largely superficial embrace of data use in teaching.

    Given the volume of data the authors collected and discuss in the book’s appendix, Data and Teaching could have been much longer. Though its brevity makes for an accessible read that will extend readership beyond the academy, it could leave some readers wanting more. Also, the reasons why the authors chose to focus on just four of the nine schools they examined could have been clearer. And though these four portraits are called “deep dives,” at times they could have delved still deeper. For example, “focal teachers” at each site frequently became mouthpieces for school faculties. And while we see practitioners in action and hear reflections from administrators and data coaches, the authors’ emphasis on “we-ness” perhaps demands a more comprehensive look at the individual school contexts than they offer here. That said, the researchers’ narrowing of data use to teaching literacy specifically is helpful. Empirical explorations documenting teachers’ implementation of broad innovations are seldom so specific; when researchers cast such wide nets, exploring a given innovation across different subjects all at once, the impact of that innovation becomes more diffuse and harder to gauge.

    Overall, Data and Teaching: Moving Beyond Magical Thinking to Effective Practice is an accessible, illuminating read that will prove valuable to school leaders, entrepreneurs, teachers, and researchers. The school portraits featured are captivating and complex, and the “New Direction” sections sandwiching each of these portraits offer pithy, actionable avenues for administrators and practitioners looking to shift the way they do things in schools. Topics such as navigating an overwhelming marketplace of educational resources (in the authors’ words, a “material world”), taking steps to distribute leadership among school faculties, and cultivating professional learning communities of teachers within schools are just a few of the promising new directions raised.

    While authors McDonald, Isacoff, and Karin believe deeply in the promise of data, what they offer readers widens the scope of “data” considerably and espouses a discernment that is anything but magical. The result is a sobering glimpse into what it takes to actualize data use in schools and a call for more cohesive school faculties to ensure meaningful implementation of a disruptive, pervasive innovation.

    Jeremy T. Murphy
  2. Share


    “Los Músicos”
    Mexican Corridos, the Aural Border, and the Evocative Musical Renderings of Transnational Youth

    Book Notes

    Land-Grant Universities for the Future
    Stephen M. Gavazzi and E. Gordon Gee

    Classroom Cultures
    Michelle G. Knight-Manuel and Joanne E. Marciano

    Ghosts in the Schoolyard
    Eve L. Ewing

    Data and Teaching
    Joseph P. McDonald, Nora M. Isacoff, and Dana Karin