Voices in Education

Lessons from the Data Wise Project
The following article originally appeared in The Harvard Education Letter (volume 29, number 3). Copyright 2013 President and Fellows of Harvard College. All rights reserved.

Policy initiatives around the Common Core State Standards and teacher evaluation present schools with opportunity and risk. The opportunity is to make more effective use of evidence to inform the education of all of our children; the risk is to think that evidence alone will do the job.

For over a decade, the Data Wise Project at the Harvard Graduate School of Education has trained school-based teams in an eight-step process for using data to improve learning and teaching, described in our book, Data Wise. Over the years, we have adapted our courses to accommodate what we have learned from the field. For example, we no longer have to spend time persuading people that using data is a good idea; indeed, data has become an established component of educational policies worldwide. There is now an expectation that the data should be used to track student progress, set goals, identify action steps for improving instruction, and monitor progress—tasks at the heart of the Data Wise Improvement Process that seemed revolutionary when we first started advocating for them 10 years ago.

One thing that hasn’t changed much, however, is that we still find we need to make the case that educators should engage in the process collaboratively, treating Data Wise not as a “program” to “implement” but, rather, as a means of organizing and bringing coherence to staff members’ collective efforts at improvement. And, perhaps most importantly, we have learned to warn educators that simply going through the eight steps of the process will not guarantee success. Schools that use data to make real improvements in learning and teaching bring a distinctive approach to their work, and we have come to call this disciplined way of thinking the ACE Habits of Mind, where A stands for a shared commitment to Action, Assessment, and Adjustment; C stands for Intentional Collaboration; and E stands for a relentless focus on Evidence.

We have integrated these habits into our own teaching and daily work and into the revised edition of our book because we have learned that, for most of us, they do not come naturally. And most of us have, by default, some “bad” habits that can work against our desire to use data effectively.

A: Shared Commitment to Action, Assessment, and Adjustment
The first of the ACE habits boils down to this: All of the work you do is geared toward doing, not just talking. Every meeting has clear objectives and a plan for making sure those objectives are realized. For example, if you are a member of a teacher team meeting to dig into student data (Step 4), the deliverable you are working toward is a statement of the learner-centered problem you and your colleagues are committed to addressing. When you meet to examine instruction (Step 5), your objective is to identify a problem of practice that you all share. By the time you are ready to develop an action plan (Step 6), you know you need to identify specific instructional strategies that will make your teaching more effective. And, almost from the moment you begin implementation, you start assessing the extent to which that action is leading you in the direction you want to go and adjusting your actions when they fall short of the mark.

We contrast this habit with bad habits that are easy to slip into. One involves falling into a state of analysis paralysis, in which meeting after meeting is devoted to collecting more information, yet nothing ever gets done. Another is forging blindly ahead. In this case, the inclination is to stay the course with a plan, usually out of fear that midcourse corrections would be a sign of failure, or to abandon it entirely, which often means that the good ideas are thrown out with the bad. Cultivating a shared commitment to action, assessment, and adjustment helps teams follow the middle path toward real improvement.

At schools that cultivate a shared commitment to action, assessment, and adjustment, there is a sense of perpetual, purposeful forward motion. Educators talk about the confidence they get from taking lots of intentional small steps together—instead of taking (or avoiding) big leaps alone in the dark. Most importantly, they recognize that midcourse corrections are not signs of weakness but signs of strength.

C: Intentional Collaboration
This second habit involves being extremely deliberate in how and when you engage groups with the work of improvement. This involves making conscious decisions about whom to bring to the table for a particular conversation and structuring that conversation so that the collective wisdom of the group is brought to bear. The default bad habit here is to assume that just because you have put people together in the same room productive interaction will take place. In reality, what often happens is that people skirt the most important conversations in an effort to keep the interaction pleasant. Or people fall back on preestablished patterns of participation that leave some voices out.

Intentional collaboration is all about building trust. Creating and regularly reviewing group norms helps build trust, as does bringing many people together to make sense of assessment data, rather than having that knowledge reside in a few “data experts” at a school. Discussion protocols can be powerful tools for supporting this habit, especially if they are designed to make it clear when to contribute ideas and when to listen. A powerful way to build trust is to engage educators in understanding what’s happening with learning and teaching, why it’s happening, and what tools and training they need to do their jobs better.

For example, as a teacher you may appreciate that to improve instruction you and your colleagues need to look at the instruction that is happening in your classrooms. But what if this sounds really scary? If your team is cultivating the habit of mind of intentional collaboration, you will make deliberate choices about how to work together as you explore this unfamiliar ground. Maybe you begin by talking about your hopes and fears of opening your classroom doors. Then you get clear on the purpose of observation and set clear expectations for the collective work you will do before, during, and after an evaluation to make sure that the experience feels safe. You might test the waters by allowing teachers to videotape their own lesson and then choose what part of the video to show, along with specific things they want their colleagues to be looking for as they watch. If you make clear choices about how you will do this work together, you may find yourselves embracing a practice that at first seemed daunting.

E: Relentless Focus on Evidence
This habit often draws a laugh when we first mention it, since it can sound a bit obsessive. But we offer this habit in all seriousness, since time and time again school leaders have told us that the single most important thing that builds trust as staff members work through the Data Wise process is sticking to evidence. The goal is to create a culture where people make decisions based on specific, objective, and descriptive statements about a wide range of data sources. But reading all kinds of things into the data is a deeply ingrained habit for most people, so it requires a determined effort to break it.

Have you ever noticed that instead of making a factual statement about what we see, most of us tend to make an inference about why we see what we do? The statement “The teacher doesn’t know how to let students do their own thinking” leads to a very different conversation from “Four out of five times the teacher asked a question, she answered it herself.” In the first case, there is an inference and a judgment that can put someone (quite naturally!) on the defensive and close off the possibility of exploring alternative reasons for the observed data. In the second, there is a simple statement of fact, something that is much harder to argue with—and much easier to remedy. When interpretations are offered as facts (and when those interpretations vary widely), it is extremely difficult to get consensus about what the problem is, let alone how to solve it.

A simple way to help a colleague hone the habit of sticking to evidence is to ask, “What evidence do you see that makes you say that?” All of us make inferences based on something, but sometimes our brains are so quick to leap to conclusions that we need help naming and describing that something. A metaphor we use to put language to this challenge is “the ladder of inference,” an idea developed by Peter Senge and others. Picture a ladder with description at the bottom and inference and action at the top. If you start at the bottom with description and climb your way up to inference and action, you will be steady on the ladder and your action will be supported by the lower rungs of the ladder. If you leap to the top, it’s hard for anyone else to make the journey with you, and your conclusions are usually precarious. We’ve found that the metaphor of being “high on the ladder” and “coming down the ladder” can make the process of developing the habit of evidence more playful and easier to talk about and do.

More Than Data
We have made the ACE Habits of Mind central to our own teaching and practice because we believe they are essential to creating a culture of continuous improvement. When used daily and in combination, these habits draw out the diverse perspectives on a team, build the trust that allows educators to open their practice, make effective use of limited precious time, and provide a pathway to turn piles and piles of data into a discrete and actionable plan. It is true that it may take many small steps to build the ACE Habits of Mind into the work of improvement. But don’t underestimate where those small steps can lead. In our experience, this work is transformative—for both students and adults. It is about so much more than data: It is about finding a way to allow every student and every educator to fulfill their potential.


About the Author: Kathryn Parker Boudett and Elizabeth A. City are co-editors (with Richard J. Murnane) of Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning Revised and Expanded Edition (Harvard Education Press, 2013), from which this article is adapted.