Like most states, California has long required prospective teachers, whether they attended education schools or entered the profession through alternate routes, to pass standardized tests in basic skills and subject knowledge in order to earn their licenses.

However, there is little evidence that performance on these tests is associated with future performance in the classroom. Teacher educators have therefore begun to look for ways to assess the quality of a candidate’s work in the classroom, the skills he or she has mastered, and the effects on student performance.

For the past five years, the teacher-education faculty at the University of California, Santa Barbara (UCSB), has been piloting a new tool that they believe allows them to better assess both the qualifications of their graduates and the strengths of their own academic program. The Performance Assessment for California Teachers (PACT), developed by a consortium of universities in that state, requires candidates to prepare a portfolio that includes lesson plans, reflective essays, videos, and examples of student work, all drawn from the candidates’ internship or student-teaching experience.

“For the first time, we had concrete evidence of what our candidates were doing [in the classroom],” says Tine Sloan, acting director of the teacher-education program at UCSB’s Gevirtz School of Education.

Beginning this summer, institutions across the state will be able to get similar information about their students. Under a state law—the first of its kind in the nation—as of July 1, 2008, candidates in all California teacher-education programs will be required to pass a performance assessment in order to earn a state teaching license. Thirty-one institutions, which collectively educate about 30 percent of new teachers in California, will use the PACT assessment piloted by UCSB and a dozen other universities.

Assessment of Classroom Performance

California’s switch to performance-based assessment is one of a handful of efforts in teacher evaluation to shift away from judging teacher quality by the credentials they bring to schools and toward measuring—and improving—their actual performance in the classroom. The nonprofit National Board for Professional Teaching Standards, established in 1987, has granted advanced certification to 63,000 teachers nationwide in more than a dozen subjects. Candidates—experienced teachers who apply individually—are required to submit a portfolio of materials, including videotapes of themselves teaching, and must complete a series of subject-specific online exercises. The Teacher Advancement Program, launched in 1999 and operated by the California-based National Institute for Excellence in Teaching, uses a standards-based framework and evaluation by trained master teachers as the basis for teacher evaluation and compensation on a schoolwide basis. The Toledo Public Schools and the Toledo Federation of Teachers jointly sponsor a longstanding “peer assistance and review” program to evaluate inexperienced or underperforming teachers. And researchers at the University of Virginia have developed the Classroom Assessment Scoring System (CLASS) as an observational instrument for evaluating teachers in preK and the lower elementary grades (see "Neither Art nor Accident," HEL, January/February 2008).

Although in a few states, such as Connecticut, practicing teachers must pass performance assessments to earn full licenses (see sidebar "Connecticut’s BEST") Connecticut's BEST

Connecticut has been in the forefront of teacher evaluation since 1989, when it created the nation’s first statewide teacher-evaluation system to help raise the quality of teachers being licensed in the state.

The Connecticut Beginning Educator Support and Training Program (BEST) combines two years of mentoring and training for every new Connecticut teacher with an evaluation of the teacher’s performance against statewide instructional standards. Teachers must earn a satisfactory rating on the evaluation in order to become fully licensed.

The evaluations are based on portfolios that chronicle a unit of instruction five to eight hours long. The portfolios include information about the students’ demographic and academic backgrounds, unit goals, daily activity logs, student work with teacher feedback, and a reflective analysis. Teachers must also submit a videotape of at least 20 minutes of instruction.

Each portfolio is scored by several state-trained evaluators. The scorers evaluate the portfolios from four perspectives: instructional design, instructional implementation, assessment of learning, and teachers’ ability to analyze teaching and learning.

Both state officials and teachers are pleased with the program. “We’ve seen performance levels shoot up,” says Catherine Fiske Natale, director of education support and assessment for the state department of education. More than 90 percent of beginning teachers tell state education officials that the BEST program improves their teaching. A 2002 study also showed that the program may help keep attrition rates low among new teachers.

New Mexico and Wisconsin have recently introduced portfolio-based evaluations of new teachers similar to Connecticut’s.

Adapted with permission from Rush to Judgment: Teacher Evaluation in Public Education, by Thomas Toch and Robert Rothman (Washington, DC: Education Sector, 2008) California is the first state in the nation to require a performance assessment of teachers as a condition of initial licensure.

Frederick M. Hess, director of education policy studies at the American Enterprise Institute, cautions that the evidence is mixed on whether portfolio assessments accurately determine teacher competence. Moreover, he sees such an assessment as the kind of procedural hurdle that may keep potentially effective teachers out of the classroom. He would prefer instead to see states set minimal qualifications for licensure and leave it up to principals and school districts to decide what information is relevant to their needs. “I’m skeptical that this is a useful way to maximize the amount of talent in the labor pool,” he says.

But Sharon Robinson, president of the American Association of Colleges for Teacher Education, says that states ought to use the best information available in determining whether prospective teachers should earn licenses. She suggests that portfolio systems like PACT are more informative than the “snapshot approach” standardized tests use. “States spend money, and candidates spend money, on the credentialing process,” Robinson says. “Why not devote some of that money to a process that’s more informative to the institution and to the candidate?”

The legislation that authorized California’s new assessment requirement, passed back in 1998, was prompted by a chorus of concern over the shortage of qualified teachers in certain districts. For instance, a report by the Santa Cruz–based Center for the Future of Teaching and Learning had found that three-quarters of new teachers in the state were hired with emergency credentials.

A key provision in the new law called for the creation of a performance assessment to ensure that prospective teachers met state teaching standards. Teacher-education programs could also create their own alternatives, subject to approval by the California Commission on Teacher Credentialing.

A consortium of 30 universities, led by Stanford, decided to take advantage of that opportunity. According to Raymond L. Pecheone, codirector of the School Redesign Network at Stanford University, teacher educators at these institutions wanted to develop an instrument that would be tailored to specific subject areas and grade levels. They felt that this level of information would be useful to institutions in assessing their own programs. The instrument developed and piloted by the consortium, PACT, was approved for use in the licensure process by the commission in 2007.

“It’s the Best Thing I Have Done”

At the heart of PACT is what is known as the teaching event, in which candidates design and implement a series of lessons as part of their student teaching or internship. Candidates prepare a portfolio describing their teaching event and include a variety of evidence of what they did and what students—and the candidates themselves—learned during the lessons. Evidence might include the lesson plans, a video of the candidate teaching, assessment instruments, examples of student work, and the candidate’s written reflection on these materials. The portfolios are scored by trained assessors in each institution, who evaluate candidates’ performance in five categories: planning, instruction, assessment, reflection, and “academic language,” or the ability of teacher candidates to develop students’ language skills. (This last category is particularly important in a state in which large numbers of students are English-language learners, officials say.)

For instance, in the assessment for licensure in elementary literacy, there are three main tasks in the planning category: establishing a balanced instructional focus, making content accessible, and designing assessments. In the instruction category, candidates are evaluated on engaging students in learning and monitoring student learning during discussions. In the academic language section, they must demonstrate an understanding of language demands posed by classroom tasks and support of students’ academic language development. Evaluators rate the students’ portfolios by assigning a score of 1 to 4 for each task in each category, based on a rubric developed by the PACT consortium. Candidates pass the teaching event if they pass all five categories and have no more than three scores of “1” across all 11 tasks.

Teacher educators at the participating institutions admit that the process of compiling the portfolios is time consuming, but say many students appreciate the experience. “They struggle, but in the end they come back to us and say, ‘I have learned so much,’” says Carolyn Nelson, associate dean of the Connie L. Lurie School of Education at San José State University, which is part of the PACT consortium. “It has pushed students in ways we think are constructive and valuable.”

Patricia Brower, who participated in PACT as a graduate student at San José State University after 10 years as a classroom teacher, agrees. “It took four hours a week for over 15 weeks,” she says, acknowledging that she probably wrote more than her peers. “It ends up being an enormous amount of writing.”

Nonetheless, she is convinced that preparing the portfolio made her better prepared to teach. Brower says the effort helped her tie together all the theory she had learned in her classes and apply it in a classroom setting. “It was the best thing I have done in teacher preparation,” she says.

A Shared Vision

As the designers of PACT intended, the assessment provides information that teacher educators can use to improve teacher preparation, says Lynne H. Cook, dean of the college of education at California State University–Dominguez Hills. For one thing, she notes, the training that the faculty members receive to evaluate PACT portfolios helps ensure that they share ideas of what good teaching looks like. Even if the assessment were not required for licensure, she says, she would implement it so that the faculty could share a vision of high-quality instruction.

“The faculty are all using the same language,” she says. “I particularly like that they are talking about what they mean by ‘performance.’ I don’t know anything that will give you a guarantee [of a new teacher’s performance in the classroom], but [PACT is] a better predictor than what we’ve had in the past,” she adds.

At UCSB’s Gevirtz School of Education, the faculty used PACT data to gauge the strengths and weaknesses of their teacher-education program and plan improvements. “At first, the faculty felt threatened,” program director Sloan recalls. “They felt like the program was working and meeting student needs. But we put the student work on the table and realized we weren’t doing what we thought we were doing, and candidates weren’t well prepared.

“The faculty can still identify the day we took four samples of student work and did a read-around of certain portfolios,” Sloan continues. When results indicated weaknesses, particularly in classroom assessment and developing students’ academic language, she says, “we were absolutely surprised by what we saw. It changed our thinking.”

In response to those findings, she notes, the department revamped the foundations curriculum to add content in developing academic language for English-language learners, as well as in using data to guide instruction. In addition, faculty members who teach methods courses developed a common lesson-design framework, which asks candidates to state the criteria for assessing student work. “You can see the PACT influence” in the framework, Sloan says.

As a result of this process, she says, faculty members began to recognize that their responsibility toward teacher candidates extended beyond the particular subject area in which they taught. “Our faculty could not be comfortable any more focusing just on their piece of the puzzle,” Sloan says. “The special education faculty needed to be better versed in English as a second language. Field supervisors needed to be better versed in assessment support.

“Everybody is more knowledgeable overall,” she adds.

Predicting Practice

Stanford’s Pecheone says assessment results from the teacher-education programs that piloted PACT suggest that other institutions are making similar changes. “We’ve found that, over the last three years, [scores on the] assessment dimensions are significantly increasing each year,” he says. “It appears to be that universities are using the data for program improvement.”

Pecheone and others in the consortium hope that, over time, the new instrument can raise standards for teachers by clarifying the definition of good teaching, helping faculty members refine their own curriculum and instruction, and providing stronger evidence of what teacher candidates have learned or need to learn.

Pecheone says he plans to conduct a study that will follow graduates over several years and match their scores on the assessment with their students’ achievement test scores. He notes that a similar study in Connecticut found that students of teachers who earned high scores on that state’s portfolios outperformed by three or more academic months students whose teachers scored lower.

“There is some evidence that this credible, defensible measure of teaching has some relationship to student learning, as you would hope it has,” Pecheone says.

Robert Rothman is a senior editor at the Annenberg Institute for School Reform at Brown University and editor of Voices in Urban Education. He is coauthor, with Thomas Toch, of Rush to Judgment: Teacher Evaluation in Public Education (Education Sector, January 2008).