NASHVILLE -- Ever since the advent of the federal Race to the Top program, data has played an increasing role in evaluating teacher performance with "value-added" test scores measuring educators' impact on student achievement in Tennessee and other states.
But a newly released multi-state study by Vanderbilt University researchers finds that simply observing teachers in a classroom rather than "value-added" testing is what really drives principals' decisions on hiring, firing, pay and other things.
And in the view of principals, those observations actually "may be more reliable than the 'value-added' measurements."
"Our data suggest that teacher observations, associated evidence and rubric scoring are becoming the main driver of principals' data use regarding teaching effectiveness and human-capital decisions in districts that have invested in these systems," said research team leader Ellen Goldring, Patricia and Rodes Hart Professor of Education Policy and Leadership at Vandy.
As a result, Goldring said in a Vanderbilt news release, "we believe that as these rigorous, observation-focused evaluation systems develop, value-added measures are playing a less exclusive role in principals' human-capital decision-making, despite policy mandates that suggest otherwise."
In the study, principals and central office personnel were surveyed or interviewed in six urban school districts in five states -- Tennessee (Nashville and Memphis); Maryland (Baltimore); Colorado (Denver); Florida (Hillsborough County) and Texas (Houston) during the 2012-13 and 2013-14 school years.
The team measured and evaluated key factors driving teacher hiring, contract renewal, classroom assignments and professional development.
Principals across all school districts identified a number of shortcomings in the usefulness of student test score-based models. For example, test results are generally not yet available when decisions are being made.
The study found many principals believed the numbers were unreliable because students are taught by multiple teachers. Moreover, the statistical models were too complicated to provide clearinterpretation.
In contrast, principals see actual classroom observation of teachers as both useful and reliable.
One principal told researchers that "I use observation data more than I use anything else. ... It wouldn't be fair for me to use that value-added data to judge who [a teacher] is. I take more seriously the observation data ... because it's what I see. That's real data to me."
The study found that 41 percent of the principals used teacher observation data twice a month to daily, while just 18 percent reported using teacher growth measures twice a month to daily.
"Acknowledging clear differences between the two types of data, our research suggests that student test scores are less central to principals' human capital decision-making, as rigorous teacher observation systems take root and become more widespread," said co-lead investigator Jason Grissom, assistant professor of public policy and education at Peabody, in the news release.
Contact staff writer Andy Sher at asher@timefreepress.com or 615-255-0550.