AT writes:
A Facebook friend pointed me to this Gladwell piece discussing how you can(‘t) predict whether a teacher will be successful, but more importantly, on the range of advancement of a class depending on a teacher’s ability.
The claim that a good teacher can make as much as a full year’s difference in their students’ advancement isn’t too surprising to me [AT]. What I want to know is whether there are good studies around that look at the interaction effects between teachers and their students’ backgrounds. In particular, I’d say that there are a number of teachers I’ve had who are very polarizing — some make their students advance far, some stall them in their paths and make them give up the discipline.
My quick reply: From the work of Jonah Rockoff and others, I am convinced that teacher effects are real and they are large. And school effects are mostly the composition of teacher effects. I’m not sure about how large the interactions are (i.e., if some teachers do better with good students and others do better with poor students). Jennifer and I have talked about estimating such interactions (with a big multilevel model, of course) but I don’t know what’s up with that.
And, of course, this has no implications for university teaching, where as we know the sole qualification is to publish technical articles in obscure journals.
There was recently a working paper published by a labor market evaluation institute in Sweden on a similar topic*. They focused on the cognitive ability and non-cognitive skills among teachers. They found that teachers with higher (lower) cognitive ability were good for students with higher (lower) cognitive ability, but not the other way around. They actually found that it was worse for a student with low cognitive ability if they had a high cognitive ability teacher. High cognitive ability teachers were generally somewhat better for boys compared for girls. For students with poor academic results, it was more important to have a teacher with good "leadership skills" rather than a teacher with high cognitive ability.
* http://www.ifau.se/upload/pdf/se/2008/wp08-25.pdf
If you want to understand teacher effects and/or value-added models, you should check out this paper by Jesse Rothstein:
ftp://www.tinbergen.nl/pub/papers/Temp/rothstein.pdf
Some ideas only sound good if you say them fast. Malcolm Gladwell espoused two such ideas in his essay "Most Likely to Succeed" (NYer 12/15/08). The first was that the college performance of a quarterback is not predictive of his performance in the NFL, and the second is that teachers’ classroom performance is not predictable from prior academic achievement. Neither is correct and Gladwell’s supporting evidence for both conclusions is flawed for the same reason – a lack of random assignment.
The quality of a quarterback is determined by many things, but a key one is the success that their team has when they are at the helm. Quarterbacks are not assigned at random to teams — in fact, the lowest ranked teams get first crack at the highest ranked quarterbacks (think of Archie Manning, a whiz out of Ole Miss, who languished his entire career with a team that couldn't block, run or catch passes). If you adjust for team quality the draft rank of the quarterback matters.
Gladwell's conclusions about teachers rests heavily on estimates of value added. This is a slippery idea whose validity rests on several unlikely assumptions — random assignment of students to teachers, the quality of the test's score scale so that changes have equal meaning throughout their range, and how missing data are treated. Gladwell's conclusion that pre-teaching qualifications have nothing to do with teacher success is foolish on its face (e.g note how successful "Teach For America " teachers have been, where their principal qualification was that they were graduates of highly selective colleges).
Howard,
Good point about the QB's. Presumably it would be possible to adjust for some of those team effects with a more elaborate model.
For teachers, I see what you're saying, but when Jonah Rockoff told us about some of his research in this area a couple years ago, he made a pretty convincing case that selection was not a plausible explanation for his findings.