Collin Hitt writes:
I study wrong answers, per your blog post today. My research focuses mostly on surveys of schoolchildren. I study the kids who appear to be just filling in the bubbles, who by accident actually reveal something of use for education researchers.
Here’s his most recent paper, “Just Filling in the Bubbles: Using Careless Answer Patterns on Surveys as a Proxy Measure of Noncognitive Skills,” which concludes:
Using two national longitudinal datasets, I [Hitt] show that careless answer patterns from adolescent respondents are negatively predictive of later educational attainment, independent of cognitive ability and other traditionally-measured noncognitive skills. I posit that careless answers, as I have quantified them, proxy as a behavioral measure of a negative noncognitive trait.
No graphs, though. Gotta work on the graphs. Need some graphs. graphs graphs graphs
I’m getting mixed signals here — do you like graphical displays or not?
No, they’re not the way science really gets done.
I’m lost. Really need a /sarc tag.
I thought blogs were not the way science really gets done. Graphs bad too?
Graphs bad. Tables good. See here.
The ironie.
so behavioral measures of “f*** this test” proxy for (the apparently negative) non-cognitive traits related to “f*** your rules and values”?
sorry – sometimes I worry that what we are actually measuring when we evaluate non-congnitive traits of this sort is “docility”. if you look at the measures (standardized test scores, attendance, disciplinary record, final attainment, etc.) they all seem like measures of kids who stay in line and do what their told.
the economist in me is also not sure that dropping out earlier because you are a person who dislikes school (as revealed in your test taking habits) is somehow reflective of something “negative” in that person – it is just who they are, and it isn’t like we think there is one right way to be, right? i mean, i do think people make poorly informed schooling decisions, but i’m inclined to believe that goes both ways in modern America.
i guess i would prefer that we talk about non-cognitive traits not as “negative” or “positive” but as more/less useful for specific purposes, and if we want to benchmark a purpose (say, with earnings) that is fine, but then they are “useful or not-useful” for achieving higher earnings (or maybe “valued” or “not-valued” in the market place)… but they aren’t “positive” or “negative”.
The ‘ironie’ as I saw it was that the label for the ‘proxy’ for a psychological construct was acknowledged to not be supported by any evidence or logical argument, but was then applied anyway. One might even say carelessly.
As they say “attitude is not conducive to learning”.
But it is conducive to disrupting academic fields and industries. So I wouldn’t totally rule it out as invaluable.