Philosophy professor Gary Gutting writes:
Public policy debates often involve appeals to results of work in social sciences like economics and sociology. . . . How much authority should we give to such work in our policy decisions? . . . The core natural sciences (e.g., physics, chemistry, biology) are so well established that we readily accept their best-supported conclusions as definitive. . . . But how reliable is even the best work on the effects of teaching? How, for example, does it compare with the best work by biochemists on the effects of light on plant growth? Since humans are much more complex than plants and biochemists have far more refined techniques for studying plants, we may well expect the biochemical work to be far more reliable. . . . While the physical sciences produce many detailed and precise predictions, the social sciences do not.
OK, fine. But then comes the punchline:
Given the limited predictive success and the lack of consensus in social sciences, their conclusions can seldom be primary guides to setting policy. At best, they can supplement the general knowledge, practical experience, good sense and critical intelligence that we can only hope our political leaders will have.
This all makes sense but I’m a bit confused. In no area are scientific conclusions “the primary guides to setting policy.” Political and business leaders rule policy; the rest of us can just supply advice. It’s not like physicists are in charge of energy policy (if so, I expect we’d have a big fat carbon tax), nor are biologists in charge of the teaching of evolution in many states. So I’m not quite sure what Gutting is talking about.
The only place where I see social scientists controlling policy is (some) economists’ influence over economic policy. But this is a well-known issue, usually framed not as a matter of scientific expertise (or lack thereof) but in terms of massive conflicts of interests (for example, Lawrence Summers’s multimillion-dollar payoff from a hedge fund). But this can’t be Gutting’s point: if it were, he’d talk about economists, not social science in general. But in that case I just don’t get it. It’s not like there are a bunch of number-crunching sociologists running around telling the government what to do!
It seems to me that, the field economics aside, policy is run just as Gutting would like: “our political leaders” (as he puts it) can pretty much do what they’d like, constrained by the political opposition but feeling no particular obligation for science (social or otherwise) to be “primary guides to setting policy.” I’m not complaining here—I don’t know that science should be a primary guide in most contexts (if it were, maybe we’d all be riding flying cars by now, with all the parking problems that would entail)—I just don’t see what Gutting is getting at.
Like you, I accept the statements about the uncertainty around social sciences; however, I find it strange coming from a philosophy professor. Is the implication that policy should be informed by philosophy? In this case, I imagine there is just as much to say about the “definitive conclusions” as there is about social science.
What is the alternative to social science in informing social policy?
“I don’t know that science should be a primary guide in most contexts”
What’s your alternative then? Seems to me there is a spectrum with purely rational science at one end and pure guessing at the other. Personally. I can’t imagine any reason to feel inclined to move (or stay) in the latter direction.
My own perception is that Gutting hits the nail right on the head. Our national policies float on a sea of social science expertise. For example Head Start and all the rest of the debris from the Johnson presidency were firmly based on social science knowledge. Several of our most important US Supreme Court cases have been based on social science, like Brown v. Board and Griggs v. Duke Power. Our public school policies are based on their own version of social science.
One can object that what get called social science in these cases is not really social science but is instead something else. Fair enough. Reading right wing blogs and the like quickly reveals that social science has become a dirty word in those circles, associated with “liberals.” This is not right, in my opinion, since I am a liberal while those people are not.
On the other hand at my university it is certainly true that most faculty I know in social science departments have a defensive and pervasive belief that they are on the “right track” with their work and that they occupy the moral high ground.
Henry:
Given that your most recent example comes from 1971, this suggests that maybe social-science-based policy used to be a problem, but not for the past forty years.
Henry is yanking your chain. The problem is that what good social science has revealed at least since the 1965 Coleman Report (parents matter, genes matter, race matters, sex matters, IQ matters, social engineering usually fails, blank slatism is bogus, etc.) is the exact opposite of what the vast majority of social scientists want it to reveal.
I suspect this is a dig at policy thinktanks more than anything.
I think the point is that if we want to solve a physical problem, such as how to improve the performance of a cell phone network or increase the yield of a coffee plantation, the physical sciences have a lot to contribute. These problems involve policy decisions too, but in general, given a set of alternative policies, the physical sciences can be used to predict the outcome of each one, and decision makers can use those predictions to select the best policies.
But in the social realm, where we want to do things like increase high school graduation rates or improve access to health care, the social sciences provide much less information. Policymakers — and the public in general — shouldn’t consider predictions made by social scientists with the same degree of confidence that they consider predictions made using the physical sciences.
This was my understanding as well.
John S., I would like to be generous to the writer, but that is NOT the implication of the punchline quotation. In fact, given the contingent nature of social phenomena, many of us would argue that relying on careful scientific analysis (this includes quantitative and qualitative research findings, by the way) is MORE, not less important.
The takeaway could have been something like “given lower confidence levels, policy makers need to be circumspect about the immediate impact of proposed programs and be willing to adjust policy solutions after additional evidence is collected.” Instead, the author suggests just relying on gut intuition or “general knowledge” which could easily be translated into “just substitute my own biased perceptions and political biases.”
He has a pretty naive model of how the natural and physical sciences inform public policy. Sure, physics can make some highly precise and accurate predictions (e.g. about eclipses), but this sort of basic science doesn’t bear much on public policy. When you start to look at practical policy contexts – how to deal with the outbreak of bird flu, whether to restrict aviation after a a volcanic eruption – the underlying science is a lot more contingent and value-laden. Obviously that doesn’t mean you just ignore it. But it does mean that the physics model of science-for-policy is a pretty useless one.
Andrew,
This is such a badly misinformed and unintelligible posting, I’m shocked that the Times would give this any sort of credibility. Let’s start with the main premise, that scientifically supported findings in the social sciences, because they have some level of random error, are therefore less valid as a guide to policy than “general knowledge, good sense, and critical intelligence.” This is really a philosopher arguing that “good sense” ought to be our guide to policy making? Good sense or critical intelligence based on what?
His main guide to the philosophy of science is a book written by a management consultant. Besides, the author seems completely unaware of the growth in controlled field experiments in the social sciences, which he describes as “seldom possible.” HIs examples are tendentious, comparing “the confidence we should place in astronomers’ calculations of eclipses and a SMALL marketing study suggesting that consumers prefer laundry soap in blue boxes.” (Caps added).
This is horribly uninformed. What the heck is the Times thinking?
Isn’t this just garden-variety physics envy?
As a public policy doctoral student, I think this is just a bad job of writing. Public policy wants to look at the impact of policies (causality is sort of assumed). If we can nail the mechanism of what happened that is gravy. But form my perspective the important portion is looking at the actual outcomes of the policy. Hence why I am a big fan of the ATE, it is simple and tells me what the policy did rather than me having to generate some theoretical construct (more economics if anything). I think policy too easily gets lumped into economics (a lot of policy students I know want to go that route). But we really are ignoring the merits of what public policy can do, we only get a bad reputation by the masters students and bachelors students that did not seek to go for that doctoral level.
At the end of the day policy is about reporting what happened in the real world .
Philosophers of science always revere physicists. True, the study appealed to for the causal inference does seem flawed on a number of levels (like predicting future salaries?), and it wouldn’t help if their “predictions” were affirmed. That Obama seems to have presented it as strong evidence is doubtless what is irksome. Still, this kind of criticism, taken at its word, tends to encourage the use of RCTs in all kinds of social and economic policy evaluations, despite the fact that they are not genuinely controlled causal studies of the factors researchers claim to be studying (an example is in developmental econ).
Forget the parking problems! Crash avoidance in 3-d is a lot harder than 2-d (and thank your lucky stars we don’t have 4-d).
And physics envy, maybe, but the last time I checked, we didn’t have the ear of POTUS either.
It’s kind of funny to look at the reactions education researchers have to No Child Left Behind. It effects so much of, and is discussed in so much of, their literature. And yet, they had essentially no part in creating it, and if it does get repealed (as they want it to so badly), it won’t be because of anything the academics have done. It’s as if there were an absolute monarchy where the king’s decrees are absolute policy, and in this monarchy you have a tight-knit opposition who solemnly write journal articles in opposition to the king– but nothing they write matters whatsoever, to anyone except themselves.
Anon:
As a sometime education researcher myself, I don’t think your story is “kind of funny” at all. Education research is not perfect, but I prefer research-based solutions to the alternative. I do agree with you about the “solemn” journal articles, though. I’d prefer if scholarly publications could be more humorous. When I try to throw in a joke or two in my research papers, I’m often swatted down by referees. And, no, we don’t have an absolute monarchy. We have a representative democracy, and our elected representatives are in charge (within the bounds of the law).
The whole post was idiotic. His argument seems to be that because social science is imperfect we should ignore it completely and just operate based on belief? No knowledge is his substitute for imperfect knowledge. I am not sure how this can make sense. A policymaker may start with uninformed priors, but if the social science results appear they ought to lead to some adjustment in posterior beliefs. Not zero impact. What kind of decision making policy is Gutting suggesting.
For better or worse, “the general knowledge, practical experience, good sense and critical intelligence” of our leaders is usually based on some form of social science knowledge (however rudimentary).
As Keynes so eloquently put it, “Practical men, who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist. Madmen in authority, who hear voices in the air, are distilling their frenzy from some academic scribbler of a few years back.”
The ironic reality is that social science has been quite successful at demonstrating the failures of social engineering.
Steve is certainly correct but hardly anyone notices what he points out. The social science/education folks at my university are behind a push for more and more early childhood education in spite of good quality research that shows that there is no payoff. Robert Putnam does quality research on diversity yet his work elicits fear-grimaces from the social science crew around here. The health sociology folks have the same kind of grimace when one mentions the recent CDC report on Hispanic Health.
So my reaction to the Gutting article is knee-jerk approval because I am confusing “social science” with “social scientists”. The latter in my experience really are awash in ideology, most of them.
Henry:
I agree that many social scientists seem to be blinded by ideology, but I still don’t understand what’s to agree with in Gutting’s article. He writes that the “conclusions [of social scientists] can seldom be primary guides to setting policy. At best, they can supplement the general knowledge, practical experience, good sense and critical intelligence that we can only hope our political leaders will have.” But that’s the system we currently have! With the exception of economists, social scientists are not setting policy (as noted above, it’s not like there are a bunch of number-crunching sociologists running around telling the government what to do). So I don’t see Gutting’s point. We already have the system he wants.
I’ve been a social science aficionado since I was 13 in 1972. My first published bit in a national magazine was a 1973 letter to the editor of National Review responding to a review of Christopher Jencks’s book Inequality, a re-analysis of the Coleman Report:
“Having read Ernest van den Haag’s article on Christopher Jencks, I am reminded of an old psychiatry joke: A psychotic (egalitarian, in this little morality story) says. “All people are equal, and I’ll fight anyone who says I’m wrong.” A neurotic (Jencks) says, “People aren’t equal, and I just can’t stand it.””
STEVEN SAILER
And that pretty much sums up my writing career over the subsequent 39 years: I read social scientists extremely carefully and then point out their findings that they don’t want people to notice.
For example, one of the giant policy questions of the last generation has been: how quickly and how broadly will Mexican illegal immigrants assimilate into the educated middle class? Looking around me here in my native Southern California, I’m much more pessimistic than the Washington and New York-based pundits who don’t have decades of personal experience with Mexican-Americans to draw upon.
But how much social science work has been done to answer this crucial question? The answer is very little, mostly because experts pretty much know what the answer is and don’t want to get in trouble for pointing it out. So, I spend a lot of effort reading obscure stuff from Chicano Studies departments and the like that provide clues, such as Ortiz and Telles’s book “Generations of Exclusion,” which found a 6% college graduation rate for fourth-generation Mexican Americans:
http://www.vdare.com/articles/roll-over-michael-barone-even-fourth-generation-mexicans-are-failing
Henry, flogging social scientists instead of social science is hardly an improvement. I’m pleased that you are happy to make a broad statement about social scientists based on your experience. As a social scientist, I don’t use my personal experience to draw general conclusions about any group.
Part of the issue appears to be that pretty much anyone can call themselves a “social scientist”, and social science covers a range of methodologies unlike other disciplines, such as physics. For example, qualitative methods are acceptable social science techniques. Thus, there is a definition problem with “social scientist” which contributes to the issue at hand.
I have formal qualifications in social science (primarily psychology, but also human development and sociology) and statistics, and other social scientists I have worked with have not had this skill combination, however as I have not performed a survey of social scientists, I do not know how typical or unusual I am with this skill mix.
Those who support policy making based on “the general knowledge, practical experience, good sense and critical intelligence” have not kept up on the research in decision making. Thinking, Fast and Slow, By Kahnaman nicely summarizes our current knowledge about human decision making, and it is not at all flattering.
A separate issue is identifying the “social sciences” and “social scientists.” In recent readings I am finding a wealth of “social science” “research” conducted by humanists in which they draw sometimes large theoretical conclusions based on little more than introspection. Looking around we see many notable “social scientists” who are, shall we say, self-trained having little or no genuine background in social science.
Sadly many social scientists/educational researchers receive too little training in science because there is such a great need for them as service providers. While they have the right credentials they lack the depth of training and so, as some have already complained, they are hopelessly biased. These, however, are not scientists.
Happily there are many legitimate social scientists who do good work and from whom policy makers could learn a lot. Sadly these are not the folks that policy makers hear from or hear about. The people policy makers hear from and about are generally not genuine scientists, social or otherwise.
The first task then is to separate the real social science from the stuff that masquerades as science.
http://www.nber.org/papers/w17969
Arnaud Costinot, Dave Donaldson
“When asked to name one proposition in the social sciences that is both true and non-trivial, Paul Samuelson famously replied: ‘Ricardo’s theory of comparative advantage’.”
“we find that Ricardo’s theory of comparative advantage has significant explanatory power in the data, at least within the scope of our analysis. “