Torture talk: An uncontrolled experiment is still an experiment.

Paul Alper points us to this horrifying op-ed by M. Gregg Bloche about scientific study of data from U.S. military torture programs.

I’ll leave the torture stuff to the experts or this guy who you’ve probably heard of.

Instead, I have a technical point to make. In the op-ed, Bloche writes:

In a true experimental study, the C.I.A. would have had to test its interrogation strategy against one or more standard interrogation methods, using experimental and control groups of captives. There’s no evidence the agency did this.

They [the psychologists, Mitchell and Jessen] argued that interrogation strategies can’t be standardized and therefore can’t be compared, like medical treatments, in randomized, prospective fashion.

No one, though, is claiming that C.I.A. review efforts involved experimental and control groups and so were “experimentation” as science defines it.

This statement, that a true experiment requires a control group, is wrong. In a controlled experiment there needs to be a comparison, but an uncontrolled experiment is still a form of experiment. To put it another way, we use the term “controlled experiment” because control is not a necessary part of an experiment. In many cases control is good practice, and control makes it easier to perform certain inferences, but you can do experimentation with out a control group.

14 thoughts on “Torture talk: An uncontrolled experiment is still an experiment.

    • A reasonable experiment to do without a control is something you do on yourself, like changing your habits in some way, or changing your business practices.

      It’s an experiment – there ought to be a time afterwards where you consider whether it worked as expected, or whether you should go back to the old – but you have to live with a lot of uncertainty, and you can pretty much forget about quantifying it.

      Maybe Mitchell and Jessen should torture themselves for a bit and see if it makes them more inclined to help their prosecutors.

      • jrc, Yes, it does require a control. The pre-Mentos cola is the control. If you had never seen cola absent Mentos you would have no idea that the candy could cause such a reaction. As Shravan suggests, all experiments have a control (comparison) condition. Control groups are obvious; units serving as their own control is less obvious but still very much a control. This applies to self-experimentation as well. In fact, from the potential outcomes perspective, the unit absent the treatment immediately before receiving the treatment is the closest you can get to an ideal control under real-world conditions. There is no such thing as an ‘uncontrolled’ experiment. To suggest otherwise is a fundamental misunderstanding of what an experiment is and how it works. Now, whether a control is good is a different question.

  1. If memory serves, we hanged German and Japanese “researchers” for this type of thing right after WWII.

    Still, if you are going to do it do it right so that after we hang you, you at least get an acknowledgement in the subsequent paper.

  2. From the article by Bloche

    “Our real concern about what Dr. Mitchell and Dr. Jessen did or didn’t do isn’t human experimentation; it’s torture. Collection of data from victims is morally incidental.”

    Recall how the Nazis did terrible “experiments” such as seeing how long prisoners could survive in cold water. The moral issue after the war involved whether or not this data, taken under inhuman circumstances, could or should be used for the better understanding of hypothermia. From

    https://en.wikipedia.org/wiki/Nazi_human_experimentation#Modern_ethical_issues

    “This, together with the recent use of data from Nazi research into the effects of phosgene gas, has proven controversial and presents an ethical dilemma for modern physicians who do not agree with the methods used to obtain this data. Some object on an ethical basis, and others have rejected Nazi research purely on scientific grounds, pointing out methodological inconsistencies.”

Leave a Reply to jrc Cancel reply

Your email address will not be published. Required fields are marked *