Jeff points us to this news article by Asher Mullard: Bayer halts nearly two-thirds of its target-validation projects because in-house experimental findings fail to match up with published literature claims, finds a first-of-a-kind analysis on data irreproducibility. An unspoken industry rule alleges that at least 50% of published studies from academic laboratories cannot be repeated [...]
Archive of posts filed under the Public Health category.
The acupuncture paradox
The scientific consensus appears to be that, to the extent that acupuncture makes people feel better, it is through relaxing the patient, also the acupuncturist might help in other ways, encouraging the patient to focus on his or her lifestyle. A friend recommended an acupuncturist to me awhile ago and I told her the above [...]
Going viral — not!
Sharad explains: HIV/AIDS, like many other contagious diseases, exemplifies the common view of so-called viral propagation, growing from a few initial cases to millions through close person-to-person interactions. (Ironically, not all viruses in fact exhibit “viral” transmission patterns. For example, Hepatitis A often spreads through contaminated drinking water.[1]) By analogy to such biological epidemics, the [...]
Introductory overview lectures at the Joint Statistical Meetings in Miami this coming week
Political arguments and political representation often rely on statistics, whether it’s counting votes, measuring public opinion, or assessing the effects of policies. Sunday afternoon, Andrew Gelman discusses how models of political behavior can be improved by moving toward a more data-based perspective. Evidence-based medicine has the potential to transform health care, to focus resources on [...]
Design of nonrandomized cluster sample study
Rhoderick Machekano writes: I have a design question which has been bothering me and wonder if you can clear for me. In my line of work, we often conveniently select health centers and from those sample patients. When I am doing sample size estimation under this design do I account for the design effect – [...]
Still more Mr. P in public health
When it rains it pours . . . John Transue writes: I saw a post on Andrew Sullivan’s blog today about life expectancy in different US counties. With a bunch of the worst counties being in Mississippi, I thought that it might be another case of analysts getting extreme values from small counties. However, the [...]
Mr. P by another name . . . is still great!
Brendan Nyhan points me to this from Don Taylor: Can national data be used to estimate state-level results? . . . A challenge is the fact that the sample size in many states is very small . . . Richard [Gonzales] used a regression approach to extrapolate this information to provide a state-level support for [...]
A survey’s not a survey if they don’t tell you how they did it
Since we’re on the topic of nonreplicable research . . . see here (link from here) for a story of a survey that’s so bad that the people who did it won’t say how they did it. I know too many cases where people screwed up in a survey when they were actually trying to [...]
Christakis-Fowler update
After I posted on Russ Lyons’s criticisms of the work of Nicholas Christakis and James Fowler’s work on social networks, several people emailed in with links to related articles. (Nobody wants to comment on the blog anymore; all I get is emails.)
Here they are:
Controversy over the Christakis-Fowler findings on the contagion of obesity
Nicholas Christakis and James Fowler are famous for finding that obesity is contagious. Their claims, which have been received with both respect and skepticism (perhaps we need a new word for this: “respecticism”?) are based on analysis of data from the Framingham heart study, a large longitudinal public-health study that happened to have some social network data (for the odd reason that each participant was asked to provide the name of a friend who could help the researchers locate them if they were to move away during the study period.
The short story is that if your close contact became obese, you were likely to become obese also. The long story is a debate about the reliability of this finding (that is, can it be explained by measurement error and sampling variability) and its causal implications.
This sort of study is in my wheelhouse, as it were, but I have never looked at the Christakis-Fowler work in detail. Thus, my previous and current comments are more along the lines of reporting, along with general statistical thoughts.
We last encountered Christakis-Fowler last April, when Dave Johns reported on some criticisms coming from economists Jason Fletcher and Ethan Cohen-Cole and mathematician Russell Lyons.
Lyons’s paper was recently published under the title, The Spread of Evidence-Poor Medicine via Flawed Social-Network Analysis. Lyons has a pretty aggressive tone–he starts the abstract with the phrase “chronic widespread misuse of statistics” and it gets worse from there–and he’s a bit rougher on Christakis and Fowler than I would be, but this shouldn’t stop us from evaluating his statistical arguments. Here are my thoughts:
Statistical methods for healthcare regulation: rating, screening and surveillance
Here is my discussion of a recent article by David Spiegelhalter, Christopher
Sherlaw-Johnson, Martin Bardsley, Ian Blunt, Christopher Wood and Olivia Grigg, that is scheduled to appear in the Journal of the Royal Statistical Society:
I applaud the authors’ use of a mix of statistical methods to attack an important real-world problem. Policymakers need results right away, and I admire the authors’ ability and willingness to combine several different modeling and significance testing ideas for the purposes of rating and surveillance.
That said, I am uncomfortable with the statistical ideas here, for three reasons. First, I feel that the proposed methods, centered as they are around data manipulation and corrections for uncertainty, has serious defects compared to a more model-based approach. My problem with methods based on p-values and z-scores–however they happen to be adjusted–is that they draw discussion toward error rates, sequential analysis, and other technical statistical concepts. In contrast, a model-based approach draws discussion toward the model and, from there, the process being modeled. I understand the appeal of p-value adjustments–lots of quantitatively-trained people know about p-values–but I’d much rather draw the statistics toward the data rather than the other way around. Once you have to bring out the funnel plot, this is to me a sign of (partial) failure, that you’re talking about properties of a statistical summary rather than about the underlying process that generates the observed data.
My second difficulty is closely related: to me, the mapping seems tenuous from statistical significance to the ultimate healthcare and financial goals. I’d prefer a more direct decision-theoretic approach that focuses on practical significance.
That said, the authors of the article under discussion are doing the work and I’m not. I’m sure they have good reasons for using what I consider to be inferior methods, and I believe that one of the points of this discussion is to give them a chance to give this explanation.
Finally, I am glad that these methods result in ratings rather than rankings. As has been discussed by Louis (1984), Lockwood et al. (2002), and others, two huge problems arise when constructing ranks from noisy data. First, with unbalanced data (for example, different sample sizes in different hospitals) there is no way to simultaneously get reasonable point estimates of parameters and their rankings. Second, ranks are notoriously noisy. Even with moderately large samples, estimated ranks are unstable and can be misleading, violating well-known principles of quality control by encouraging decision makers to chase noise rather than understanding and reducing variation (Deming, 2000). Thus, although I am unhappy with the components of the methods being used here, I like some aspects of the output.
Should kids be able to bring their own lunches to school?
I encountered this news article, “Chicago school bans some lunches brought from home”: At Little Village, most students must take the meals served in the cafeteria or go hungry or both. . . . students are not allowed to pack lunches from home. Unless they have a medical excuse, they must eat the food served [...]
Is the internet causing half the rapes in Norway? I wanna see the scatterplot.
Ryan King writes:
This involves causal inference, hierarchical setup, small effect sizes (in absolute terms), and will doubtless be heavily reported in the media.
The article is by Manudeep Bhuller, Tarjei Havnes, Edwin Leuven, and Magne Mogstad and begins as follows:
Does internet use trigger sex crime? We use unique Norwegian data on crime and internet adoption to shed light on this question. A public program with limited funding rolled out broadband access points in 2000-2008, and provides plausibly exogenous variation in internet use. Our instrumental variables and fixed effect estimates show that internet use is associated with a substantial increase in reported incidences of rape and other sex crimes. We present a theoretical framework that highlights three mechanisms for how internet use may affect reported sex crime, namely a reporting effect, a matching effect on potential offenders and victims, and a direct effect on crime propensity. Our results indicate that the direct effect is non-negligible and positive, plausibly as a result of increased consumption of pornography.
How big is the effect?
The happiness gene: My bottom line (for now)
I had a couple of email exchanges with Jan-Emmanuel De Neve and James Fowler, two of the authors of the article on the gene that is associated with life satisfaction which we blogged the other day. (Bruno Frey, the third author of the article in question, is out of town according to his email.) Fowler [...]
Suspicious pattern of too-strong replications of medical research
Howard Wainer writes in the Statistics Forum: The Chinese scientific literature is rarely read or cited outside of China. But the authors of this work are usually knowledgeable of the non-Chinese literature — at least the A-list journals. And so they too try to replicate the alpha finding. But do they? One would think that [...]
Statistics ethics question
A graduate student in public health writes: I have been asked to do the statistical analysis for a medical unit that is delivering a pilot study of a program to [details redacted to prevent identification]. They are using a prospective, nonrandomized, cohort-controlled trial study design. The investigator thinks they can recruit only a small number [...]
Data mining and allergies
With all this data floating around, there are some interesting analyses one can do. I came across “The Association of Tree Pollen Concentration Peaks and Allergy Medication Sales in New York City: 2003-2008″ by Perry Sheffield. There they correlate pollen counts with anti-allergy medicine sales – and indeed find that two days after high pollen [...]
Job opening at NIH for an experienced statistician
This announcement might be of interest to some of you. The application deadline is in just a few days: The National Center for Complementary and Alternative Medicine at the National Institutes of Health is seeking an additional experienced statistician to join our Office of Clinical and Regulatory Affairs team. www.usajobs.gov is accepting applications through April [...]
A possible resolution of the albedo mystery!
Remember that bizarre episode in Freakonomics 2, where Levitt and Dubner went to the Batcave-like lair of a genius billionaire who told them that “the problem with solar panels is that they’re black.” I’m not the only one who wondered at the time: of all the issues to bring up about solar power, why that [...]
Annals of really really stupid spam
This came in the inbox today:
Fattening of the world and good use of the alpha channel
In the spirit of Gapminder, Washington Post created an interactive scatterplot viewer that’s using alpha channel to tell apart overlapping fat dots better than sorting-by-circle-size Gapminder is using: Good news: the rate of fattening of the USA appears to be slowing down. Maybe because of high gas prices? But what’s happening with Oceania?
NYT shills for personal DNA tests
Kaiser nails it. The offending article, by John Tierney, somehow ended up in the Science section rather than the Opinion section. As an opinion piece (or, for that matter, a blog), Tierney’s article would be nothing special. But I agree with Kaiser that it doesn’t work as a newspaper article. As Kaiser notes, this story [...]
“If it saves the life of a single child…” and other nonsense
This post is by Phil Price. An Oregon legislator, Mitch Greenlick, has proposed to make it illegal in Oregon to carry a child under six years old on one’s bike (including in a child seat) or in a bike trailer. The guy says “”We’ve just done a study showing that 30 percent of riders biking [...]
“The truth wears off: Is there something wrong with the scientific method?”
Gur Huberman asks what I think of this magazine article by Johah Lehrer (see also here). My reply is that it reminds me a bit of what I wrote here. Or see here for the quick powerpoint version: The short story is that if you screen for statistical significance when estimating small effects, you will [...]
$3M health care prediction challenge
i received the following press release from the Heritage Provider Network, “the largest limited Knox-Keene licensed managed care organization in California.” I have no idea what this means, but I assume it’s some sort of HMO. In any case, this looks like it could be interesting: Participants in the Health Prize challenge will be given [...]