The file drawer’s on fire!

Kevin Lewis sends along this article, commenting, “That’s one smokin’ file drawer!”

Here’s the story, courtesy of Clayton Velicer, Gideon St. Helen, and Stanton Glantz:

We examined the relationship between the tobacco industry and the journal Regulatory Toxicology and Pharmacology (RTP) using the Truth Tobacco Industry Documents Library and internet sources. We determined the funding relationships, and categorised the conclusions of all 52 RTP papers on tobacco or nicotine between January 2013 and June 2015, as “positive”, “negative” or “neutral” for the tobacco industry. RTP’s editor, 57% (4/7) of associate editors and 37% (14/38) of editorial board members had worked or consulted for tobacco companies. Almost all (96%, 50/52) of the papers had authors with tobacco industry ties. Seventy-six percent (38/50) of these papers drew conclusions positive for industry; none drew negative conclusions. The two papers by authors not related to the tobacco industry reached conclusions negative to the industry (p < .001). These results call into question the confidence that members of the scientific community and tobacco product regulators worldwide can have in the conclusions of papers published in RTP.

I wonder what statisticians Herbert Solomon, Richard Tweedie, Arnold Zellner, Paul Switzer, Joseph Fleiss, Nathan Mantel, Joseph Berkson, Ingram Olkin, Donald Rubin, and Ronald Fisher would have said about this sort of selection bias.

20 thoughts on “The file drawer’s on fire!

      • “How the heck is junk science popularized in the media so often, while stuff like this is buried?”

        *Puts on “all is lost/skeptical/disillusion” -hat*

        This sentence sounds like a) the media popularizing junk science, and b) not writing about editors and/or authors having conflicts of interest (in this case ties to the tobacco industry) are opposites of each other and/or distinct. But, perhaps they are (the result of) the same problematic issues.

        (Side note: perhaps this is also a good moment to again applaud the people behind “Registered Reports” for bringing into this world, and promoting, a format where editors and authors can leave out any pre-registration information for the reader, hereby getting rid off 2 crucial aspects or pre-registration: 1) transparency, and 2) accountability.

        E.g. see:

        1) https://www.psychologicalscience.org/observer/preregistration-becoming-the-norm-in-psychological-science#comment-8352965

        2) https://osf.io/preprints/bitss/fzpcy/

        Also, a special thanks to all the “open science” people who have (uncritically?) endorsed and/or promoted “Registered Reports” via twitter and at conferences in the past few years. You know who you are. It’s all really super “open”, “transparent”, and an “improvement”. Keep up the good work!)

        *Takes off “all is lost/skeptical/disillusion” -hat*

  1. Its interesting to look up what must be done to get tobacco smoke to moderately increase the rate of lung cancer in animals.

    It took over 50 years to figure out a way, it wasn’t until 2005 that anyone accomplished it. They basically need to start exposing the animal within hours of birth, do it throughout their entire youth until adulthood, and then stop the exposure (simulating quitting smoking) and wait just as long as the exposure period. Other carcinogens, eg radioactive dust, don’t require this type of scheme.

    • It’s known that humans can smoke for decades without dying of cancer. yet, those who do routinely die much younger than those who don’t, age say mid 60’s instead of 80’s. Mice don’t live for decades, a few years at best. They’re just not the right animal to study long term exposure in.

      • As far as I know the mouse model is the only one that seems to work at all: https://www.ncbi.nlm.nih.gov/pubmed/29370344

        Also, I don’t think years of life *necessarily* matters. What directly matters is the number of tissue stem cell divisions, genetic errors (mutations/chromosomal instability) per division, sensitivity of the cells to the presence of errors (eg, ease of triggering apoptosis), and the effectiveness of immune surveillance for mutant cells.

        Eg, if cancer is caused by accumulated errors (the Armitage-Doll model[1]), then the cumulative probability a given lineage becomes cancerous would be something like: Prob(tumorigensis|nDivisions >= d) = k*(1 – (1 – p)^d)^n

        k = proportion of cancerous cells that become noticed
        p = probability of a contributing error per division
        d = number of divisions since zygote (healthy cell)
        n = number of errors that must accumulate per cell

        There is plenty of room for interspecies differences there to allow for similar effects, but yea its also wrong to just assume things would work out that way.

        [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2007940/

        • Ultimately your point is that dimensionless ratios are the ones that matter, not actual years or etc. I agree with that. Someone should do a dimensional analysis and figure out how to try to match the appropriate dimensionless ratios. Probably what you’d find is that to make the carcinogen reaction rates to cell division rate sufficiently high so that the mice have similar risks to humans you’d have to have them breath 100% pure cigarette smoke at 3atm pressure, and unfortunately you can’t then keep them alive because the O2 delivery rate compared to minimal O2 consumption rate required for life is not high enough.

          Something like that analysis should be done though, it’d bring cancer science into line with 1850’s era fluid mechanics…

        • Probably what you’d find is that to make the carcinogen reaction rates to cell division rate sufficiently high so that the mice have similar risks to humans you’d have to have them breath 100% pure cigarette smoke at 3atm pressure

          This issue isn’t found for other carcinogens though. Eg, here is a review from 1986 that mentions rats, mice, hamsters, dogs, and primates:

          Neoplasia in the lung as a direct consequence of inhalation or instillation of radioactive materials is easily demonstrated in animals

          https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1474274/

          Compare to:

          In spite of the outstanding role of tobacco smoking in human carcinogenesis, it is difficult to reproduce its effects in experimental animals.

          https://www.ncbi.nlm.nih.gov/pubmed/29370344

        • I imagine that the effect of radioactivity is to directly damage the DNA of cells by having high energy electrons smash into the DNA or things like that, whereas the effect of cigarette smoke is mediated through thousands of downstream chemical reactions which have a wide variety of reaction rates, and may in fact have reaction rates that vary as a function of things like vitamin C levels or nicotinic acid levels (Niacin for NAD synthesis) or glutathione levels in the tissues or SNP mutations in glutathione S transferase that lead to reduced effectiveness or… blablabla

          So, a plausible mechanism of damage from cigarette smoke can depend easily on things like frequency and dosage of smoke inhalation, dietary intake, whether there are periods of time smoke-free when circulating chemicals can be replenished, etc etc etc.

          You’ve frequently commented on how poorly people think about these issues. How they don’t even make any attempt at all to imagine a variety of plausible mechanisms and create equations that could give relatively specific quantitative predictions, and etc. If you and I sat down and came up with a model like the kind of thing I’m talking about here, it’d reveal a lot about what factors matter (conditional on the model being true). If we came up with 4 or 5 plausible models, and used just the existing data you’re mentioning here we could probably eliminate several of them right off the bat, and design a set of experiments that probed the differences between the other two. I suspect we could probably get somewhere useful to guide research within a couple of afternoons and less than $25 in coffee or tea consumption. My impression is this isn’t done because too many biologists simply don’t have any quantitative modeling skills *at all* and even if you do, getting funding from people evaluating the grants who have no skills is impossible. Poor science leads to poor science.

        • Note, I’m not arguing we could solve the problem of how cigarettes cause lung cancer in a couple afternoons, what I’m arguing is that we could think up directions for research that probe a wide variety of differences between plausible models to help direct new theory in a few afternoons.

          Without decent quantitative theory what we’re left with is more of the same: experiments and observational data whose results could be explained by any number of things and which had no real hope of improving our understanding, which we could have known even before carrying them out if someone had really thought hard about the range of plausible mechanisms.

        • I’d suspect since they need to have a “quit smoking” phase, the smoke is damaging the lung tissue and then upon stopping the exposure it begins to heal. This increases the number of stem cell divisions, ie increase “d” in the formula: k*(1 – (1 – p)^d)^n.

          The radiation is probably also damaging the tissue leading to a healing requirement, but in addition is increasing the mutation rate (ie p). Both effects shift the curve to left (ie tumors will form earlier than in unexposed animals). Any effect of immune surveillance, etc that removes cancerous lineages before they form a detectable tumor, will show up via “k”, this alters the height of the curve but not the timing of onset. Ie the model behaves like this:

          r = division rate (eg days/division)
          cancer_cdf = function(k = 1, d = 1:70, n = 4, p = 1e-1, r = 1){
          res = data.frame(t = d*r, pr = k*(1 – (1 – p)^d)^n)
          }

          dat = list()
          dat[[1]] = cancer_cdf(p = 1e-1, r = 1)
          dat[[2]] = cancer_cdf(p = 2e-1, r = 1)
          dat[[3]] = cancer_cdf(p = 1e-1, r = 0.5)
          dat[[4]] = cancer_cdf(p = 2e-1, r = 0.5)

          https://image.ibb.co/dJd1Fe/cancercdf.png

        • I never mentioned mortality, only lung cancer.

          Anyway, there are issues with the epidemiological data such as tuberculosis looks just like lung cancer and the doctors/morticians are not blinded to smoking status yet use that to make the diagnosis.

          Of course, I agree that mice aren’t men. If you do animal research you’ll also learn “mice aren’t rats”, “female mice aren’t male mice”, “4 week old mice aren’t 5 week old mice”, “enriched environment mice aren’t mice that never leave their cage”, etc. My point is that there is an endless list of excuses like that for any theoretical difficulty. If it was easy for tobacco smoke to cause lung cancer in mice you can bet it would be taken as evidence in favor though…

    • Do you (Anoneuiod) happen to know about heart disease, or COPD? Smoking (supposedly) increases lung cancer risk by something like 20-25x in humans, but the base rate is very low. I think it’s something like a 3x increase in heart disease and COPD, but since these have much higher base rates these are actually bigger contributors to human mortality. Any idea about the situation for rats?

  2. Fisher was a tobacco industry apologist, probably for reasons related to his eugenics: “The initial critique by Fisher, epidemiological work by Heimann, and a later survey by The Roper Organization were a part of this effort to fund research to counter the growing body of evidence in support of a causal link between smoking and a host of diseases.”

    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2911634/

  3. While I would quietly cancel my subscription to RTP if I had one, I also have concerns about the paper itself.

    (1) It sounds like an ad-hoc analysis aimed at denouncing a particular journal. If ProPublica does that, fine. But is it deserve the academic veneer? Why only look at RTP and not journals in general?

    (2) How many researcher degrees of freedom can you find in the abstact?

    (3) It would be one thing to conclude that RTP contains a biased sample of research on smoking. But the abstract further asserts that the individual research papers that it publishes are untrustworthy. I do not see how their methodology could justify that.

    Suppose the RTP had published a balanced portfolio of papers, simply by throwing in extra papers with negative conclusions and no industy ties. Should you have more confidence in the conclusions of the individual papers?

    The whole reduction of the question to “industry funded” and “pro/con” seems to reject a scientific approach to knowledge in favor of a political one.

Leave a Reply to Daniel Lakeland Cancel reply

Your email address will not be published. Required fields are marked *