Selection bias, or, some things are better off left unsaid

Posted on by

I got two of these in the same day!

1. A colleague emails me that a colleague emailed him regarding a study on women in the workplace. The headline conclusion is: “Corporate America is not on a path to gender equality.” My colleague’s colleague writes:

This coincides with my prior beliefs, but for exactly that reason I thought it important to dig into the evidence and see what the latest numbers show. . . .

He then went into all sorts of problems he had with the report, and then summarized:

I am just curious why the report chooses to focus on “we aren’t there yet, and therefore are not on the path” when in fact it looks from the data like we are *exactly* on the path.

I *do* have a lot of sympathy for the report. I think the recommendations at the end are sensible, and that many of the statements about forms of cultural bias and inequality in the home, etc., are correct. But these are based on other data. This study claims to present novel conclusions that are not supported by the data presented.

My colleague sent me the above with the note that it might be of interest.

I replied: The last thing I want to do, as a man, is to say that progress in gender equality is just fine—so I think I’ll stay out of this one!

2. Someone else points me to a news article entitled, “Height May Be Linked to Increased Cancer Risk, Study Contends,” and asks, “What do we do with something so absurd?”

I replied: Too difficult for me to understand . . . I don’t want to touch this one!

P.S. The real selection bias, though, comes when I don’t even tell you I’m not writing about a topic!

33 thoughts on “Selection bias, or, some things are better off left unsaid”

  1. Ha. It’s a McKinsey study! Lots of colorful, cool-looking bad graphs ahead, I expect.

    Also, my priors make me suspect that the recommendations were written before the data analysis.

    Reply ↓

      • I love how the stock images used throughout the Report are so carefully chosen to convey diversity along all dimensions: ethnic / race / sex / age / marital-status / blue collar / white collar.

        Painstakingly curated naturalness. Some poor junior analyst must have spent days trawling Getty Images.

        Reply ↓

  2. >Someone else points me to a news article entitled, “Height May Be Linked to Increased Cancer Risk, Study Contends,” and asks, “What do we do with something so absurd?”

    Cancer is linked to number of cell divisions so anything linked to number of cell divisions will also be linked to cancer. Obviously the bigger you are the more cells you require. What is absurd about that?

    Reply ↓

    • If genotype causes height variations why might it not also cause mutation-probablity variation?

      And given natural selection why might tall person cells be less likely per cell to turn cancerous?

      One can speculate many different ways…

      Reply ↓

    • Cancer is also linked to many causative environmental agents. Other things being equal, a smaller person can have a higher concentration of carcinogens. I.e. they would drink similar amounts of water, so a smaller person could be expected to have an increased risk from water born carcinogens.

      Reply ↓

    • Anon:

      Just to be clear, I’m not saying that I think the link between height and cancer is absurd, I’m just saying that this is a topic I know so little about, that I don’t think it makes sense for me to get involved in it.

      Reply ↓

      • Ok, but what is the excuse of the cancer researchers being interviewed? For at least 50 years the hard core of the cancer research programme has been: as cells divide they accumulate mutations, which leads to cancer.

        It boggles the mind to consider how much time/money has been spent based on this premise, yet it remains controversial/mysterious/surprising (to people getting paid to study cancer) that more cell division correlates with more cancer. There is no reason to invoke “early life exposures” or anything else unless the role of cell division has been somehow ruled out or accounted for.

        Reply ↓

      • I’m a little over 6’4″ and I had non-Hodgkins Lymphoma in 1996.

        N=1, of course. But this commonly reported correlation seems plausible to me.

        Here’s my suggestion for an easy follow-up study: Some people are tall because they have long legs (e.g., basketball player Michael Jordan). Other people are tall because they have a long torso (e.g., swimmer Michael Phelps, the human surfboard). (I’m built more like Phelps than Jordan.)

        My theory is that because almost nobody gets Leg Cancer, cancer rates would be higher, all else being equal, in people with longer torsos (Phelps) than legs (Jordan).

        This seems pretty checkable.

        Reply ↓

  3. Height being linked to cancer doesn’t seem all that absurd to me, if that’s what your correspondent was saying. Given a constant probability for each cell to develop a cancer-causing mutation, all things being equal having more cells *should* yield a higher chance for cancer. And taller people would tend to have more cells than shorter people.

    You could then ask why whales and elephants aren’t constantly getting cancer, since they have so many more cells than humans. I would guess that those animals have evolved more stringent anti-cancer mechanisms precisely because they’re so large, and that humans haven’t evolved the same mechanisms because they come at a cost and it’s not worth it for them, given their smaller number of cells. But that’s a between-species comparison. Within a given species, unless the body has some way of counting of the number of cells in the body and adjusting its anti-cancer mechanisms accordingly, you would naively expect that more cells = more cancer.

    Things this theory would predict: among different species, no strong correlation between cancer risk and number of cells, but a correlation between number of cells and amount/type/stringency of anti-cancer mechanisms. Within a given species, correlation between cancer risk and size/number of cells.

    Although actually, now that I think of it there could be pleiotrophic genes that (for example) code for both height and more stringent anti-cancer mechanisms at the same time. That would effectively be a way for the body to “count” the number of cells it has. Plus the whole thing is complicated by the fact that in the case of humans, selection was operating on a population that had a much shorter lifespan than modern humans do, which meant they had a much lower overall risk of getting cancer, which would alter the payoffs when trading off cancer risk against other things.

    Bottom line: no idea if it’s true, and I’m not an evolutionary biologist, but it at least seems superficially plausible.

    Reply ↓

    • Paul Ewald’s theory is that most cancer is caused by infections, like with HPV and ovarian cancer. That would explain the lack of a correlation between number of cells and cancer risk (I’ve heard ferrets are extremely cancer prone). There are also cases where the cancer is infectious itself, as is the case with Tasmanian Devils who keep biting each other’s faces.

      Reply ↓

    • “Plus the whole thing is complicated by the fact that in the case of humans, selection was operating on a population that had a much shorter lifespan than modern humans do, which meant they had a much lower overall risk of getting cancer, which would alter the payoffs when trading off cancer risk against other things.”

      So, I’m no expert in evolutionary biology, but I’m going to play one on the internet and take issue with this point. ;)

      Actually, going by this graph of cancer incident by age:
      http://www.cancerresearchuk.org/health-professional/cancer-statistics/incidence/age#heading-Zero

      Cancer overwhelmingly affects people *after* prime child-bearing years. So even if prior populations had the same average life expectancy of modern humans, I wouldn’t expect it to have a strong evolutionary pressure.

      Furthermore, keep in mind that the dramatically lower life expectancy from birth that we see in prior populations is driven heavily by high infant mortality. Those who survived to puberty had lifespans much closer to modern humans, and well into prime cancer forming years.
      https://en.wikipedia.org/wiki/Life_expectancy#Variation_over_time

      Reply ↓

    • ‘Given a constant probability for each cell to develop a cancer-causing mutation, all things being equal having more cells *should* yield a higher chance for cancer.’

      Cancer rates are certainly not constant across tissue types. And it’s not clear that tissues that are related to differences in height are those likely to be involved in cancer. We don’t even know that every type of cancer starts with “a mutation”. Some types of cancer have very strong links to certain genes, while others defy such an easy characterization.

      Certainly the write-up seems to be playing a game with numbers.
      “For every 4 additional inches of adult height, the study found that cancer risk was linked to an 18 percent increased cancer risk in women and 11 percent in men.”

      I’m going to have to guess that when they say “18 percent increased cancer risk” then mean a percentage increase in the _percentage_ itself (e.g., from 1% to 1.18%, not from 1% to 19%). (Aside: I loathe this usage of “percent increase” when referring to percentages.)

      They claim they have height ranges from 3’3″ to 7′. A plain language interpretation of what they say would mean that a man 7′ tall would be 66% more likely to get cancer than one who is 5′ tall. That probably means r2 = 1.66*r1, not r2 = r1 + 0.66.
      And the changes for women are even more dramatic.

      It’s entirely possible that genes related to cancer are also related to other functions, so some phenotypes are more likely to be at a high risk for cancer than others. For example, if blonde hair correlates with height, and it also correlates with pale skin, which correlates with skin cancer, then we may see a correlation between height and cancer which has nothing to do with height at all.

      I’ll have to see if I can track down the paper. At the very least I’m impressed by the sheer volume of the data set. They claim to have reviewed information on 5.5 million people.

      One caveat I would make is that results gained from studying Swedes may not generalize to the rest of us.

      Reply ↓

  5. From the article on the claimed cancer-height link: “Findings presented at meetings are typically viewed as preliminary until they’ve been published in a peer-reviewed journal.”

    That’s… unusually responsible for scientific news reporting.

    Reply ↓

    • Kevin:

      Yes, but the news article also had this bizarre quote:

      “But Gapstur, who is vice president of epidemiology at the society, cautioned that these findings only show an association between height and cancer risk. They do not prove that being tall causes cancer.”

      I have no idea what this means, as it seems to imply that it might be possible to prove that being tall causes cancer. Huh?

      Reply ↓

  6. I heard a “More or Less” broadcast on this study last fall. It can be read or listened to here:
    http://www.bbc.com/news/magazine-34515952

    But there’s some good news also:
    “But Tim Cole, a professor of medical statistics at University College London, says there’s no need for tall people to worry – especially as increased height normally means you’re less likely to get other conditions, such as heart disease.”

    Reply ↓

  8. Sounds like what Tyler Cowen calls “mood affiliation”. If you want to indicate you take an issue seriously, you’re not supposed to seem happy about the current path of progress.

    Reply ↓

  9. The height/cancer comment illustrates a worrisome risk associated with the whack-a-mole approach. If we spend too much time pointing out silly research, many careless readers will come away with the impression that all the research they hear about is silly. This naive, knee-jerk skepticism is no better than the other extreme.

    Reply ↓

  10. Some of the comments suggest that, duh, more cells dividing means more cancer, so what’s the deal. But being taller does not imply more cells. Human height is determined almost entirely by skeletal growth. Skeletal growth is a process in which intercellular matrix (sort of like collagen) is laid down and then calcifies: it does not involve making more cells. Similarly, obesity acquired in adulthood does not mean you have more fat cells: it means those cells are larger because they are engorged with more lipids.

    While some tissues are highly cellular, many are mostly intercellular material with cells dispersed. So there is no necessary correlation between overall body size in any dimension and number of cells.

    Reply ↓

  11. C’mon people, think out of the box. The results are clearly caused by gamma radiation — tall people simply have less atmosphere between themselves and outer space. We can do a control experiment where we compare the cancer rate between women who do and don’t wear high heels. That should confirm the phenomenon. Any funders out there?

    Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *