## No no no no no on “The oldest human lived to 122. Why no person will likely break her record.”

I came across this news article by Brian Resnick entitled:

The oldest human lived to 122. Why no person will likely break her record. Even with better medicine, living past 120 years will be extremely unlikely.

I was skeptical, and I really didn’t buy it after reading the research article, “Evidence for a limit to human lifespan,” by Xiao Dong, Brandon Milholland and Jan Vijg, that appeared in Nature.

As I wrote in an email to Resnick: “No no no no no on ‘The oldest human lived to 122. Why no person will likely break her record.'”

So much of it seems ridiculous to me.

The news article says, “In all, they determined the probability that someone will reach age 125 in any given year ‘is less than 1 in 10,000.’ Or put another way: A 125-year-old human is a once-in-10,000-year occurrence.”

But the headline refers to someone living to 122 or 123, not to 125. And that’s already happened once, right?

tl;dr: If someone has a mathematical model claiming that something that actually did happen, is extremely unlikely to happen, this to me is evidence that the model is flawed. I can see how Nature—which is a bit of a “tabloid”—would publish such a thing, but I was unhappy to see a neutral journalist falling for this. I recommend a bit of skepticism.

The news article concludes: “Calment, meanwhile, should rest easy in her grave that her record will be around for a long, long time.”

I wouldn’t be so sure.

I clicked through, and the paper has various weird things. For example, they report that maximum reported age of death has been decreasing in recent years, but if you look carefully these estimates have huge uncertainties (that’s what it means when they say P=0.27 and P=0.70). Their curves look pretty but are basically overfitting; that is, they’re correct when they write that one “could explain these results simply as fluctuations.” They write, “we modelled the MRAD as a Poisson distribution; we found that the probability of an MRAD exceeding 125 in any given year is less than 1 in 10,000.” But there’s no reason at all that this model should make sense at all.

To summarize: There’s nothing wrong with them rooting around in the data and looking for patterns; we can learn a lot that way. But it’s a mistake to present such speculations as anything more than speculation. I don’t think statements such as “In fact, the human race is not very likely to break that record, ever,” are doing anyone any favors.

To put it another way: If you saw such extreme claims from a political advocacy group, you’d be skeptical, right? I recommend the same skepticism when you see something in a scientific publication. Please please please don’t think that, just cos something’s published in Nature, that this is a guarantee that it’s sound science. You really have to look carefully at the paper. And this one isn’t so hard to look at; they’re not doing anything really technical here.

I also sent this to science journalist Ed Yong, who was quoted in the news article. Yong replied:

So Vijg was clear to me in our interview that the change after the mid-90s shouldn’t be seen as a decrease since it’s non-significant. He billed it as a plateau; it’s more that the significant increase before that point no longer continues.

I did ask him about things like outliers and the choice of 1995 as a breakpoint. He said that the results are the same even if you take out Calment as the most obvious outlier, and whichever year you pick as the breakpoint. From him:

There simply is no significant increase from the early 1990s onwards. I am sure that some people will argue that the upward trend may continue soon enough. While we agree that the data are noisy, which is to be expected, the statistics are clear. Fortunately, all databases are public so everyone who wishes can do the math and disagree with us.

To which I responded:

Let me put it this way, then: My problem is in going from “A linear regression with a small number of data points has a trend coefficient which, when fit to the past twenty years, is not statistically significantly different from zero” to “they determined the probability that someone will reach age 125 in any given year ‘is less than 1 in 10,000′” and “In fact, the human race is not very likely to break that record, ever.”

Also, I think it’s a bit strange for them to say both “the data are noisy” and “the statistics are clear.” Their Poisson distribution seems to come out of nowhere.

There was also this quote from Vijg: “When Calment died at 122, everyone said it’ll only be a matter of time before we have someone who’s 125 or 130.”

That also seems a bit misleading in that there’s a big difference between 122 and 125, and a really big difference between 125 and 130! Each year becomes harder to achieve (at least, until there’s some medical breakthrough).

From a news perspective, this is not serious science, it’s just a fun feature story. I think Vijg is misunderstanding the difference between interpolation and extrapolation, but, hey, that’s how he got published in Nature!

1. John Hall says:

Sounds like another case of people not understanding extreme value theory.

• Statsgirl says:

Some of these are not so convincing. One of them involves moving around data points (!), but doesn’t even succeed in coming to a substantially different conclusion. Another is just an analogy with sports that is not very informative.

• gatesofforkingpaths says:

setting aside methods, what was unconvincing about this: “Even disregarding the serious problem of a wide-ranging claim hinging on just one observation, it is curious that the fact that this remarkable woman lived to the age of 122 should be such a crucial part of the argument that maximal human lifespan has plateaued at 115.”

The original article’s conclusion is logically inconsistent with one of its main premises.

• Statsgirl says:

The original paper never said living past 115 was impossible, it said that the oldest presonance would be around 115 (give or take) with no upward trend. That sentence you quoted is a serious misrepresentation of the article and I’m surprised a critique so off-base got past the editors.

• gatesofforkingpaths says:

I mean, we’re talking about Nature here. Lots of crap has gotten past the editors there.

• Nick Brown says:

I am the author of that sentence… I stand by it. If you look at our reanalysis (and that of the other authors), the trend in the original article that “proved” the plateau was driven entirely by the presence of Calment. If she had been killed in WW2 or something, no trend would have been discernable.

The whole premise of Dong et al.’s “proof” is silly because the authors made the completely arbitrary decision to consider “the oldest person to die in any given year”. First, it’s not clear why one would choose the oldest person to die, rather than the oldest person alive at that time, if one is interested in studying longevity. Second, why only one person year? Why should the rate at which the earth orbits the sun determine our sampling strategy? Human longevity is properly measured in days (as a shorthand, ultimately, for seconds or some even smaller unit); the calendar year, as a unit of administration and organisation of data collection, has nothing to do with it. This might not matter with large sample sizes, but the result of the authors’ choice (constrained, I suspect, by the data at hand) was to introduce a great deal of scope for random variation.

• Nick Brown says:

PS: Normally when I comment here I sign myself just “Nick”; the other comments in this discussion from “Nick” are not by me. :-)

• Nick S says:

Sorry, I commit to avoiding ambiguity in future :-)

• Statsgirl says:

Nick, I have read your work on Wansink, so I know you to be a person who values precision and has a low tolerance for nonsense. So that sentence and your defense of it here are uncharacteristically sloppy. First, you have not addressed the fact that you conflated the 115 year average with the 125 year limit. Second, your assertion that the whole paper relies on one data point is false. Third, you use the words “prove” and “proof” in quotation marks, implying they are quotes from the original paper, but they do not appear in the paper at all, except when they explicitly say their resultry suggest but “do not prove” there is a limit to lifespan! Your use of punctuation makes the situation seem the opposite of what it is. Finally, the whole a-year-is-a-rotation-around-the-sun-and-therefore-arbitrary line of reasoning is a bit first-year philosophy class and not very convincing (or should we throw out all time-based results?). I really hope that you have not been so eager to issue a high-profile attack on a paper that you based everything on a misunderstanding, misconstrual or motivated analysis of the paper and the data.

• Nick Brown says:

OK, “proof” was sloppy. But I stand by my criticism of the authors’ choice of data points. These people are outliers among outliers; to attempt to extract meaning from 30 or so of them is just modelling noise.

Our alternative model (in which we imagined that Calment had been born 7 years later) of course shows nothing in itself, other than how arbitrary and subject to small random variations the authors’ model is. As a reviewer of our comment wrote: “[Brown et al.] … show that the results in the original paper are based on several arbitrary decisions, hence the original findings could be uncertain, and they provide quantitative estimates of the uncertainties. The responses [i.e., Dong et al.’s reply to us] are along the lines of “our arbitrary decisions are better than yours””.

• Statsgirl says:

In your alternative model, the R^2 goes from 0.46 before the breakpoint (pretty decent correlation) to 0.03 after (basically no correlation). And the p-value goes from 0.00007 to 0.579 (a bigger increase than in the original data). You could have changed the data any way you liked, but you basically got the same result as the original paper.

• Statsgirl says:

I actually read through those. The first link says the GRG data supports the authors model. The second one says “the trimmed maximum (90th percentile) seems to be constant” which is not inconsistent with the paper. I see disagreements about minor details here, not evidence for “very screwed up”.

• amoeba says:

The point of these rebuttals is that Figure 2 of the original paper is a joke. The analysis there is wrong in so many *different* ways that it is actually hard to conceive.

But it might of course be nevertheless true that max life span is not increasing. That’s another issue.

If you read through my analysis on https://github.com/adamlenart/random/tree/master/supercentenarian , you could have seen that I argued that the observed maximum age at death is a function of sample size, and registered supercentenarian sample sizes, in the GRG data, have been leveling off since 1995. Not necessarily because there are fewer supercentenarians around but rather because of data collections issues.

2. Statsgirl says:

Re your tldr, the original article in Nature never said that living to 122 (which has happened) was unlikely, it said that living to 125 (which hasn’t happened) is unlikely. And only 1 person out of the 100 billion or so overt the years has reached 122, so I don’t think it’s inherently unreasonable to say that is also unlikely.

• Nick says:

But you can’t just take 100 billion as the denominator. Becoming a supercentenarian (not to mention having the birth records to prove it) is much likelier to happen with access to modern medicine, and the overwhelming majority of people who have ever lived did not have this privilege. All of the oldest verified living people ever as of 2017 were born between 1870-1900 (and almost all were born in Europe, North America or Japan). Around 40 million annual global births (in those years) times 30 years = ~1.2 billion. So Jeanne Calment living to 122 is a one in a billion event, not a one in a 100 billion event. And even that’s an overestimate, as I’ve included all births across the globe.

• Statsgirl says:

You can play with the numbers a bit, sure, but I still think a 1 in a billion or even 1 in hundreds of millions event counts as extremely unlikely.

• Nick says:

But, of course, by definition becoming one of the “the very oldest people ever” (out of billions of humans) is unlikely. That would stay the same even if 10 years from now someone has lived to 125.

• Andrew says:

Statsgirl:

Yes, 1 in a billion is extremely unlikely, and if someone were to write a paper saying it’s extremely unlikely that a randomly selected person will live past the age of 125, I’d have no problem. But that’s not what was claimed. They were claiming it was extremely unlikely that the maximum age of everyone in the world will exceed 125. There are over 7 billion people in the world today, so we’d expect a few one-in-a-billion things to happen to them.

• Statsgirl says:

Yes, but this discussion started about 122, not 125. Living to 122 may be a 1 in a billion event, but that doesn’t mean living to 125 is likely to happen. Assuming mortality after age 122 is 90%, living to 125 is a 1 in a trillion event, so not likely to happen to anyone in the planet for quite some time. Now, maybe you can question some of these assumptions, but that doesn’t make them obviously unreasonable.

• Nick says:

But if Jeanne Calment had died (instead of just being badly sickened) from eating contaminated cherries in 1942, people would be saying “just because Sarah Knauss lived to 119, that doesn’t mean living to 122 is likely to happen, it could be a 1 trillion to 1 event, Knauss was an outlier, no one else has lived past 117, etc, etc.”

There are ~500,000 living centarians in the world (a far larger number than ever before). If getting to 125yo is a trillion to 1 event, then the odds that one of these living centenarians will do so is 1 in 10^12/(5*10^5)=1 in 2 million.

Does Statsgirl or anybody else want to make a long bet with me (at 2 million to one odds) that, by November 2042, the oldest verified human ever will be less than 125? I might be willing to drop a few zeroes from the 2,000,000…

• Nick says:

Personally I think that the odds that the first person to live to 125 has already celebrated her 100th birthday are around even.

3. “If someone has a mathematical model claiming that something that actually did happen, is extremely unlikely to happen, this to me is evidence that the model is flawed.”

The fact that the current oldest person living (at least according to official records) is 5 years younger than Calment was when she died implies that no one will break the record at least until 2022, and given the numbers of very old people, it is likely to be even later. So a model suggesting that Calment was a pretty extreme outlier does not seem particularly unlikely.

• Statsgirl says:

+1

• Andrew says:

Entirely:

If the research article, and the news article, had just suggested that Calment was a pretty extreme outlier by historical standards, I’d have no problem. The problem is when they go further and say things like “the probability of an MRAD [maximum reported age at death] exceeding 125 in any given year is less than 1 in 10,000.” 10,000 is a big number, and, from a Bayesian perspective, I think the best interpretation of such a result is as a model check: If your model comes up with such an outlandish prediction, that’s a sign that you should look more carefully at your model and see what went wrong. In this case, there were a lot of iffy things about the model.

• jrc says:

Andrew,

Yeah, especially since, if EntirelyUseless is right, the probability of someone exceeding 125 next year is 0. And the year after that. And the year after that.

You think the author’s would take a 100,000 to 1 bet next year? I bet a bank would lend me some money on that.

I love when other people are also terrible at probability counting. Makes me feel less bad when I see one of those “red balls in an urn” problems.

4. Nilay Argon says:

Holger Rootzen (http://www.math.chalmers.se/~rootzen/) recently gave a talk at our department on the very same topic. I am not sure if his work is published yet but he talked about this Nature article in detail and how misleading it was. Here is a link to the abstract of his talk: http://stat-or.unc.edu/event/stor-colloquium-holger-rootzen-chalmers-university-of-technology

• Statsgirl says:

Judging by the abstract, he should agree with the paper. In the abstract he says it’s unlikely anyone will live past 128, which is not far ofrom the estimate in the other paper.

Occasionally I think that Nature and Science publish certain articles with the intention of “hahaha, you guys all gotta see what someone actually submitted!”

I would say that the odds of a human living to the age of 125 is about the same as the odds of someone publishing a scientific paper with 50 authors !

7. Jerzy says:

“If someone has a mathematical model claiming that something that actually did happen, is extremely unlikely to happen, this to me is evidence that the model is flawed.”

Sounds like a p-value to me!

8. Elin says:

It’s like 100 year floods and 100 year hurricanes. What was originally a concept useful for decision making for construction projects and similar items (i.e. the windows in this building must be built for 100 year hurricane, buildings along wet areas must be elevated to above the 100 year flood line) had people thinking that having Katrina and Sandy should mean that it would be unlikely that there would be another sever hurricane in the near future. Many, many points of failure in that thinking.

9. Marcel van Assen says:

Ha ha ha, I had a great laugh in the train.

“it’ll only be a matter of time before we have someone who’s 125 or 130.”
Yeah, a few years, right?
Ha ha ha.

10. Thomas Lumley says:

This, from the Chronicle of Higher Education (http://www.chronicle.com/article/A-New-Theory-on-How/240470), seems relevant to the discussion:

‘Mr. Vijg said repeatedly that his Nature paper made no “definitive statement” about a maximum human age and that he felt “amazement” that anyone might think otherwise. But he acknowledged approving a news release about his study issued by Albert Einstein College with the headline: “Maximum human lifespan has already been reached, Einstein researchers conclude.”‘

• Nick Brown says:

Yeah, the media coverage was a pretty big motivation (for me, at least; my co-authors can speak for themselves) in writing our reply to this. We see, time and again, how researchers make out (via the thinly-veiled proxy of a press release or a friendly journalist who is eager for a cute story) that their research means X, while hiding behind “Oh, of course, I never said X, I would never make claims that were not justified by the available empirical evidence” when talking to their academic colleagues. Quite a few stories of this type have been featured in this blog over the past couple of years.

I also note this comment, from Hannah Devlin’s piece in the Guardian on 2017-06-28: “Vijg is equally strident, implying that his critics are, to some extent, simply upset at being confronted with their own mortality”. I note that he presented no evidence for this claim, which, if correctly reported, seems to be somewhat ad hominem, even if the target is not identified by name.

• Andrew says:

Pointeroutguy:

I followed the first link, where Xiao Dong, Brandon Milholland, and Jan Vijg write:

And far from being fame-seekers, we are simply scientists doing our best to come to a better understanding of the world. Indeed, the furor sparked by our paper has been at times an unwelcome distraction from our research, and we have been disappointed to see careful consideration of our nuanced findings eschewed in favor of baseless speculation about our personal motives.

Fine. I agree that what’s important is the work, not the personal motives. Even if their personal motives are to seek fame, so what? If the work is good, who cares?

Dong, Milholland, and Vijg also write:

we reported in our paper the odds of someone living past age 125, which we calculated to be less than 1 in 10,000. This number was also reported in the press release, and we were careful to emphasize in interviews with journalists that the limit we were describing was not one of impossibility, just extreme improbability. A journalist could stop just at the title of the press release, or could pore over our many statements to cherry-pick one that we neglected to nuance, but that would say more about the quality of reporting than the science itself.

But there’s also this:

“From now on, this is it,” one of the three authors, Jan Vijg, a professor of genetics at Albert Einstein, told The New York Times one of several major news outlets that helped promote the sobering news. “Humans will never get older than 115.”

My problem is not with fame-seeking; my problem is with researchers who make extreme claims not supported by data, and who then go into a defensive crouch and can’t handle legitimate criticism.

• Nick Brown says:

But if you point that out, you are creating a chilling atmosphere of personal attacks and bullying.

• Pointeroutguy says:

I think they were trying to point out that they might have made a few statements that weren’t entirely correct, but those weren’t representative of the overall content of all their statements. Think of the dumbest thing you’ve ever said. You probably wouldn’t likeit if people kept holding it against you and ignoring everything else you said. This CHE article seemed overly eager to paint academics in a bad light.

11. A.V. says:

> If someone has a mathematical model claiming that something that actually did happen, is extremely unlikely to happen, this to me is evidence that the model is flawed.

How do you apply this to all the stuff like “it’s extremely unlikely a priori that you’d flip coins HHHTHHTHHHHHTHHHHTTTTTTHHHTHTTTTTHTTTTTTTHTHTHTHTHTHTHTHTHTHHHHTHTT” but you did? Or “it’s extremely unlikely a priori that the world would exist and have life on it”?

• MCN says:

Interesting counter-examples! But each is different from the case at hand in a critical way. In the first, every possible string of H’s and T’s is equally extremely unlikely, so there’s a 100% chance of obtaining an unlikely outcome. So unless your theory is capable of predicting long strings of H’s and T’s for a fair coin in advance of the coin being flipped (a revolutionary theory!), the “unlikely” outcome was always certain. In the second example, we can only ask the probability of our existence if we already exist, so the probability that we exist *conditioned on* the fact that we’re asking about it is 100% and therefore also not unlikely at all.

12. If you want to check with someone who has actually spent a considerable portion of his academic career studying this question, look up John Wilmoth, currently Director of the UN Population Division.