Numbers too good to be true? Or: Thanks, Obama!?

This post is by Phil.

Flat until April 2010; steady drop until October 2012; flat since then.

Readmissions to hospital within 30 days, by quarter

The “Affordable Care Act” a.k.a. “Obamacare” was passed in 2010, with its various pieces coming into play over the following few years. One of those pieces is penalties for hospitals that see high readmission rates. The theory here, or at least one of the theories here, was that hospitals could reduce readmission rates if they wanted to, but they didn’t have a strong incentive to do so, and indeed there was a moral hazard: if a hospital sends a patient home for good, they’re done collecting money from them, but if the patient has to come back for more treatment…cha-ching.

I have to admit I didn’t think this was going to be a big deal. I know doctors, I’ve seen doctors, some of my good friends are doctors, and I know they’re not scheming to make more money by providing bad treatment so the patients have to come back for more.

But…well, check out this plot, from the Department of Health and Human Services. The plot does us all a disservice by not starting its y-axis at 0, but still…wow. If the data are real and the plot is real, this is pretty stunning: a 20% reduction (or a 3.5 percentage point reduction) in readmissions for “HRRP”, and a similar scale reduction in all other readmissions.

My first thought was that the hospitals are gaming the system somehow by readmitting patients but not reporting them, but I’m not the first person to suggest this and supposedly “The new research shows that this isn’t the case. The number of observation stays are very small compared to readmissions and have increased steadily since at least 2008, with no acceleration after the Affordable Care Act was enacted.”

Of course, there are other possibilities, like maybe the hospitals are refusing to readmit patients even if they really should, or maybe they put them off a bit and readmit them on day 31 instead of day 28 or 29 or 30. But something like this would make their fatality numbers go up, and I assume someone tracks those.

One of the interesting things about this is that you really don’t need a statistician: the signal is so clear that the only questions are related to the definitions of things like “readmission.” A sixth-grader can look at the numbers and come up with a good estimate of the effect of the law on readmissions.

82 thoughts on “Numbers too good to be true? Or: Thanks, Obama!?

  1. I have to admit I didn’t think this was going to be a big deal. I know doctors, I’ve seen doctors, some of my good friends are doctors, and I know they’re not scheming to make more money by providing bad treatment so the patients have to come back for more.

    Gee. Rational choice theory would predict that if you reward someone for something, that behavior will increase (as will economic theory, operant conditioning, etc.). You’re showing your ivory tower biases here I’m afraid. I’ve unfortunately had to deal with academic statisticians over the last few years (including some of your erstwhile colleagues at Berkeley) and what they will come up with is amazing–complicated theories to get published in JASA or Annals of Statistics of whatever when a simpler approach works better and is simpler to understand. I’m sure (in their mind) they aren’t scheming to “make more money” (or get a grant or whatever) but they are (and if you point out problems, they blow you off with “I hear you”). People respond to incentives. Period.

    • Sure. but the satisfaction of helping people is also an incentive. Or, if you prefer, people respond to monetary incentives but they’re not the only things people respond to. Many people give a substantial amount of time and money to charitable causes, for example.

      Perhaps one important effect is that some people are more sensitive to monetary incentives than others, and the ones who care a lot about those things naturally gravitate towards positions where they come into play. This reminds me of Marwell and Ames’s classic paper, “Economists free-ride, does anyone else?”

      There have been, and are, plenty of cases of people either not responding to incentives or not responding as expected. You can look it up. As with many things, it can be obvious after the fact that something should be the expected outcome, but it’s not always obvious before you have the data. There were plenty of conservative economists who thought this kind of Obamacare savings would be small…or at least they _said_ they thought it would be small. Which, I suppose, is another way to look at it: I thought there would be a small effect. You, numeric, evidently expected a big effect like the one seen. But why not twice as big? Or half as big? It’s not so easy to know how big an effect to expect. I expected something much smaller. Bully for you if you got it exactly right!

      • See “Public goods provision in an experimental environment” by Plott et.al. for a refutation of the Marwell and Ames thesis (Marwell and Ames was a one-shot experiment). Economists may be quicker on the draw but other people wise up real quick (after one try by the experiments in the Plott paper). Anyone who has eaten at a church pot luck knows that underprovision of public goods is rampant (casseroles are not, in my opinion a “good” but rather a “bad”). Economists are loathsome but then so are most social scientists.

      • As Phil says, yes, incentives work but there are many incentives in play, and the net effect of a policy change cannot always be predicted.

        Indeed, the same hard-headed cynics who tell us (correctly) that incentives matter also love to talk about unintended consequences of policies. So it’s worth looking at what actually happens when a new policy is implemented.

        In the health care realm, of course the U.S. is famous for spending so much compared to other countries and not getting much extra in return, so, yes, it should be no surprise that there are benefits to be had by changing the system. Still, changes can be unpredictable etc.

        • Everything Phil says is fine except the bit about “some of my good friends are doctors, and I know they’re not scheming to make more money”.

          That seemed a bit incongruous. It’s not how I’d calibrate my priors about this. Active scheming is not needed. Just incompetence, inertia, negligence or apathy can suffice. Or a mix of a lot of different factors.

        • Rahul:

          Also there are some doctors who scheme, even if Phil’s friends are not among that group. And a lot of schemers are also probably pretty good at coming up with stories to convince themselves why their scheming is public-spirited.

        • Andrew:

          Indeed. Often people are not even aware that they are doing something wrong or inefficient. But that does not mean that the inefficiency does not exist.

        • A lot of emphasis is going on the doctors here. My guess is that the people most responsive to the change would be hospital administrators, who have a number of levers at their disposal to reduce readmission (not just passing on the incentives to the doctors).

        • Nick:

          Yes, it seems like a bit of availability bias that we focus on doctors, because these are the people most clearly associated with health care.

        • Yes, this is a point I was going to make to “numeric” as well. Everyone agrees that incentives matter, but some people predicted they wouldn’t make much difference in hospital admissions because the people who control the care (doctors and nurses) don’t share in the benefits or penalties of the hospitals, except in an indirect and diluted way.

          I’d be interested in a deeper look at why re-admissions have gone down. Are patients being offered in-home care, or more physical therapy and better follow-up (to make sure they’re taking their meds correctly or whatever), or are they being treated differently when they are in the hospital, or are they being kept in the hospital longer the first time so they’re less likely to have to come back later, or what? Different people would presumably be making the decisions relevant to these different possibilities. Presumably somehow it tracks back to some specific people who will be compensated more if things go well and badly if they don’t. That may very well not be the doctors and nurses.

        • Interesting but very complex stuff!

          (Here in Canada some of us are hoping there will be an ongoing well funded institute to continually study this sort of thing – to help divert us from surpassing the US as the country getting the worst value for healthcare.) http://www.healthycanadians.gc.ca/publications/health-system-systeme-sante/report-healthcare-innovation-rapport-soins/index-eng.php#app1

          Drawing from my past experiences in health care research and statistics in clinical applications, numerics and Hal A comments seem least wrong.

          Doctors are big drivers of cost, they seem very responsive (in Canada) to changes in remuneration and lots of lost opportunities exist in non-glorious work that gets neglected which numeric nicely points to happening in statistics in clinical applications where simple methods are avoided as they lessen opportunities for methodological publication/awards.

          But this is not inconsistent with most have good intentions especially given implications to others well being – we are just complicated creatures – “These individuals believe that their training as scientists and their devotion to professionalism protects them from external influences that might bias their opinions. However, this view may be based on an incorrect understanding of human psychology. Conflicts of interest are problematic, not only because they are widespread but also because most people incorrectly think that succumbing to them is due to intentional corruption, a problem for only a few bad apples.” – http://jama.jamanetwork.com/article.aspx?articleid=182115

        • Here’s what Phil said.

          Sure. but the satisfaction of helping people is also an incentive.

          Altruism is not a particularly prevalent characteristic in any culture (see The Rational Peasant, for example). Even a Mother Teresa is now considered by many to be the anti-altruist. You (and Phil) have shifted the conversation away from my point, which is that those in authority (and highly paid for the most part) make decisions in self-interest first, whether they are doctors, statisticians, or prison guards for that matter. Systems get built up that are not, from an outsiders point of view (or economic point of view) particularly rational and they have associated with them those who benefit and desire to perpetuate the status quo. Doctors benefited from a system that rewarded them handsomely and they had no incentive to look under the hood (and the vast majority did not). The fact that a third of the American public was undertreated or not treated was immaterial as it did not affect their pocket book–in fact, it actually raised their income (compare doctor compensation in the US to other G7 countries).
          What this has to do with unintended consequences I don’t understand.

          I will say (since we are talking about policy) is that the best policy is one that gives maximum flexibility to reach a goal and rewards those who reach it. That’s why not paying for readmissions is a good policy–it leaves the details up to the individual health providers (insurance plans, hospitals, etc). What will inevitably happen (and has) is that readmissions will be reclassified not as readmissions (you can be in the hospital for days and not be “admitted”). So some sort of enforcement mechanism is necessary in order to keep people from gaming the system, just as markets need this.

          “People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public, or in some contrivance to raise prices.”

          I would say this is true of all professions, not just trades.

        • numeric, do YOU always make decisions that are purely in your economic self-interest? I certainly don’t, except in the interesting-to-college-freshman-philosophy-students sense that “if I choose to do something then by definition it must be in my self-interest.” Indeed it’s ridiculous to claim that people only act in their economic self-interest; exceptions are legion.

          I’m not making the much stronger claim that you might try to ascribe to me, which is that people are purely altruistic. That’s not true of anyone.

          Monetary incentives matter, but they aren’t the only thing that matter. I’m not sure if you are actually disagreeing with this?

        • Of course not, but when the economic impacts are large (say a few thousand dollars), I would say that dominates most everyone’s decisions. In particular, would doctors in the US want to be paid the average of the physician’s salary in the other G7 countries? Of course not (they’re already green with envy over the dotcom salaries–look at the percent of Harvard undergraduates that became doctors/lawyers in 1980 and look at the percent who become finance/dot com types now–they’re following the money). In particular, my best friends are doctors and they are good people so they aren’t messing things up is so hopelessly naive that it indicates someone has spent too long in academia. The point is, those who go along get along and doctors have been going along for decades (in fact, worse–Truman’s health plan died because of AMA opposition and the AMA despised Medicare (recall Reagan’s famous recording on behalf of the AMA–Krugman is always referencing it)). I would say that organized opposition to affordable health care for all people in this country by doctors has been the single most important factor in the US not having it (now with the ACA we have it to some extent). A century since Roosevelt first proposed it in 1912. Money may not be the only thing, but it drives policy in this country (Bartels work, Trump’s appeal).

        • And yet, I know someone who gave up a $115K job as an engineer to become a teacher in a low-income public school.

          But also, I just have a feeling that somewhere in a parallel universe a guy named lihp just wrote a blog post that said “I thought that having a penalty for hospital readmissions would lead to a large drop in readmissions but instead the drop has been small”, and ciremun says in the comments that that isn’t surprising because the incentives that apply to the hospital as a whole aren’t efficiently passed down to the individuals in charge of the care, and anyway if there were a way to get hospital readmissions down then big employers would have moved to insurance plans that have that effect so of course we can’t expect any improvement.

          Explaining something after the fact is a lot easier than predicting it ahead of time. Yes “people respond to incentives” but how big the effect will be, that’s not easy.

        • Out of room on the comment below but I think Phil is making the error of the exceptional individual, the uberman, the Ayn Rand acolyte, the anecdotal. It is this–and statisticians should appreciate it–the average effect of incentives is the most important thing in policy analysis. There will be variation to be sure–the engineer who gives up a lucrative job to teach in a low-income school, the person that throws down all of his goods and follows Christ (how often does that happen, really? And Christian theology says he is a God and should be obeyed). But in the aggregate, on the average, it doesn’t happen much. And this attitude can have pernicious effects, an obvious one being on the lamentable status of African-Americans in our society. Yes, there is Obama, and Oprah, and Drake, and Lebron, but read some articles by Coates (nice graphs, by the way, in his Atlantic article–see http://www.theatlantic.com/politics/archive/2016/02/why-we-write/459909/ and the persistence of spatially concentrated incarceration. The point is that as statisticians, we can see that, on average, it is harder to escape these circumstances than those that most other groups in America are born into. The exceptional will often make it out of adverse circumstances, or become a teacher in a low income school, or whatever, but these are the exceptions that prove the rule. In policy, and as statisticians, we look at the mean and the variance, not the outliers (of course we look at them but usually to confirm the model, not to make them the model).

        • I agree with Phil it’s hard to know ahead of time how big the effect size will be. And with numeric, that the average effect is what matters for policy. And I disagree with numeric that we don’t look at the tails to make them the model, clearly you don’t model floods, hurricanes, or financial crashes :-)

        • And I disagree with numeric that we don’t look at the tails to make them the model, clearly you don’t model floods, hurricanes, or financial crashes :-)

          I was referring to public policy in the traditional sense (social welfare, health care, etc), given the context we were discussing, which was individual behavior. One exceptional individual can’t stop a flood, or a hurricane, or for that matter a financial crises, but they can give up money to work at a noble profession. Also, I would argue that the public policy solutions to floods and hurricanes (exacerbated by global warming) would be changes in a small ways in a vast number of people (incentives for solar, carbon taxes, etc). Some people would have very aberrant reactions and be on the tails (an oil company executive blowing up solar installations?) but we’d look at the vast majority of reactions (x% switching to solar given a monetary incentive y). Financial crises, while a single event, are caused by a large number of actors and are once again amenable to interventions that the average is the most important (Glass-Steagall restrictions, higher margin requirements, restrictions on leverage). Out of 1000 firms, a certain percent may go bankrupt with these restrictions, which we would estimate with a standard deviation by feeding in parameters to a model (maybe based on historical experience).

    • It’s too bad that this whole thread is devoid of knowledge of what really happened in any of these hospitals. Yes, I know this is a statistics blog, but it would help to know who is making these decisions (specific people, not “hospitals”) and how they perceive the factors in those decisions, and how those decisions affect them personally (whose pocket those penalties come out of), it would be good to have a case study or two or more with real doctors and real hospital administrators and staff and maybe even real patients. Absent that, although we know what happened, we’re all just guessing about how it happened.

      • Totally agree; if I had read all of the comments first, before adding one of my own, I would have modified my comment a few lines above this one to say “I agree with Jay Livingston that…”

      • Sure. One can go through the expense of collecting all of that information. Or one can simply understand how hospitals operate and how hospital administrators operate, and how doctors operate and come to a fairly robust conclusion that it is most likely administrators who have both the incentive and the power to drive these results.

  2. Assuming this is working as the chart suggests and not being gamed, one theory for the effectiveness of the policy is that it redirects capable resources (ie people who are smart and get stuff done) to boring but high yield tasks. You can probably save more lives, reduce hospital stays and complications by being effective in getting everybody to wash their hands than improving surgery techniques. There’s not a lot of glory in being in charge of that. Well suddenly there is if whoever is in charge of the hospital has real incentive to make it a priority. There’s bound to be a bunch of unglamorous but hugely important jobs like this in any hospital that aren’t done as well as they can be because of that lack of glamour. What level of vigilance is appropriate given it’s a hassle to enforce and encourage, chewing people out for cutting corners sucks, they’re colleagues and nice people. Well now it’s a higher level of vigilance. It would be interesting to find a breakdown on the causes of re-admission, eg how much of it is infection picked while at the hospital. It would be interesting to see antibiotic usage at hospitals over the same period.

    One would hope that health policy technocrats have looked closely at all that data to find this piece of fat – again IF it really is as it appears. There’s always risks with these kinds of metrics but if you direct them at a place where there really is a big win so it’s easier and more effective to get that win rather than game the metric it can work. Now how can we know if that is what happened here?

    • Hal’s theory sounds highly plausible to me; among other things, one of the talents of “people who are smart and get things done” is to figure out how to entice people to get on board in paying attention to things they have been neglecting.

    • Which is perhaps why you see diffusion of benefits from HRPP to the admissions, if there is a benefit of the penalties, which seems tiny. If anything the penalties seemed to be associated with dramatically reducing the rate of change. This could be because the hospitals made a lot of change in anticipation of the penalties. Or it could be that the penalties started and they decided that they could live with whatever level of penalty there was. Lots of potential stories there.

  3. 1) That version of the chart is mislabeled. The penalties were implemented beginning with the first vertical line. From the paper:

    “October 2007 through March 2010 was the period before enactment of the Affordable Care Act (ACA); April 2010 through September 2012 was the period of implementation of the Hospital Readmissions Reduction Program, which set financial penalties for hospitals that had higher-than-expected readmission rates for targeted conditions; and October 2012 through May 2015 was the long-term follow-up period after penalties were initiated. Dashed lines indicate divisions between periods”
    http://www.nejm.org/doi/full/10.1056/NEJMsa1513024

    2)The decrease in that chart is from Jan-2011 to Jan-2012, not from Apr-2010 to Oct-2012. This can also be seen their Fig S2 which combines observation stays and readmissions. So we are looking for something that began in Jan-2011 and was implemented throughout that year.

    3) As Andrew notes, such a decrease could occur due to increased fatalities. Alternatively, an increase in spurious initial admissions would lead to decreased readmission for the same thing. But as noted in #2, this looks a lot like some procedural effect/artifact.

    4) In the supplement they write: “All index stays were identified using the Center’s for Medicare and Medicaid Services’ (CMS) hospital-wide readmission measure’s inclusion and exclusion criteria…We also identified and removed THA/TKA and COPD admissions.”

    Ignoring that no justification is offered for those additional exclusions, that means we should look at the CMS HWR measure and see if anything influencing it changed specifically during 2011:

    Inclusion criteria:
    Enrolled in Medicare fee-for-service (FFS)*;
    Aged 65 or over;
    Discharged from non-federal acute care hospitals;
    Without an in-hospital death;
    Not transferred to another acute care facility; and,
    Enrolled in Part A Medicare for the 12 months prior to the date of the index admission.

    Exclusion Criteria*:
    Admitted to Prospective Payment System (PPS)-exempt cancer hospitals;
    Without at least 30 days post-discharge enrollment in FFS Medicare;
    Discharged against medical advice (AMA);
    Admitted for primary psychiatric diagnoses;
    Admitted for rehabilitation; or,
    Admitted for medical treatment of cancer.

    *As a part of data processing prior to the measure calculation, records are removed for non-short-term acute care facilities such as psychiatric facilities, rehabilitation facilities, or long-term care hospitals. Additional data-cleaning steps include removing claims with stays longer than one year and with overlapping dates and records for providers with invalid provider IDs.
    http://altarum.org/sites/default/files/uploaded-publication-files/Rdmsn_Msr_Updts_HWR_0714_0.pdf

    Anyone familiar with this? Maybe it has to do with ICD-10 adoption? Perhaps rehab became a more inclusive category, or definition of “against medical advice” changed for some influential categories? Also, I wonder if hospitals make “deals” with each other to transfer patients likely to require unplanned readmission. Like trading for supplies, etc.

    • Regarding my point 1. I missed (in the second sentence…): “Since October 2012, the start of fiscal year (FY) 2013, the program has penalized hospitals with higher-than expected 30-day readmission rates for selected clinical conditions.”

      So then I wonder what “implementation of the Hospital Readmissions Reduction Program” means in the Fig 1 caption. They show data going back to 2007, so it isn’t collecting the data.

    • Looking at the article, what is sad is that they use a p value to decide that the observed increase in use of observational stays was not important.

  4. Speaking of incentives, which way are they aligned for the “Department of Health and Human Services”?

    I’m worrying more about whoever commissioned this report gaming the data than the Doctors or Hospitals gaming it.

    I’d wait for some third party analyses to vet this before we conclude anything more.

  5. I think I’m missing something. According to the chart, the readmission rate started to drop precipitously after the ACA passed but then it kind of leveled off after the HRRP penalties kicked in. Why would it level off after the penalties kick in? I’d have expected the post-ACA trend to continue or for the readmission rate to drop even further after the penalties kicked in. What am I missing?

        • The “precisely” bit is exaggerated because that’s how the piece-wise linear fit thresholds were chosen.

          But in general, if you give a sector an enforcement deadline it’s not too surprising that the cost-effective level of the aggregate result was produced roughly to coincide with the impending pre-announced deadline. Do the changes slower & you get hit by a penalty. Do them faster & you are over-allocating resources.

        • Yup. That makes sense to me. As a potential user of hospital services it bothers me that they don’t continue to try to improve until they establish that they’ve reached the floor but your explanation makes sense. They hit the deadline, decide “We’re good enough.”, stop whatever special actions they were taking to reduce the rate, and the trend returns to more or less its pre-ACA value.

        • Well the non HRPP had the same pattern so as everything else, but starts at a higher level. The real impact is the ACA, which took some period of phase in, then things stabilized with more people having insurance. The penalties don’t seem to have had much impact though who knows what a complex analysis might reveal.

        • > Well the non HRPP had the same pattern so as everything else, but starts at a higher level.

          Okay, so why the inflection at 10/2012 for other admissions as well as HRRP? My conjecture: Hospitals instituted policies upon passage of the ACA to reduce the HRRP readmission penalties they’d face if they didn’t bring the rate down. The policies they pursued were focused on HRRP readmission rates but had the effect of driving down rates for other admissions as well.

          [Pausing to go read the HHS post again, more closely this time]

          Zuckerman writes: “Conditions that weren’t targeted by the Hospital Readmissions Reduction Program probably saw spillover benefits from actions hospitals took in response to new incentives. For both the targeted conditions and other hospitalizations, the drop in readmissions mostly occurred during the period between the enactment of the Affordable Care Act in March 2010 and the start of the Hospital Readmissions Reduction Program in October 2012, when hospitals would have taken action to avoid facing penalties.”

          So I’m on board with Zuckerman re spillover benefits. It’s still a bit puzzling to me that hospitals would stop taking action as soon as HRRP kicked in but that would seem to explain the data. (Honestly, why would you not maintain whatever actions were driving down the readmission rate until you had evidence that you’d found the floor? Penalties or no penalties, it just seems wrong not to do so.)

        • @Chris

          No, I meant differently. I *don’t* think that “The hospitals hit the deadline, decide “We’re good enough.”, stop whatever special actions they were taking to reduce the rate,”

          My explanation was that the hospitals pre-plan so that most of the expected reduction from obvious optimizations happen before the deadline. The impending penalty drives this rational optimization in the aggregate.

          That’s not to say further reductions won’t happen. But those aren’t obvious ones. So like all innovations they will work on their own timeline. The hospitals couldn’t plan them in to kick in pre-deadline.

          Other reduction measures may be known but too expensive. You would see them kick in, say, if you increased the penalty or enforcement effectiveness to some high (probably non-optimal) value.

        • @Chris I agree about spillover/diffusion of benefits, whether the impact is do the ACA as a whole, this specific aspect of the ACA or some other aspect of the ACA. That the slope of the decline changed dramatically after the penalties started is an argument in favor of the penalties because the hospitals did what they could in anticipation of the penalties (and this is in line with my experience in universities, there is some say change in how federal financial aid works being imposed in a year, they work to get ready for that). Once the penalties started (and if this previous story is true) they may decide that this is a level they can live with, which is likely the level slightly below that which would trigger the penalties. The penalties do not require 0 readmissions. So of course you try to get low enough that with the amount of variation you expect, you predict that you won’t have to pay any penalties.

        • If you ignore the trendlines, you’ll see that the first regime change happened around Dec 2010, not Apr 2010 as implied by the graph. There’s a second regime change around Jul 2011, after the most extreme decreases have taken place. To my eye, full saturation doesn’t take place until around Jul 2013, well after the penalties kick.

        • Hmmm… Yes, the trend lines is probably biasing my interpretation. If I try to ignore the lines drawn then 12/2010 looks like a plausible break point and the decay looks more or less exponential after that. It would be interesting to see if a linear-decay-to-Dec-2010-then-exponential-decay-afterwards model fits the data better. It would also be interesting to know if hospitals made significant changes in their approach to reducing readmissions after 10/2012.

        • +1 It’s more like an exponential decay that continued even after the deadline, albeit slowly. But progress slows down close to saturation. The low hanging fruit have already been picked. Shaving the first half percent of the readmission rate is much much easier than the last.

          The analyst’s superimposing those linear segments sort of tricks the eye in falsely concluding that the deadline date caused some magic leveling off of rate. It actually didn’t. But you are just progressing so slowly at this point that the effect is hard to tell from a graph.

        • Oops. I should have read the comments more closely. Anoneuoid above: “That version of the chart is mislabeled. The penalties were implemented beginning with the first vertical line…”

    • To take this in a different direction, suppose we want to establish a useful model for readmission rate. What’s “useful”? I’d say one criteria is that you’d want to predict what the readmission rate floor is or, if there isn’t enough data to establish that, know how long you’ll have to collect before you’ll be able to estimate it to reasonable accuracy. You’d also like to know prediction error so that you know your ability to detect rate changes due to policy over unmodeled variations. Thoughts?

  6. It is possible that patients aren’t being readmitted because they are not being admitted at all. Patients might be staged in the hospital, cared for and discharged ( an observational stay) — and never be actually admitted. This is one way to game the system. Who knows if it is statistically relevant, however. Incentives do matter — but sometimes there are unexpected consequences from them.

    http://www.fiercehealthcare.com/story/readmissions-fall-observation-status-rises-even-hospitals-outside-readmissi/2015-10-28

    • Or conversely, it is possible that more patients are being admitted initially rather than staged, cared for, and discharged. That is, the readmission rate could be reduced by increasing the denominator.

      If having health insurance induces people to show up to the hospital sooner, this could increase the percentage that just need observation as well as increasing the admissions that get an intervention early enough to prevent a need for readmission.

      This also fits well with the fact that the effect began with the ACA and not the HRRP penalties.

      Just a theory without data. And not a very satisfying theory: it is more satisfying to assume someone is gaming the system.

  7. Do they similarly track duration of stay or billing codes for using social workers? Holding more patients an extra day or two to hedge against readmission wouldn’t be surprising especially now that more patients have health coverage that can pay for it. It may also be interesting to see if this has had any effect on social work and other hospital staff tasked with helping line up follow up care and home health which may help keep someone from being re-admitted if say they already have a scheduled follow up with a cardiologist or will have a nurse visiting their home. Again I imagine obtaining and using follow up appointment, home health, therapy services, medications etc would increase with improved health insurance coverage.

    • From my limited experience, length of stay seems to be one of the most heavily managed aspects of patient care: especially with a lot of top medicare procedures like hip/knee replacement. Unless you have complications they’re probably not going to keep you around just to hedge against readmission.

  8. All of the caveats and speculations are worthwhile and should be pursued (by someone) as this is an important issue and a potentially important finding. I only want to comment on the speculations on the importance of incentives. Yes, there are many incentives involved, but I am not surprised to see that monetary incentives have large effects. For those of you (us) in academia, just imagine that your own department is given its budget of $X and told that you can do anything you want with it (other than changing salaries, perhaps) and that $X will not change (and assume that is a credible promise). Would you be surprised to find that all of us PhDs might suddenly find ways that we could become more efficient? I would not be surprised.

  9. It isn’t surprising that the change in rate begins after passage of the act, rather than when the penalties kick in. The whole reason for having delayed implementation of major changes is to give the system time to plan and implement changes. Every institution in the health care system is a battleship: they do not change course quickly or easily.

    Working as I do in a clinical environment, I can tell you that very little, if any, of the change in readmission rates will be attributable to anything having to do with doctors. The major issue is discharge planning and post-hospital care. A typical situation is congestive heart failure. The patient is admitted in florid heart failure, and is given a “tune up” with intensive medical management of his/her condition. He is then discharged home. In the old days, nobody checked in on them to see how they were doing afterwards. There are early signs when heart failure begins to decompensate, and you can prevent serious episodes if you act on those signs. But nobody looked for them. Now patients discharged after being tuned up get visits or phone calls from nurses or other personnel to look for early warning signs, and, if they see them, do something about them. They also get help from social workers and other lay health workers to assist them with adhering to complicated medical regimens, dietary restrictons, etc. In the past, all that was just left up to the patient to navigate and when, predictably, it proved too difficult for them, they would decompensate and end up back in the hospital.

    There are a number of other situations like congestive heart failure where post-discharge monitoring and support are the key. I’m pretty sure that this is where the improvement is coming from. Hospitals previously had no reason to assure that appropriate post-discharge care was in place: their responsibility for the patient ended when the patient walked (was wheeled) out the door. Now that they experience the consequences of that failure in a negative, rather than a positive way, they are taking action and allocating resources to the problem.

    • I agree that the y-axis doesn’t always have to start at 0, but disagree with Kaiser as to why. He seems to emphasize the type of chart, but I think the characteristics of the data, in particular the scale and variability, are the important factors. If you had a plot of populations by country, why would you ever show 0? Maybe if you had the scale in terms of millions, but certainly not if it were the raw number. Or for a smaller number, say life expectancy, where the variability never puts you close to 0. Not starting at 0 can certainly be used/abused to make differences look larger than they are, but that assumes a failure to account for variability and the scale of meaningful differences.

    • In my experience the general attitude about y-axes not starting at zero is that it is sacrilege when others do it but when I do it I have a justifiable exception.

  10. I definitely agree that the y-axis shouldn’t _always_ start at zero, and I have made (and published) many many plots where it doesn’t.

    But if what you care about is relative rather than absolute change, the axis should start at zero. And that’s the case here.

    • Phil, do you suggest it specifically in the relative case so that you get a sense of the scale of the change? Like you could visually intuit the 20% change instead of running the numbers? If that’s the case, why not push for plotting the difference instead of the raw numbers (we care about the relative change, after all, not the absolute values)? What if 0 isn’t the practical floor for this measure, because some amount of readmissions will always occur?

        • In any kind of exploration or summary there’s no single plot that is good for everything. To me, at least, I’m interested in seeing both the percentage of admissions that lead to readmission (this is Daniel’s point) and also how much they have changed as a percentage of the initial numbers. You can do both of those with a single plot if you start it at zero.

          The actual rate is important. Suppose only 0.1% of hospital stays resulted in readmission. Then cutting that number by 20% wouldn’t be such a big deal.

          If the authors had chosen to display some other reasonable choice of plot, I wouldn’t have complained. But once they _did_ decide to display the percentages (and I think that’s a good choice) it would have been better if they had started the axis at 0.

          Interestingly, I might feel differently if the percent change were smaller. Suppose readmissions had gone down less than one percentage point…let’s say a 5% reduction; that change would be hard to see on a plot that starts at zero, but would nevertheless be of practical importance. It might then be reasonable to have an axis that doesn’t start at zero, but I would definitely point out in the caption (and perhaps on the plot itself) that the axis doesn’t start at zero. Or maybe I would consider plotting something different like “percent difference from the 2009 average” on the left side and absolute readmission rate on the right side.

          Also, I’d like to point out that saying that someone has “done us a disservice” with their choice of axis is not the same as saying they’ve done something wrong, evil, unethical, very bad, etc. It’s just not the best choice in this case, in my opinion.

  11. *patients go to different hospitals (did this account for readmission to different hospitals?)
    *patients pass away – so they dont get “readmitted”
    *readmission rates may be lower because inpatient stay durations are longer- now that the govt can get the bill. That may mean increased not decreased healthcare costs.
    *you are missing an important point here- – hospitals can deny (re) admission because the admitting diagnosis is not financially net positive. That doesnt exclude the possibility that sick patients are being turned away. ” oh you have obamacare? here , take this antibiotic and follow up with your PCP, you dont need to be admitted”

    • * The wording in the study is a bit opaque to me, but I think they only counted “re-admission” to the same hospital. I don’t know how big a deal this is.
      * If patients are more likely to die, that would show up in the hospital’s mortality rate, which is also tracked. I assume, but don’t know for sure, that there are people looking at the relationship between readmissions and mortality, and keeping track of which hospitals do best and worst, etc.
      * I’m not sure I can be said to be “missing an important point here” when the point is one that I mentioned in the post!
      * There’s no such thing as “having Obamacare”; as far as the patient is concerned, “Obamacare” just provides more incentives (and, for some people, the means) to buy insurance. So you either have insurance or you don’t. I assume the incentives for the hospital can run either direction when it comes to admitting someone with insurance: on the one hand, on average they presumably make money when they admit a patient, but on the other hand if they get dinged for a readmission then that runs the other direction. Probably the best thing from the perspective of patient care would be having the people who decide on whether a patient is readmitted or not — I assume this is the doctors — be insulated from the financial consequences of the decision.

      You’re right that there are lots of metrics for “quality of care” and it would be a huge mistake to just focus on a single one.

  12. Furthermore, I would argue that if the ACA was indeed effective, we should see a net INCREASE in new admissions (since newly insured patients are now going to the hospital, instead of toughing it out because of a lack of coverage) coupled with a net decrease in hosp readmissions. Though other adjustments must be made- as in my previous post.

    the other metric to follow is admitting diagnosis, and number of prev hospitalizations per admitted/re-admitted patient. That should separate out the causal pathways. Are we dealing with more admissions for a small subset of severe conditions that were otherwise not managed due to insurance issues?

    • Plausibly, ACA would initially result in a net increase in new admissions, but as time goes on would ideally produce a decrease in hospital admissions, since people would be getting non-hospital medical care that would prevent a condition’s progression the the point where hospital admission was indicated.

    • IT all depends if you an get coverage for less serious things. Pre-ACA people may have had to wait until they were so bad off that they had to go, or the only way they could get treatment covered was to be admitted. I think this is part of the story behind the changes in some diagnoses since ACA.

Leave a Reply

Your email address will not be published. Required fields are marked *