Can we say whether a drug would have enabled someone to live longer? Sadly not.

As of the 23rd May 2022 this website is archived and will receive no further updates.

understandinguncertainty.org was produced by the Winton programme for the public understanding of risk based in the Statistical Laboratory in the University of Cambridge. The aim was to help improve the way that uncertainty and risk are discussed in society, and show how probability and statistics can be both useful and entertaining.

Many of the animations were produced using Flash and will no longer work.

In the first televised election debate last Thursday, David Cameron stated that “I have a man in my constituency … who had kidney cancer who came to see me with seven others. Tragically, two of them have died because they couldn't get the drug Sutent that they wanted..”. How reasonable was it to claim that two would not have died had they had access to Sutent? Some statistical analysis can give us an insight.

After a process of price negotiation with manufacturers Pfizer, Sutent (the market name for sunitinib) was licensed in February 2009 by NICE (National Institute for Health and Clinical Excellence) for some but not all patients with kidney cancer.

NICE did not recommend the drug for all cases due to lack of evidence on its benefits:

No data were presented to the Committee on the clinical or cost effectiveness of sunitinib compared with best supportive care as a first-line treatment for people with a poor prognosis who were unsuitable for immunotherapy. In the absence of robust data, the Committee concluded that sunitinib could not be considered a clinically effective first-line treatment for people with a poor prognosis who are unsuitable for immunotherapy (paragraph 4.3.16).

I do not know the details of the patients that came to see Cameron, and in any case it is never possible to say what exactly would have happened had something else been done - what is known as a 'counterfactual'. However, presumably the patients were in the group that was not recommended for Sutent. They may well have felt they too could have benefited, but even if Sutent was as effective for them as it is for the those suitable for treatment, this unfortunately still does not guarantee they would have lived longer with the new drug.

As can be seen from the technical details in the NICE appraisal, survival benefits of drugs are often expressed in terms of the hazard ratio of, say, 0.80, which would mean that, each month, someone on the new drug has 80% of the chance of dying in the following month as they would were they not taking the drug. Rather remarkably, by a neat bit of mathematics, we can show that knowing the hazard ratio means we can work out the chance that an individual getting the treatment will outlive someone who does not.

Specifically, if the hazard ratio is [math]h[/math], then someone taking the drug has a probability [math]1/(1+h) [/math] of outliving someone who is not taking the drug. So if [math]h [/math]= 0.5, that is the monthly risk of dying is halved by the drug, then the probability of living longer by taking the drug is 1/1.5 = .67, or 67%. This extraordinarily simple result does not seem to be widely known, although discussed in Henderson and Keiding’s excellent paper on applying survival models to individuals. The result is proved at the bottom of this post for those with an interest in survival analysis.

So what is the hazard ratio for Sutent? The NICE Committee assumed median overall survival on the current standard treatment to be 27 months, compared to 37 months with Sutent (paragraph 4.3.9). They assumed Weibull survival models which means that the hazard ratio is not constant across time, but if we make an assumption of exponential distributions then the ratio of their median survival is also their hazard ratio, i.e. 0.73, which seems reasonable given the hazard ratios quoted elsewhere in the report.

So even if Sutent was as effective for those currently not recommended by NICE, it is sad but true that there would still have been only 1/1.73 = 58% chance that someone with access to the drug would have survived longer than a similar person denied the drug. Drugs may be beneficial on average, but this does not mean that we can be confident that patients given the drug will live longer than those denied it.. [NB the preceding bold text replaces the following italic text which is inappropriate -see helpful comment below would have survived longer had they had access to the drug. Drugs may be beneficial on average, but this does not mean that we can be confident that a patient will live longer if they take the drug.].

Proof of the result (for consenting statisticians only).

For an unknown survival time $X_0$ without the drug, denote the survival function at time $t$ by $S_0(t) = P(X_0>t)$, the survival density by $f_0(t)$, and the hazard function by $h_0(t)$, where $h_0(t) = f_0(t)/S_0(t)$, essentially the risk of dying having survived until time $t$.

Then the integrated hazard $H_0(t) = - \log S_0(t)$, and so $S_0(t) = e^{-H_0(t)}$. If $\lambda$ is the hazard ratio associated with a drug, assumed independent of $t$ (a proportional hazards assumption), then someone taking drug has integrated hazard $H_1(t) = \lambda H_0(t)$ and so $S_1(t) =S_0^\lambda(t).$

So the probability that the survival time $X_1$ for a random individual given the drug is greater than the survival time $X_0$ for someone not given the drug is
$$P(X_1 > X_0) = \int P(X_1 > t) f_0(t) dt = \int S_1(t) f_0(t) dt = \int S_0^\lambda (t) dS_0(t) = - \left[\frac{S_0^{\lambda+1}}{\lambda+1}\right]_1^0 = \frac{1}{\lambda+1}.$$

Levels: 

Comments

I suspect the result is so simple to state and to prove because, if you are actually in the position of knowing the hazard ratio accurately, most of the difficult maths has already been done. Even so, it's a useful result, and useful to know a little more about how NICE makes its decisions.

A fantastic blog always comes-up with new and helpful information and while reading I have feel that this blog is really have all those quality that qualify a blog to be a good one. Cancer is until today the most feared disease of all.Today everybody want to prevent himself dangerous disease like carcinoma cancer .So people should be aware from its symptoms. To learn about acs

>>> we can show that knowing the hazard ratio means we can work out the chance that an individual getting the treatment will outlive someone who does not. that's correct. >>> the probability of living longer by taking the drug is 1/1.5 = .67 that's not correct. The problem is that we cannot tell from the hazard rate alone whether the drug slightly extends the life of all patients taking it, or extends the life of some and shortens the life of others (with a gain on average). If we know the distribution of lifespan of patients on the drug, and the distribution of lifespan of patients not on the drug, we can calculate the probability that patient A on the drug will outlive patient B not on the drug. The uniform hazard ratio is a simply case of this. But this is not the same as the probability that patient A on the drug will outlive the same patient A not on the drug. To estimate that, we'd need to know the distribution of effects on individual patients.

yes you are right, and it shows how easy it is to use the wrong language of 'counterfactuals' (what would have happened if someone had done something different), which can never be determined and so should not be given probabilities. The probability should only be applied to a statement that is, in principle, confirmable: eg that patient X will live longer than patient Y.

I have edited the blog.

This exchange is very interesting and seems to me to be very relevant to a number of new (and very expensive) biologicals that are in clinical trials, or recently introduced to the market. Quite a few of these drugs seem to exhibit bi-modal distributions in that a proportion of patients show little or no benefit and another proportion can receive a great benefit. The problem is of course that patients who are selected to receive the new drug normally then do not receive the previous standard therapy. If that previous therapy had shown a wider acting but more modest activity, including some benefit for patients who fail on the new drug, then a proportion of individual patients end up being much worse off than before the new drug was introduced. The problem is that patient pressure groups tend to highlight the successes and not the downside for some patients. I guess this is where personalised medicine comes in to play! Mike Clark

Incredible Article. As a middle school student I am unable to understand some aspects of the article, but my family recently had this issue with my grandma as she was in the hospital.

Nice proof of the relative hazard result. Here is another using only words (but also calculations offline). It makes no difference if we assume that X_0 has the exponential distribution with parameter 1. Then X_1 is exponential with parameter lambda, and the result follows. The full proof, when written out, is longer than yours, but this one connects to things taught in elementary probability courses.

The content of your blog is exactly what I needed.This topic has always fascinated me. All about seo what is seo | How we can enhance our earning seo tips | seo optimization I really appriciate you for this great job.

Hi! Could we interpret 58% odds from the sample as a statement that among 100 patients who recieves this drug only 8 will live longer than patients who doesn't? Thank you.