Can we say whether a drug would have enabled someone to live longer? Sadly not.

In the first televised election debate last Thursday, David Cameron stated that “I have a man in my constituency … who had kidney cancer who came to see me with seven others. Tragically, two of them have died because they couldn't get the drug Sutent that they wanted..”. How reasonable was it to claim that two would not have died had they had access to Sutent? Some statistical analysis can give us an insight.

After a process of price negotiation with manufacturers Pfizer, Sutent (the market name for sunitinib) was licensed in February 2009 by NICE (National Institute for Health and Clinical Excellence) for some but not all patients with kidney cancer.

NICE did not recommend the drug for all cases due to lack of evidence on its benefits:

No data were presented to the Committee on the clinical or cost effectiveness of sunitinib compared with best supportive care as a first-line treatment for people with a poor prognosis who were unsuitable for immunotherapy. In the absence of robust data, the Committee concluded that sunitinib could not be considered a clinically effective first-line treatment for people with a poor prognosis who are unsuitable for immunotherapy (paragraph 4.3.16).

I do not know the details of the patients that came to see Cameron, and in any case it is never possible to say what exactly would have happened had something else been done - what is known as a 'counterfactual'. However, presumably the patients were in the group that was not recommended for Sutent. They may well have felt they too could have benefited, but even if Sutent was as effective for them as it is for the those suitable for treatment, this unfortunately still does not guarantee they would have lived longer with the new drug.

As can be seen from the technical details in the NICE appraisal, survival benefits of drugs are often expressed in terms of the hazard ratio of, say, 0.80, which would mean that, each month, someone on the new drug has 80% of the chance of dying in the following month as they would were they not taking the drug. Rather remarkably, by a neat bit of mathematics, we can show that knowing the hazard ratio means we can work out the chance that an individual getting the treatment will outlive someone who does not.

Specifically, if the hazard ratio is [math]h[/math], then someone taking the drug has a probability [math]1/(1+h) [/math] of outliving someone who is not taking the drug. So if [math]h [/math]= 0.5, that is the monthly risk of dying is halved by the drug, then the probability of living longer by taking the drug is 1/1.5 = .67, or 67%. This extraordinarily simple result does not seem to be widely known, although discussed in Henderson and Keiding’s excellent paper on applying survival models to individuals. The result is proved at the bottom of this post for those with an interest in survival analysis.

So what is the hazard ratio for Sutent? The NICE Committee assumed median overall survival on the current standard treatment to be 27 months, compared to 37 months with Sutent (paragraph 4.3.9). They assumed Weibull survival models which means that the hazard ratio is not constant across time, but if we make an assumption of exponential distributions then the ratio of their median survival is also their hazard ratio, i.e. 0.73, which seems reasonable given the hazard ratios quoted elsewhere in the report.

So even if Sutent was as effective for those currently not recommended by NICE, it is sad but true that there would still have been only 1/1.73 = 58% chance that someone with access to the drug would have survived longer than a similar person denied the drug. Drugs may be beneficial on average, but this does not mean that we can be confident that patients given the drug will live longer than those denied it.. [NB the preceding bold text replaces the following italic text which is inappropriate -see helpful comment below would have survived longer had they had access to the drug. Drugs may be beneficial on average, but this does not mean that we can be confident that a patient will live longer if they take the drug.].

Proof of the result (for consenting statisticians only).

For an unknown survival time $X_0$ without the drug, denote the survival function at time $t$ by $S_0(t) = P(X_0>t)$, the survival density by $f_0(t)$, and the hazard function by $h_0(t)$, where $h_0(t) = f_0(t)/S_0(t)$, essentially the risk of dying having survived until time $t$.

Then the integrated hazard $H_0(t) = - \log S_0(t)$, and so $S_0(t) = e^{-H_0(t)}$. If $\lambda$ is the hazard ratio associated with a drug, assumed independent of $t$ (a proportional hazards assumption), then someone taking drug has integrated hazard $H_1(t) = \lambda H_0(t)$ and so $S_1(t) =S_0^\lambda(t).$

So the probability that the survival time $X_1$ for a random individual given the drug is greater than the survival time $X_0$ for someone not given the drug is
$$P(X_1 > X_0) = \int P(X_1 > t) f_0(t) dt = \int S_1(t) f_0(t) dt = \int S_0^\lambda (t) dS_0(t) = - \left[\frac{S_0^{\lambda+1}}{\lambda+1}\right]_1^0 = \frac{1}{\lambda+1}.$$