Evidence-based medicine

Can measurements show if a treatment works?

In the 1980s, millions of people had treatment with drugs intended to prevent sudden cardiac death. But it later turned out that the exact opposite happened: The drugs caused more people to die. What went wrong, and what can we learn from this experience?

Some people develop a specific form of irregular heartbeat (arrhythmia) after a heart attack. They have an increased risk of sudden cardiac death. To try to make this irregular heartbeat return to normal, researchers developed drugs called class I antiarrhythmic agents in the 1970s. Clinical studies showed that these drugs did make heartbeats return to normal in electrocardiograms (ECGs, also sometimes called EKGs). These antiarrhythmic agents were then used in large numbers in the 1980s.

In the late 1980s, a big study (the CAST study) looked at the effect of antiarrhythmic agents on heart beat as well as at the number of people who took the drugs and died of cardiac arrest. The results of the study were alarming: Compared to the group who had taken a dummy drug (placebo), the rate of sudden cardiac death was twice as high in the group who had used an antiarrhythmic agent.

Fatally wrong conclusion

Why had people been treated with harmful drugs for so many years? Because experts had drawn the wrong conclusions: Having an irregular heartbeat was known to increase the risk of sudden cardiac death. So they concluded that drugs for the treatment of an irregular heartbeat would be able to lower this risk. From a medical point of view, this conclusion seemed to be plausible. But it still turned out to be wrong.

The results of the CAST study are now considered to be a prime example of why measurements alone can't be relied upon. For a long time, ECG measurements were considered to be good predictors of the risk of dying. Criteria that are used in studies to substitute an important outcome are also called surrogate endpoints or surrogate markers (from the Latin surrogatum, meaning substitute).

Outcomes that are important to patients are called patient-relevant endpoints. The term “patient-relevant” reflects the fact that it concerns issues that are important to the people who have a medical condition – for example whether a treatment

  • helps them live longer,
  • spares them from going to the hospital,
  • reduces their symptoms,
  • prevents complications, or
  • helps them cope better with their condition in daily life.
Table: Examples of surrogate endpoints and the corresponding patient-relevant endpoints

Surrogate endpoints

Patient-relevant endpoints

Cholesterol levels

Heart attack

Bone density

Bone fractures

Irregular heartbeat

Sudden cardiac death

Blood pressure

Stroke, heart attack

Tumor does (not) respond to treatment

Mortality, quality of life

Surrogate endpoints: Tempting, but only rarely reliable

Particular diseases are often associated with abnormal measurements or laboratory values. Links like this can help understand a disease better. But it cannot be assumed that a treatment that improves a certain laboratory value will also automatically reduce the risk of a related medical condition.

Most surrogate endpoints do not take into account the complex processes happening in the body. Sometimes particular values deviate from the norm in healthy people too. Or a treatment influences the patient-relevant endpoint, but not the surrogate endpoint. A treatment can also affect a surrogate endpoint without influencing the patient-relevant endpoint.

Studies often only use surrogates rather than the endpoints that are important for patients because they're a lot easier to measure. For example, a study will quickly show whether a medication lowers blood pressure. But it can take years for researchers to find out whether this also prevents diseases like heart attacks.

Another reason is that studies on surrogate endpoints need far fewer participants. There are statistical reasons for this: Heart attacks, for example, are quite rare, so a large number of people have to be monitored to see clear differences between the different treatment groups. A study on blood pressure only needs few participants in order to show an effect. This is because it's possible to measure changes in blood pressure for each individual participant.

So caution is needed when laboratory values and physical measurements are used as surrogates in studies to determine how much patients benefit from a treatment: Just because a drug lowers blood pressure or blood sugar, it will not necessarily prevent heart attacks or strokes.

When are surrogate endpoints useful?

Some laboratory values and physical measurements are anything but useless in medicine, though. They are used to make diagnoses, gauge or monitor the progress of a condition, or check whether a treatment is working or the dose is right. Someone with type 1 diabetes, for example, will regularly monitor their blood sugar levels in order to adjust their insulin dose. Laboratory tests and ECGs help to diagnose a heart attack.

Even though there are usually problems with using these kinds of measurements as substitutes for patient-relevant endpoints in studies, there are sometimes reasons for using surrogate endpoints provisionally. For instance, if there is a serious disease for which there is no effective treatment, it may make sense to introduce a new treatment based on surrogate endpoints to start off with. It is then important to take a close look at its effect as soon as possible in studies with patient-relevant endpoints.

The approval process of the earliest drugs for HIV is one example of how surrogate endpoints can be useful in certain circumstances: Studies had shown that as the disease progresses the number of immune cells in the blood decreases. The treatment made this number increase again. But there were no studies showing that this leads to fewer people developing AIDS or dying. Because there were no alternative treatments and HIV progresses rapidly if left untreated, the drug regulatory authorities approved these drugs anyway.