Phase III clinical trials are conducted to find out how effective a new treatment is against a current gold-standard treatment. They're large-scale trials typically involving several hundred people who are usually randomized (chosen at random) to get one treatment or the other. The National Cancer Institute sponsors a good number of clinical trials, many of which are run by NCI-sponsored cancer cooperative groups. Pharmaceutical and biotech companies also sponsor clinical trials.

"We can't find out if one treatment is better than another if patients aren't involved. Somehow more patients need to realize how important they are to the process," said Djulbegovic. Part of the answer is to do a better job of explaining how clinical trials work and to help patients better understand the principles involved, he says.

The research team, which is based at the H. Lee Moffitt Cancer Center and Research Institute in Tampa, Florida, looked at 624 National Cancer Institute-sponsored phase III trials involving 216,451 people. Djulbegovic and his colleagues calculated the proportion of conclusive and inconclusive trials, as well was the proportion of studies that led to a "breakthrough intervention," defined either as a therapy so beneficial it immediately becomes the new standard of treatment or one that reduced the death rate by 50% or more.

About 30% of the studies had results that were statistically significant, meaning the resulting numbers clearly favored one group or the other. Of these, 80% favored the experimental treatment, and 20% favored the standard. However, according to Djulbegovic, statistical significance only goes so far in measuring a trial's success.

"Treatment has other dimensions. It's important to supplement statistical significance with other value judgments to get the whole picture,” he said. As an example, Djulbegovic cited "…the introduction of lumpectomy instead of mastectomy, which resulted in dramatic improvement in quality of life of women with breast cancer" even though it didn't change survival compared to mastectomy.

To assess subtler issues involved in identifying successful trials, Djulbegovic and his team reviewed the published judgments as written by the original researchers. About 41% of the time, the new therapies were considered superior by the trial investigators, while in 59% of the cases the standard treatments were deemed to be better.

When data from all of the studies was combined, the new treatments on average demonstrated a 5% reduction in the death rate. In 2% of cases, the death rate fell by more than 50%. Survival gains were highest among those cancers that saw advances in adjuvant and intensive chemotherapies, such as gastrointestinal cancers (cancers in the digestive tract) and blood cancers.

"Society has received a good return on its investment in the cooperative oncology group system," the authors write. They argue that success rates could improve if the number of inconclusive trials were reduced. According to their data, 29% of randomized comparisons were inconclusive – that is, they didn’t answer the question they were intended to address. This is in large part, the authors say, because the researchers were often too optimistic about what the results would be before the study started, and therefore didn’t make the study large enough to get meaningful results.

For more information on clinical trials, see Clinical Trials: What You Need to Know.