There is something that fascinates the public about the possibility of treating cancer with a vaccine. Perhaps that explains why so many abstracts and journal articles about the latest cancer vaccine research find their way into our newspapers, magazines and television reports.
A research article appearing online today in the British medical journal The Lancet describes a clinical trial which investigated whether a vaccine called vitespen could improve the survival of patients with primary kidney cancer.
Unfortunately, the study points out—yet once again—that we may be hopeful that cancer vaccines will work, but we are a long way from success.
What is even more startling about this report—aside from the fact that a journal is actually publishing what we call a “negative” clinical trial—is the editorial which discusses the results. The author of the editorial made some not-so- kind comments about how vaccine companies distort reports of vaccine trials, and how investigators make inappropriate claims regarding their research.
The design of the study was straightforward. The researchers randomly assigned otherwise healthy patients who had newly diagnosed kidney cancers that were confined to the kidney to either receive a vaccine made from their tumor tissue or not receive this vaccine.
The vaccine—which was made from something called a “heat shock protein”—was prepared in the lab from cancer tissue removed at the time of surgery. The study was international, with participants coming from several parts of the world, including the United States, Russia, Poland, Israel and Western Europe.
After the vaccine was prepared, it was then shipped back to the doctors caring for the patients and given initially once a week for 4 weeks, then every other week until the vaccine supply ran out. Both groups of patients—half of whom received the vaccine and half who did not—were followed until there was evidence that their cancer returned.
Unfortunately, as pointed out in the journal article and the editorial, there were a number of technical problems with the study. This meant that there were fewer patients to analyze for the final report. Ultimately, there were about 360 patients in each group.
The results of the study showed that there was no evidence that this vaccine made a significant difference in the time to recurrence for these kidney cancer patients. Both patients who received the vaccination and those who did not recurred at similar intervals. There was a suggestion that patients with earlier stage disease did do better than similar patients in the untreated group, but the differences were not large enough to say that the vaccine was responsible for that difference.
Then the authors did something that is not uncommon in vaccine studies. They did an analysis that was not originally planned as part of the initial study design. This is what the authors and others call a “post-hoc analysis.”
These types of analyses are generally frowned upon. Experience has taught us that these not infrequently can lead to erroneous conclusions. Most researchers have come to the conclusion that you plan your study in advance and you determine what outcomes you want to look at when your study is completed. That approach is generally accepted as removing much of the bias that could otherwise result by changing the rules as you go along during the clinical trial.
If you do a study and find an unplanned outcome with a subset of patients, then the appropriate thing to do is to run another study and prove your theory. There are several clinical trials going on right now that are outgrowths of such an approach.
In this report, the authors did in fact do one of these unplanned analyses, and reported on their results as part of this paper. They concluded that in patients with “intermediate-risk” kidney cancer the vaccine decreased recurrences compared to control patients who did not receive vaccine.
They concluded that, although their vaccine did not improve the outlook for kidney cancer patients when measured by conventionally accepted approaches, “Nevertheless, the observation that treatment with vitespen (the cancer vaccine) confers an apparent clinical benefit to patients with earlier stage disease with better prognosis is biologically plausible and consistent with a wealth of scientific and clinical evidence.”
That is somewhat like saying the vaccine didn’t work but it worked.
This is no small issue.
There has been a huge fight between vaccine proponents and the Food and Drug Administration regarding a vaccine for the treatment of advanced prostate cancer called Provenge. That disagreement—which at times has been ugly—resulted from just the same type of post-hoc analysis.
In that case, the FDA has decided not to approve the vaccine based on the unplanned analysis, pending the results of additional clinical trials. To say the least, the proponents of Provenge are not pleased. They have lobbied Congress, the FDA and everyone they can think of to get the vaccine approved based on that analysis as opposed to waiting for the completion of the clinical trial.
The editorial which accompanied the kidney cancer article in The Lancet was vehement on these issues. In fact, I don’t recall reading many (if any) editorials that have attacked the conclusions of researchers and the sponsoring pharmaceutical company so directly in a major medical publication.
The editorial, written by James Yang, MD from the National Cancer Institute, pointed out the results and technical failings of the study. He also noted that, just like several other cancer vaccines, there were early phase studies of vitespen that looked promising in the treatment of cancer but didn’t pan out when studied as part of larger clinical trial.
Dr. Yang stated that one of the allures of cancer vaccines is that they don’t have many side effects. “That lack of toxicity also makes it easier to dismiss the fact that only rare objective regressions of metastatic disease were seen…Unfortunately, the results do not support the hypothesis that vitespen is beneficial.”
He goes on to note that cancer vaccines simply don’t have the power to help the body fight the cancer cells, much like they would fight an infection. The theory behind cancer treatment vaccines makes sense, he notes. It is turning that theory into reality that has been so difficult for these past many years.
Then the editorial gets a bit more interesting and less conventional in its statements:
“Yet the credibility of the field of cancer immunotherapy is weakened when some investigators, and particularly vaccine companies, cannot accept the results of randomised trials. There has been extensive use of post-hoc subset analyses to salvage underpowered studies or those that fail to reject the null hypothesis (my note: that means studies that show the vaccines don’t work). Such practices are akin to shooting the arrow first and being permitted to draw the target afterwards…As Wood and co-workers (the authors of the article) state, ‘interpretation of post-hoc analyses must be approached with caution’, which is largely because subsequent prospective studies often cannot verify the results.”
Dr. Yang goes on to chastise the company that has sponsored the research on vitespen for misleading press releases. Those press releases dealt with the announcement that this vaccine will be available to patients in Russia in the second half of 2008.
“One final issue to be raised is the differences seen between results and analyses as presented in peer-reviewed publications and the sometimes selective reporting in company press releases. In a press release on April 8, 2008, and despite the results of the trial reported today, the manufacturer of vitespen highlighted the approval of vitespen in Russia for patients with intermediate-risk renal cell cancer. That announcement uses relative rather than absolute risk and fails to point out that the analysis was not pre-specified in the protocol…
Commercially driven efforts that spin or obfuscate the conclusions of such a trial should be vigorously resisted because such efforts seriously erode its value.”
I have been watching the efforts to develop a vaccine to treat cancer for over 30 years.
We have learned much about the human immune system. We have learned much about the various markers—or antigens—that should make cancer cells appear different to our bodies the same way an infection does. And I have seen failure after failure of vaccine trials to deliver something to patients that results in a consistent, clinical meaningful improvement in their condition and/or their survival.
I’m sorry if I seem a bit skeptical, but time and time again when these studies are presented to me for comment I have to express my skepticism, and advise that we must wait for the results of larger, more appropriate clinical trials before we can get truly excited about any particular vaccine.
Almost always when I see these early abstracts they are accompanied by glowing comments from an investigator suggesting that their vaccine research is a significant breakthrough, when in fact it is not. These abstracts are frequently accompanied by a similarly glowing press release from a major research university where the research was done which further creates a spin suggesting there is more to the research report than actually exists on careful review.
(In the interests of full disclosure, I had previously written a blog in October of 2005--shortly after this blog was started--based on research and a press release from the same company that made similar claims for patients treated with a heat shock protein vaccine for advanced melanoma. At the time, I did try to balance my enthusiasm for the results reported by the company with the same skepticism I am discussing here. On rereading that blog, I am not so certain that I accomplished my goal.)
I know how hard the researchers are working on trying to get this right. And I know how difficult it is for those with cancer to hear media tidbits that this vaccine or that vaccine may help them survive their cancer longer. And I know that the argument is going to be made that the Russians are more forward-thinking in their treatment of kidney cancer, since you can get the vaccine there and not here.
But—just as Dr. Yang so succinctly and directly pointed out—there have been too many promising early reports for cancer treatment vaccines that never panned out when tested in the real life conditions of a large clinical trial. There are more than a few reputations in cancer research and treatment that have been made on the early promises of vaccines, to be deflated after years of work when the theory didn’t result in an effective cancer treatment.
So the journalists will continue reporting on early vaccine studies that appear promising, and I will be quoted urging caution until we see real results in real clinical trials. And patients and their families will continue to be drawn into the cycle of hope and hype that has been so destructive in cancer treatment in years past.
We owe it to everyone to be as objective as possible when we report our research results or issue our press releases. As researchers, we must avoid the allure of overpromising, then under delivering.
Ultimately, what the public needs to know are the facts. Then all of us can focus on making the best decisions and recommendations in caring for our patients