Make comment about blog Blog Titles Home Page Blog submission guidelines Link to Newsletters Frequently Asked Questions Blog archives

PROTON CRITICS CLING TO FLAWED STUDY

In past blogs I have pointed out the insufficient nature of the material used by critics of proton therapy, namely:

1. Outdated statistics from the early years of proton therapy, prior to pencil beam scanning.
2. Claims that more evidence is needed to prove the efficacy of proton therapy.
3. Flawed studies designed to minimize the performance of proton therapy.
4. Downright falsehoods and misinformation.

In this article, I analyze an oft-quoted study that is the foundation of much criticism, point out its flaws, and consider its unfortunate ramifications.

To set the scene, let’s look at a study the Mayo Clinic conducted comparing chemo-radiation paired with either x-rays or proton therapy. The results:

Elderly patients experience more post-treatment heart and lung problems, and are at higher risk for death after surgery than younger patients after receiving a combination of preoperative chemotherapy and radiation therapy,” says Dr. Lester. “Our study found that patients treated with proton beam therapy experienced lower rates of postoperative heart and lung problems, such as acute respiratory distress syndrome and death.

They found that elderly patients treated with proton beam therapy had lower rates of heart and lung problems after surgery and a lower postoperative mortality rate than patients treated with conventional techniques. None of the patients treated with proton beam therapy experienced a postoperative death, which the researchers think is partially related to proton therapy's ability to reduce the dose to important structures near the esophagus, such as the heart and lungs.

Similar results have been verified in hundreds of trials and studies. The greatest claim to excellence is proton therapy’s minimal damage to healthy tissue and surrounding organs. The ability of protons to stop at the target distinguish it from X-rays, which keep going.


Yet a study from M. D. Anderson and Massachusetts General Hospital, both of which have proton centers, conducted an experiment in which that didn’t seem to be the case. (Here’s the link.) This study is widely quoted by critics as demonstrating that protons are no better than X-rays.

This article in MedPage Today heralded the results of the trial with the headline: “No Reduced Toxicity for Proton Tx vs IMRT in NSCLC.” Proton Tx stands for proton treatment. IMRT is Intensity-Moderated Radiation Therapy, a type of X-ray treatment. NSCLC stands for Non-Small Cell Lung Cancer. Because the lungs are a moving target, this is one of the most challenging areas for radiology in general. It would seem like a perfect opportunity to compare these two modalities. The purpose of the study was not to determine the effectiveness in killing the cancer. Rather, it measured levels of toxicity, which is to say, damage to tissues. The study concedes that proton therapy exposes less heart tissue to radiation than x-rays, but that wasn’t their focus.

Yet, what if it were? Suppose they were, in fact, measuring heart exposure. Then the headline would have been “Proton Therapy Exposes Heart to Less Damage than IMRT.” I think that would be extremely relevant to the patient. Instead they were measuring radiation pneumonitis (RP), which is inflammation of the lung, and local failure (LF). After one year, the combined rates of toxicity were 17.4% for for IMRT and 21.1% for proton therapy. X-rays win, or so it would seem. This is all the critics needed to hear. Have any of them looked at the study? If they did, they would find the results are not only unconvincing, they are exactly the opposite. From the midpoint of the study onward, protons heartily outperformed X-rays. Given the irregularities in the study, I must question whether it was intended to get a specific result.

First, the proton technology used was the older PSPT (passively scattered proton therapy) technique. This utilizes a shaped beam. A more up-to-date technology is pencil beam scanning, in which a series of dots are laid down in sequence. This improvement in accuracy increased the number of cancers treatable by proton therapy from 20% to 80%, due to its greater accuracy. So this study compares the best form of X-rays with a lesser form of proton therapy.


Breaking down the results a little more, we find that LF slightly favored proton therapy, 10.5% vs. 10.9% for X-rays. So the difference was in the RP inflammation factor. Specifically, at one year protons had an RP rate of 10.5% vs. 6.5% for IMRT. This is where the design of the trial comes into play. IMRT has been around for many years, whereas proton therapy at the time of this study (2012, but reported in 2018) was still developing. Whether intentionally or inadvertently, the design of the trial put protons at a considerable disadvantage, in that the X-rays were used expertly, but less so for the proton therapy.

Jeffrey Bradley, MD, director of the S. Lee Kling Center for Proton Therapy at Washington University School of Medicine in St. Louis, Missouri, responded to the treatment of the lymph nodes. When I received proton therapy for my prostate cancer, they also treated the lymph nodes, just to make sure the cancer hadn’t spread there. They were surprisingly distant from the prostate itself. In 2016, they treated me from each side, whereas in this trial, they first attempted to treat them all from one side. Here is Dr. Bradley:

"I was part of a team that reviewed their adverse events ... I went through every case that had an event -- either a failure or a pneumonitis.

"They began treating [contralateral lymph nodes] with PSPT (passively scattered proton therapy), and eventually they learned that protons were better when all of the target was on the same side of the chest. They started having some early events when they were trying to chase down lymph nodes on the opposite side of the chest. Those patients had some pneumonitis."

In other words, they tried to treat the lymph nodes from the opposite side of the target. That exposed more tissue, hence the “event” of RP.

"Their proton planning skills got better over time; the incidence of pneumonitis went to zero after trial midpoint, because they stopped trying to treat patients with contralateral lymph nodes where the proton dose was scattering to the opposite side of the chest."

Let me use my proton treatment as an example.


Above: The pink area represents my lymph nodes. You could treat them in one pass if you ignored the areas in the middle that you don’t want to irradiate. That’s what X-rays do. But proton therapy has a better possibility, as shown below.


Above: Protons can treat from either side. This compensates for any range insecurity (where the beam stops) and, in the case of the lymph nodes, can avoid radiating healthy tissue. This is what they discovered half way through the trial, at which time, any RP inflammation completely went away.

Protons have a great advantage of being able to stop at the target. So going beyond the target to treat the lymph nodes on the opposite side did not utilize the greatest advantage protons offer and increased exposure to healthy tissue, causing RP. After they improved their methods for proton therapy by treating the lymph nodes from the closest side, the incidence of RP was reduced to zero. Zero! The RP for IMRT stayed the same throughout the trial. There is no way it could get down to zero. The second half of the trial, then, properly reflected the actual difference between X-rays and protons, which was an RP of zero for protons and 6.5% for IMRT. That clearly that shows that protons are superior to X-rays.

But that’s not what they did. They reported on the whole trial, allowing the early statistics to drag down and dilute the real results achieved later in the study. And then there are the dose considerations. Here is Dr. Bradley again.

"So by design, they tried to give the same radiation dose to both arms, but you've taken away an advantage of proton beam therapy, because allowing a higher dose with protons is one of the advantages of the technology.

"Let's say you had an ice hockey team, and you can't use your best skaters -- everybody on the team has to be able to skate at the same level. You know you're going to get a similar outcome. Almost all experts in proton radiation therapy feel like they handicapped themselves by that study design."

Proton therapy’s star players had to stay on the bench. No wonder the game was a draw. The design put proton therapy at a great disadvantage. As if that weren’t enough, Dr. Bradley reveals yet one more obstacle for proton therapy to overcome, a larger target.

"In addition, the targets were larger on the proton arm than they were on the IMRT arm. It was early in the proton experience and they were worried about missing the target so they enlarged their aperture. They ended up treating, circumferentially, about 8-10 mm wider around the tumor with the proton arm. We've learned not to do that."

An aperture is a device that shapes the radiation to conform better with the target. In the case of prostate cancer, it is generally the size and shape of the prostate, or alternately, the lymph nodes. In this trial, the aperture for X-rays was smaller than the one for protons. That means protons were guaranteed to expose more tissue around the target, thereby increasing their RP. And still, even then, protons out performed X-rays in the second half of the trial.


Above: This is an aperture. The protons and X-rays are shaped by going through the aperture. But in this case, they didn’t use equal apertures. The one for proton therapy was larger. Had they been able to compare pencil beam scanning (which wasn’t available at that time) no aperture would have been needed for the protons, since it is a thin beam that scans rows and layers of the target. This study was reported last year, when half of the proton therapy centers have pencil beam scanning. The study is obsolete.

I’ll tell you what, if this is the best evidence the critics can come up with, they are in big trouble. Despite the handicap of the trial design, less skilled application, dose restrictions, and an enlarged target, protons still performed better once the technicians learned how to do it.

One might ask why they were even conducting a comparison when they weren’t yet skilled at proton therapy. Something is very wrong here. M. D. Anderson got their proton center in 2006, six years earlier. Certainly they knew how to make the best plan. Similarly, Mass General had been experimenting with proton therapy for forty years or more, even before the first proton therapy center opened. How was it that they were so inexpert with protons? Did the X-ray radiologists also conduct the proton arm? This needs some further investigation.

Dr. Feng-Ming (Spring) Kong, MD, from the Indiana University School of Medicine, wrote an editorial about this study entitled “What Happens When Proton Meets Randomization: Is There a Future for Proton Therapy?” Just the title gives me pause. Does the future of proton therapy rest with this one flawed trial? Isn't that a bit dramatic? (But then again, so am I.)

“Completion of this study is not trivial because the evaluation of the benefit of a new technology rarely has been done during the century-long history of radiation oncology practice.”

Excuse me? In the past, such trials were rarely done? The technology currently being used by all of these critical radiologists was itself never subjected to this kind of scrutiny! Go ahead, try to find conclusive randomized double blind studies in favor of most X-ray technologies. It seems the medical field clamors for scientific proof for the technologies that it opposes while holding no such standard for the modalities it embraces.

“Some may even argue that conducting a trial to test the significance of such a treatment is unethical, like performing a randomized study to test the value of parachutes, because it will put patients at risk for unneeded radiation complications."

Great analogy. How would you like to be in a trial testing parachutes. Wouldn’t that put some of the participants in a very dangerous situation if they got a parachute that didn’t work properly? In a trial comparing unequal technologies, one group of people will get inferior results. Some consider that to be unethical. However, if the participants in the trial had no other way to receive treatment, I suppose even the lesser technology would be better than nothing. Dr. Kong again.

"Such beliefs have been reflected in the history of radiotherapy technology advancement. From the first uses of x-rays and radium for cancer treatment in the early 1900s, to kilovoltage (superficial) x-ray machines and the era of cobalt-60 and megavoltage two-dimensional treatment, to Linac-based three-dimensional conformal technology and the current widespread use of IMRT, technologies have been developed and implemented routinely in the clinic without randomized trials.” 

Unbelievable. Those radiologists clamoring for randomized trials to prove the efficacy of proton therapy were never themselves subjected to the same demand. How hypocritical is that! Regardless, Dr. Kong declares that in this circumstance, we need such trials.

“Comparative clinical outcome data are needed for patients and their families to choose a cancer treatment modality that is not readily available, for physicians to make treatment recommendations, for investors/industry to determine where to spend resources, for insurance companies and government to make reimbursement policies, and for researchers to know how and where to focus their efforts. Thus, a randomized trial is needed to generate unbiased evidence for this extremely costly technology.”

Unbiased evidence. Great idea. That certainly doesn’t describe this study.

"Whether a better planning technique such as proton intensity modulation (IMPT) or pencil beam scanning (PBS) would have generated different results is hard to predict.”

Really? If we used some statistical extrapolation we can’t predict the results? The lesser PSPT technology got better results, so why not the upgraded PBS? Then Dr. Kong caves in.

“Personally, as a radiation oncologist, I would not recommend proton therapy for NSCLC outside a clinical trial setting until a clinical benefit is demonstrated in a prospective randomized study.”

How’s that for a double standard! But read between the lines. She wants “a prospective randomized study.” That means this one doesn’t fit the bill. She wants another one. A better one. Here’s why.

"In contrast to the largest retrospective study of patients from the National Cancer Database, this prospective randomized study failed to prove superiority of proton therapy." 

In contrast to? She's saying that the above trial came to a different conclusion than the large retrospective study she mentions. Let’s take a look.

The National Cancer Database was queried to capture patients with stage I-IV NSCLC treated with thoracic radiation from 2004 to 2012. A total of 243,822 patients (photon radiation therapy: 243,474; proton radiation therapy: 348) were included in the analysis. 

On multivariate analysis of all patients, non-proton therapy was associated with significantly worse survival compared with proton therapy (hazard ratio 1.21 [95% CI 1.06-1.39]; P<.01). On propensity matched analysis, proton radiation therapy (n=309) was associated with better 5-year overall survival compared with non-proton radiation therapy (n=1549), 22% versus 16% (P=.025). For stage II and III patients, non-proton radiation therapy was associated with worse survival compared with proton radiation therapy (hazard ratio 1.35 [95% CI 1.10-1.64], P<.01).

I think to the patient, survival is a rather important parameter. Even more so than inflammation. Non-proton radiation had worse survival rates. Well now, why do the critics quote one flawed study instead of this much broader perspective? X-rays had almost a quarter of a million chances to outperform the 348 proton patients, but they didn’t do it. And again, this was with the less effective passive scattering technology.

Dr. Kong gives several reasons why the results for the study may have been skewed against proton therapy. For example:

"Finally, the study design in terms of end point definition, control of confounding factors, and dealing with the lung dosimetric restriction may have confounded the results."

Confounding factors are those which would throw off the results. For example, she points out that RP was measured as “in-field radiographic changes.” That means around the tumor site. X-rays, however, affect much more tissue and could have produced RP outside of the targeted field. That wasn’t measured. Protons would be much less likely to have such effects. They simply didn’t measure a parameter likely to be unfavorable to X-rays. Nor were other important factors taken into consideration, such as which participants took chemotherapy drugs. In other words, this trial had some real design limitations, lessening the likelihood of a fair and unbiased conclusion.

"More importantly, the study required patients to meet dosimetric limits for both the PSPT and the IMRT arms, which may have resulted in not being able to enroll patients who would most likely benefit from protons."

They eliminated the best prospects for proton therapy, but not the best ones for IMRT. Going back to a sports analogy, if they take all of the best players they are more likely to win. Disappoiningly, after all of these qualifications, Dr. Kong accepts the skewed results.Then she gets political.

"Although negative results from a phase II study in NSCLC cannot exclude the potential benefit of proton therapy in other clinical situations, such as for pediatric patients, and the cost of proton therapy will be significantly reduced by newer technological changes, this trial should at least cause some pause in hospitals that are building these facilities for competitive reasons and not for cost-effectiveness reasons."

This is a favorite criticism repeated by critics, that proton therapy is a fad, or done for profit or status. Are we to believe a cancer center would spend a fifty to a hundred million dollars to stroke their ego? Where is the proof for such a broad statement? Can we not grant them the assumption that they are seeking the best tools to fight cancer? Despite this prejudicial statement, Dr. Kong believes protons are likely to prove more advantageous under better circumstances.

"With the availability of more gantry angles, better imaging guidance, more-accurate dose computation for moving lung cancer targets and low-density of lung tissue, and more-advanced treatment planning technology like pencil beam scanning, we can generate remarkably better plans at every tumor dose level that lead to meaningful benefits for many patients in the clinic and are proven as cost-effective treatment in some specific disease settings."

Dr. Kong was paid for writing her editorial by Varian, a leading manufacturer of both X-ray and proton therapy equipment. Frankly, I think she deliberately and cautiously avoided taking sides, saying both positive and negative things about proton therapy. She also may have hesitated to openly criticize those who conducted the study. So, while not directly criticizing this result, she does say we could do better. We can assume, then, that this was not, in fact, the definitive study that properly and accurately compares protons and X-rays.

"The randomized trial should only include patients for whom the use of protons provides a better dosimetric plan. Such a randomized trial will identify patients with proven dosimetric superiority from proton planning to demonstrate whether such a dosimetric advantage can be translated into clinical benefit. Another possible advantage of protons to investigate is that by delivering less dose to much of the body (if achievable), protons may decrease radiation-mediated immune suppression and thereby improve survival in patients with NSCLC." 

This is along the line of what Dr. Bradley was saying about the hockey players. Proton therapy should be able to play its best players and win the game. This study did not allow that. Plus we learn that the study didn’t quantify another potential negative, the effect on the overall immune system. IMRT irradiates far more healthy tissue, albeit at low doses. By suppressing the immune system, secondary cancers could result, which were not reported in this study. Is that suspicious? It was published in 2018, six years after the trial. Couldn’t they have calculated the five-year survival rate? Or further complications? In the end, this supposedly ground-breaking study was not a home run for IMRT and will most likely be overcome by proton therapy comparisons in future studies.

Click here for a very long, detailed, and positive look at therapeutic advances is respiratory disease.

The study we have examined claims to be the first such randomized trial. Here is another trial stating it is the first prospective study. It favors proton therapy.

Want more? here is yet another study favoring proton therapy.

So why don’t the critics cite these studies?

I believe all proton therapy advocates welcome randomized trials for comparison. Several tests in progress are struggling to find enough participants, often because lack of insurance coverage doesn’t allow the patients to be treated in the first place, so they can be placed into a trial. It’s a Catch 22; you need more trials to get insurance coverage, you need insurance coverage to conduct more trials.

Proton therapy languishes when it should be flourishing. I find this personally aggravating and upsetting. This situation is caused in part by our for-profit medical, healthcare and insurance systems. Most of the critics that are trying to slow the growth of proton therapy have a financial interest in doing so. In most cases, criticism isn’t to protect people from proton therapy, which is uncalled for, but rather, to feather one’s own nest or bottom line. As a result, some proton therapy centers are under utilized, growth of the proton industry is stifled, and prospective patients don’t avail themselves of proton therapy. Here are examples of money getting in the way.

X-ray radiologists want the business for themselves, failing to refer patients even when their own institution has a proton therapy center. Such actions are certainly unethical and in violation of the most basic medical directive to first do no harm. The American Society for Radiation Oncologists (ASTRO) is a major hindrance to proton therapy, even though proton therapy is a type of radiation and radiologists who practice proton therapy are part of the organization. ASTRO is certainly happy to receive millions of dollars from proton-related vendors at their national conventions. (See: A Drop in the Bucket: Proton Therapy at the ASTRO Convention.)

Insurance companies use outdated and unrealistic criteria for approval, in some cases breaking the law, with the decision being made by unqualified parties following rigid company guidelines calling for rejection. Their actions leave little doubt that they are trying to avoid the higher cost and thus increase their profitability. Since the huge $25.5 million judgment against Aetna for denying proton therapy, perhaps legal challenges will have some result. I have a blog suggesting some strategies and resources (See: Resources for Legal Challenges To Insurance Company Denials.)

Especially irritating are journalists who continue to spew out critical articles, making it their mission to save the unsuspecting public from the evils of proton therapy, all in the name of selling articles to make a living. (See a direct response in: An Open Letter to Jay Hancock.)

Money, not science or the lack thereof, is what drives proton criticism. Some would argue that it also drives proton therapy development.

Critics put great pressure on decision makers. Think of all of the negative attack ads we see during elections. Studies have shown that they work. People remember and believe the bad things far more easily than the good things. This likely reflects the nature of our brains and the mechanisms developed over millennia for survival. It was much more important to be aware of the threats and dangers, which could be life threatening, than to dwell on where the berry patch is located. That priority no longer serves us, but it still functions, and so we pay attention to attack ads, lest we make a fatal mistake in our own behavior, such as voting for an undeserving candidate.

The same is true when it comes to criticizing proton therapy. It has an outsized effect. Would you like to be on the hook for a hundred million dollar investment if you read in a critical article somewhere that science has not yet made a definitive evaluation? Better to err on the side of caution. This phenomenon has raised its ugly head in a number of instances.

University Hospital Seidman Cancer Center in Cleveland, Ohio, has been successful in surpassing the eighteen patients per day breakeven point, averaging close to twenty five people. Yet Cleveland's other large health system, the Cleveland Clinic, has no plans to obtain proton therapy. Dr. John Suh, chairman of the Department of Radiation Oncology at the Clinic's Taussig Cancer Center, repeats the official mantra:

"The reason we have not actively pursued proton beam therapy is that the number of indications and the evidence for the use of proton beam therapy is still lacking."

He admits it might be appropriate for some specific populations of patients, especially younger patients, but believes there are not enough clinical studies proving protons are enough superior to x-rays to be worth the cost.

In September, 2018, an article in the Greater Baton Rouge Business Report illustrates my point again. The governor of Louisiana, John Bel Edwards, favors the development of proton therapy centers and the business he believes will result. On the other hand, Dr. Rebekah Gee, the state’s health secretary, is opposed because of the cost and because she believes there is “little evidence that it is effective.” Note that she is not concerned about being equivalent, she doubts that it is even effective. Certainly she has done no independent research on her own. Rather, she bought into the critic’s misinformation.

The planned $85 million Louisiana ProtonCare Center in Baton Rouge, announced with great fanfare, was among the biggest economic development announcements in 2017. Shockingly, only one month after plans were unveiled, Baton Rouge General pulled out of the partnership. CEO Edgardo Tenreiro said that the treatment had “not demonstrated superior outcomes for the vast majority of cancers.” That statement is so vague as to be meaningless. Vast majority of cancers? He should read my book, Proton Therapy: Revolutionary Treatment for 80% of ALL Cancers.

Similarly, plans by Provision CARES for a proton center in New Orleans fell through when the original partner bailed out, citing the same reason. Was this the work of Dr. Gee behind the scenes, spreading fear and uncertainty?

JOURNALISTIC CRITICISM

But the damage is done. The flawed result has been published widely and wholeheartedly embraced by critics. Worse yet, it gets into he press, where journalists are out of their depth of expertise. Here’s an example from one of the frequent Kaiser News hatchet jobs from journalist Jay Hancock:

But (proton therapy’s) pinpoint precision has not been shown to be more effective against breast, prostate and other common cancers. One recent study of lung-cancer patients found no significant difference in outcomes between people receiving proton therapy and those getting a focused kind of traditional radiation, which is much less expensive. Other studies are still underway.

There you have a direct reference to the flawed study. Another study often quoted is a Yale study of side effects and toxicities that has a similar bias against proton therapy. This is not a small thing. Hancock continues.

But a 2013 Yale study found little difference in those conditions among patients getting proton therapy versus those getting traditional radiation. Within a year, several insurers stopped covering the therapy for prostate cancer or were reconsidering it.

There you have it. That's the damage done by fake criticism. I’m sure those insurers were happy to use these supposed negative results to stop coverage for proton therapy. That’s why the integrity of these studies is so important. And although this study is six years old, with many more favorable ones since then, insurance companies and other critics continue to use it as one basis for their rejection of proton therapy.

Here’s another example from Paul Raeburn in MedPage Today. Actually, Raeburn seems like an interesting character, but he clearly doesn’t know anything about the subject on which he is writing. On May 19, 2017, he comments on the bankruptcy of the Scripps proton therapy center in San Diego.

“Whatever effect plasmas and protein (sic) beams might have on cancer, they were evidently not having the hoped-for effect on patients.”

Hello? Plasmas? Protein beams? (Sounds like an energy bar.) I guess he means the desired effect of having them show up. Hopefully he’s not denigrating proton therapy technology itself. Failing to do proper research, he makes this statement:

“This bleak economic landscape might look a little bit brighter if researchers could say with confidence that proton beam therapy has clear advantages over conventional radiology. If that is the case, then the centers that survive this downturn might one day pay for themselves. Unfortunately for proton beam therapy's purveyors and enthusiasts, there is little evidence of its superiority.”

We know that isn't true. Just visit any proton therapy center and see the wonderful results there. I know I’m being judgmental, and my enthusiasm may be just as biased as theirs, but I’m erring in the promotion of proton therapy, they are doing the opposite. Someone once asked me if I was giving people false hope. I replied that I would prefer that to giving false no hope. Yes, I'm gung ho for proton therapy. I'm an independent voice, beholding to no one. I think history will prove me right.