Failing to Report Adverse Effects of Treatments

We have frequently advocated the evidence-based medicine (EBM) approach to improve the care of individual patients, and to improve health care quality at a reasonable cost for populations. Evidence-based medicine is not just medicine based on some sort of evidence. As Dr David Sackett, and colleagues wrote [Sackett DL, Rosenberg WM, Muir Gray JA, Haynes RB, Richardson WS. Evidence-based medicine; what it is and what it isn't. BMJ 1996; 312: 71-72. Link here. ]

Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.

One can find other definitions of EBM, but nearly all emphasize that the approach is designed to appropriately apply results from the best clinical research, critically reviewed, to the individual patient, taking into account that patient's clinical characteristics and personal values.

When making decisions about treatments for individual patients, the EBM approach suggests using the best available evidence about possible benefits and harms of treatment, so that the treatment chosen is most likely to maximize benefits and minimize harms for the individual patient. The better the evidence about specific benefits and harms applicable to a particular patient, the greater will be the likelihood that a particular decision based on this evidence will result in the best possible outcomes for the patient.

A new study in the Archives of Internal Medicine focused on how articles report adverse effects found by clinical trials. [Pitrou I, Boutron I, Ahmad N et al. Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med 2009; 169: 1756-1761. Link here.] The results were not encouraging.

The investigators assessed 133 articles reporting the results of randomized controlled trials published in 2006 in six English language journals with high impact factors, that is, the most prestigious journals, including the New England Journal of Medicine, Lancet, JAMA, British Medical Journal, and Annals of Internal Medicine. They excluded trials with less common designs, such as randomized cross-over trials. The majority of trials (54.9%) had private, or private mixed with public funding.

The major results were:
15/133 (11.3%) did not report anything about adverse events
36/133 (27.1%) did not report information about the severity of adverse events
63/133 (47.4%) did not report how many patients had to withdraw from the trial due to adverse events
43/133 (32.3%) had major limitations in how they reported adverse events, e.g., reporting only the most common events (even though most trials do not enroll enough patients to detect important but uncommon events).

The authors concluded, "the reporting of harm remains inadequate."

An accompanying editorial [Ioannidis JP. Adverse events in randomized controlled trials: neglected, distorted, and silenced. Arch Intern Med 2009; 169: 1737-1739. Link here] raised concerns about why the reporting of adverse events is so shoddy:

Perhaps conflicts of interest and marketing rather than science have shaped even the often accepted standard that randomized trials study primarily effectiveness, whereas information on harms from medical interventions can wait for case reports and nonrandomized studies. Nonrandomized data are very helpful, but they have limitations, and many harms will remain long undetected if we just wait for spontaneous reporting and other nonrandomized research to reveal them. In an environment where effectiveness benefits are small and shrinking, the randomized trials agenda may need to reprogram its whole mission, including its reporting, toward better understanding of harms.

Pitrou and colleagues have added to our knowledge about the drawbacks of the evidence about treatments that is publicly available to physicians and patients when making decisions about treatment. Even reports of studies with the best designs (randomized controlled trials) in the best journals seem to omit important information about the harms of the treatments they test.

It appears that the majority of the reports that Pitrou and colleagues studied received "private" funding, presumably meaning most were funded by drug, biotechnology, or device companies and were likely meant to evaluate the sponsoring companies' products. However, note that this article did not analyze the relationship of funding source to the completeness of information about adverse effects.

Nonetheless, on Health Care Renewal we have discussed many cases in which research has been manipulated in favor of the vested interests of research sponsors (funders), or in which research unfavorable to their interests has been suppressed. Therefore, it seems plausible that sponsors' influence over how clinical trials are designed, implemented, analyzed and reported may reduce information about the adverse effects of their products reported in journal articles. Trials may be designed not to gather information about adverse events. Analyses of some adverse events, or some aspects of these events may not be performed, or if performed, not reported. The evidence from clinical research available to make treatment decisions consequently may exaggerate the ratios of certain drugs' and devices' benefits to their harms.

Patients may thus receive treatments which are more likely to hurt than to help them, and populations of patients may be overtreated. Impressions that treatments are safer than they actually are may allow their manufacturers to overprice them, so health care costs may rise.

The article by Pitrou and colleagues adds to concerns that we physicians may too often really be practicing pseudo-evidence based medicine when we think we are practicing evidence-based medicine. We cannot judiciously balance benefits and harms of treatments to make the best decisions for patients when evidence about harms is hidden. Clearly, as Ioannidis wrote, we need to "reprogram." However, what we need to reprogram is our current dependence on drug and device manufacturers to pay for (and hence de facto run) evaluations of their own products. If health care reformers really want to improve quality while controlling costs, this is the sort of reform they need to start considering.

NB - See also the comments by Merrill Goozner in the GoozNews blog.

»»  READMORE...

High Costs and Poor Outcomes - A Lesson About the Suppression of Research

Last week, blogger Alison Bass was one of the few to comment on a new article on the risks of antidepressant medications published electronically last week by the British Medical Journal.[1] The article adds to the clinical evidence about these specific drugs. However, it also should remind us about why our health care costs seem to continue to inflate disproportionate to any improvements in health care outcomes.

Background - Why Don't Doctors Diagnose and Treat Depression According to the Evidence?

Here is the background. Since the early 1990s, based on numerous published clinical trials that showed that the newer (mainly SSRI, that is, selective serotonin reuptake inhibitor) antidepressants seemed to be quite safe and effective, authoritative guidelines have urged physicians to screen patients for depression, and have a low threshold for beginning treatment, particularly with these then new drugs. (For example, see the 1993 AHCPR guidelines,[2] and the Ambulatory Care Quality Alliance Recommended Starter Set[3].) Such guidelines targeted primary care physicians as well as psychiatrists. As a primary care physician, I often felt guilty that I didn't screen all patients, including those who did not outwardly appear depressed, for depression, and was reluctant to be the primary prescriber of antidepressants for depressed patients. I was not alone. There was evidence that many primary care physicians failed to follow these apparently evidence-based guidelines.

Suspicions that SSRIs Were Not as Effective or Safe as Thought

However, in 2004, internal documents from one drug company (Glaxo Smith Kline), published by the Canadian Medical Association Journal, suggested that the company had suppressed a study which failed to show its SSRI was effective for adolescent patients.[4] Also, then New York state attorney general Elliot Spitzer sued the company alleging it had suppressed such data.[5] Considerable anecdotal information thereafter came to light suggesting that drug companies may have suppressed data from clinical research (that is, studies done on patients, not in the laboratory) which tended to show the drugs were less effective, or more dangerous than previously thought. Many began to suspect that the drugs increased the risk of suicidal thoughts or behavior, even though one would think that if they were effective in treating depression, they should decrease such risks.

How Does Uncovering Suppressed Trials Change the Evidence About the Benefits and Harms of SSRIs?

Then, an analysis of trials of SSRIs given to adolescent patients, which included unpublished as well as published trials, did show they increased the risk of suicide.[6] But what about adult patients?

In 2008, Erick Turner and colleagues published a meta-analysis that included numerous trials of SSRIs and other antidepressants given to adult patients that had never been published, but whose results had been submitted to the US Food and Drug Administration (FDA).[7] When those trials were included in the analysis, the drugs seemed less effective than was previously believed.

The new meta-analyis by Stone et al[1] suggested how adding evidence from the suppressed trials improves our knowledge about the possible harms of these drugs. Stone et al specifically addressed whether the drugs increased the risk of suicidal thoughts or actions for adult patients. It was based on data from 372 trials that included 99,231 patients. "Most of the studies were unpublished; those that had been published in some form seldom contained information concerning suicidality in the publication." Its main result was that SSRI drugs approximately doubled the risk of suicidal behavior for patients less than 25 years old, but decreased the risk for older patients.

Suppressing Trials Distorted the Evidence-Base

Thus, the combined results of several meta-analyses suggests that SSRIs are neither as safe nor as effective for adult patients as they were previously touted to be. The drugs used to appear to be more safe and effective because the drug companies that sponsored (paid for) clinical trials of their own products suppressed the publication of studies whose results did not show their products in a favorable light. The guidelines once thought to be so authoritative are probably much to enthusiastic about the use of drugs to treat depression. It may be that those physicians who resisted pressure from these guidelines and other efforts by would-be quality improvers to diagnose more depressed patients and treat them with SSRIs weren't so dumb.

Health Policy Implications

The recently published meta-analysis of the risk of suicidal ideation or behavior due to SSRIs for adult patients[1] adds not just to the clinical evidence about the treatment of depression, but to the evidence that suppression of research by those with vested interests in its results may not be rare. There is no reason to think that suppression of research about other medical interventions used for other clinical problems does not occur just as often.

This underscores why we as a society should not have handed over control of how new drugs and devices are evaluated to the companies that stand to profit by selling them. The apparent benefits and safety of most drugs and devices brought to the market in recent years may have been exaggerated by the ability of the corporations who sponsored the research to suppress those studies that were least favorable to their products. Thus, it is very likely that doctors have over prescribed newer (and more expensive) drugs and devices based on erroneous beliefs about their safety and effectiveness, beliefs supported by an evidence base which was distorted by suppression of studies whose results did not show the results their commercial sponsors wanted to see. Furthermore, we have likely collectively paid too much for these drugs and devices based on these same erroneous beliefs. Thus, the distortion of the clinical evidence base caused by putting the makers of medical products in charge of their evaluation has likely led to increased costs and worse outcomes for patients.

Health care reformers serious about wanting to improve patient outcomes and reduce costs ought to be at the forefront of a movement to make clinical research more honest. All clinical trials completed and submitted to government regulators ought to be published in some form, so that the data they contain is available to guide clinical decisions. Going forward, all studies should be published in some form in a timely manner to prevent future research suppression. There ought to be better regulation of research sponsored, and de facto run by companies who have vested interests in the results being favorable to their products. It would not be unreasonable to prevent companies with such vested interests from running or otherwise directly influencing clinical research meant to evaluate their own products.

But here in the States, we are probably too diverted by yelling at each other about whether a "public option" is vital or nefarious to worry about such issues.. Meanwhile, expect our health care to continue to get more expensive, less accessible, and less good for patients.

ADDENDUM (25 August, 2009) - Also, see the comments by Dr Howard Brody on the Hooked: Ethics, Medicine and Pharma Blog

References

1. Stone M, Laughren T, Jones ML et al. Risk of suicidality in clinical trials of antidepressants in adults: analysis of proprietary data submitted to US Food and Drug Administration. Brit Med J 2009; 339: b2880. Link here.
2. Depression Guideline Panel. Depression in Primary Care: Volume 2 - Treatment of Major Depression. Rockville, MD: US Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research, 1993. Link here.
3. Ambulatory Care Quality Alliance. Recommended Starter Set: Clinical Performance Measures for Ambulatory Care. http://www.ahrq.gov/qual/aqastart.htm
4. Kondro W. Drug company experts advised staff to withhold data about SSRI use in children. Can Med Assoc J 2004; 170: 783. Link here.
5. Wadman M. Sptizer sues drug giant for deceiving doctors. Nature 2004; 429: 589. Link here.
6. Hammad T, Laughren T, Racoosin JA. Suicidality in pediatric patients treated with antidepressant drugs. Arch Gen Psychiatry 2006;63:332-9. Link here.
7. Turner EH, Matthews AM, Linardatos E et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008; 358: 252-260. Link here.

»»  READMORE...