Remix.run Logo
Trials avoid high risk patients and underestimate drug harms(nber.org)
31 points by bikenaga 3 hours ago | 11 comments
randcraw 2 hours ago | parent | next [-]

It's understandable that unusual patients are seen as confounding variables in any study, especially those with small numbers of patients. Though I haven't read beyond the abstract, it also makes sense that larger studies (phase 3 or 4) should not exclude such patients, but perhaps could report results in more than one way -- including only those with the primary malady as well as those with common confounding conditions.

Introducing too many secondary conditions in any trial is an invitation for the drug to fail safety and/or efficacy due to increased demands on both. And as we all know, a huge fraction of drugs fail in phase 3 already. Raising the bar further, without great care, will serve neither patients nor business.

jrapdx3 32 minutes ago | parent [-]

Having been an "investigator" in a few phase 3 and 4 trials, it is true that all actions involving subjects must strictly follow protocols governing conduct of the trial. It is extremely intricate and labor intensive work. But the smallest violations of the rules can invalidate part of or even the entire trial.

Most trials have long lists of excluded conditions. As you say, one reason is reducing variability among subjects so effects of the treatment can be determined.

This is especially true when effects of a new treatment are subtle, but still quite important. If subjects with serious comorbidities are included, treatment effects can be obscured by these conditions. For example, if a subject is hospitalized was that because of the treatment or another condition or some interaction of the condition and treatment?

Initial phase 3 studies necessarily have to strive for as "pure" a study population as possible. Later phase 3/4 studies could in principle cautiously add more severe cases and those with specific comorbidities. However there's a sharp limit to how many variations can be systematically studied due to intrinsic cost and complexity.

The reality is that the burden of sorting out use of treatments in real-world patients falls to clinicians. It's worth noting level of support for clinicians reporting their observations has if anything declined over decades. IOW valuable information is lost in the increasingly bureaucratic and compartmentalized healthcare systems that now dominate delivery of services.

ortusdux an hour ago | parent | prev | next [-]

Tangentially related, but I was surprised to learn about the lax attitude towards placebos in trials. Classes of drugs have expected side effects, so it's common to use medications with similar effects as placebos. Last I heard, there is no requirement or expectation to document placebos used, and they are often not mentioned in publications.

Aurornis 4 minutes ago | parent [-]

> Classes of drugs have expected side effects, so it's common to use medications with similar effects as placebos.

This would be called an "active placebo" and would certainly be documented.

It's common to find controlled trials against an existing drug to demonstrate that the new drug performs better in some way, or at least is equivalent with some benefit like lower toxicity or side effects. In this case, using an active comparison against another drug makes sense.

You wouldn't see a placebo-controlled trial that used an active drug but called it placebo, though. Not only would that never get past the study review, it wouldn't even benefit the study operator because it would make their medication look worse.

In some cases, if the active drug produces a very noticeable effect (e.g. psychedelics) then study operators might try to introduce another compound that produces some effect so patients in both arms feel like they've taken something. Niacin was used in the past because it produces a flushing sensation, although it's not perfect. This is all clearly documented, though.

bikenaga 3 hours ago | parent | prev | next [-]

Abstract: "The FDA does not formally regulate representativeness, but if trials under-enroll vulnerable patients, the resulting evidence may understate harm from drugs. We study the relationship between trial participation and the risk of drug-induced adverse events for cancer medications using data from the Surveillance, Epidemiology, and End Results Program linked to Medicare claims. Initiating treatment with a cancer drug increases the risk of hospitalization due to serious adverse events (SAE) by 2 percentage points per month (a 250% increase). Heterogeneity in SAE treatment effects can be predicted by patient's comorbidities, frailty, and demographic characteristics. Patients at the 90th percentile of the risk distribution experience a 2.5 times greater increase in SAEs after treatment initiation compared to patients at the 10th percentile of the risk distribution yet are 4 times less likely to enroll in trials. The predicted SAE treatment effects for the drug's target population are 15% larger than the predicted SAE treatment effects for trial enrollees, corresponding to 1 additional induced SAE hospitalization for every 25 patients per year of treatment. We formalize conditions under which regulating representativeness of SAE risk will lead to more externally valid trials, and we discuss how our results could inform regulatory requirements."

refurb 44 minutes ago | parent [-]

This seems like an odd criticism.

First off it ignore the fact that if you include frail patients you’ll confound the results of the trial. So there is a good reason for it.

Second, saying “rate of SAE is higher than rate of treatment effect” is a bit silly considering these are cancer trial - without treatment there is a risk of death so most people are willing to accept SAE in order to achieve treatment effect.

Third, saying “the sickest patients saw the highest increase in SAE” seems obvious? It’s exactly what you’d expect.

nitwit005 an hour ago | parent | prev | next [-]

This covers the trials not being fully representative, but largely neglects why that is the case.

The paper defines a population "at high risk of drug-induced serious adverse events", which presumably means they're also the most likely people to be harmed or killed by the drug trial itself.

OutOfHere an hour ago | parent | prev | next [-]

Move generally, whenever you read the percentage of patients that are noted as having a particular side effect from a medicine, the real percentage is much higher.

SoftTalker an hour ago | parent [-]

And this just goes to reinforcing the beliefs of those who are skeptical of medical research. "Trust the science" is all well and good in theory except when the scientists are telling you a selective, cherry-picked story.

venturecruelty an hour ago | parent [-]

Strange how that line of thinking always winds up in places like "vaccines are bad" or "ivermectin cures COVID".

unethical_ban an hour ago | parent | prev [-]

This was a plot in an early season of ER.