How to avoid getting duped by bad research, in 5 case studies

If you want to know for sure whether something works, you have to design a good test. Yet every day, many studies are published that are so poorly designed that you shouldn’t even wipe your table with them, let alone use them to inform personal health or policy decisions.

Some key problems they looked at include:

Healthy user bias: when studies compare healthier users of a medical intervention with less-healthy people who didn’t get the intervention. This can make medical treatments seem more effective than they really are. For example, people who seek out and receive a flu shot might be healthier overall than those who don’t. If you compare the two groups in a study, “healthy user bias” could give the false picture that it’s the vaccine that made the difference. As they explain, “Healthy user bias is a type of selection bias that occurs when investigators fail to account for the fact that individuals who are more health conscious and actively seek treatment are generally destined to be healthier than those who do not.”

Confounding by indication: This is another way the true cause of an outcome can get hidden. This type of bias can “occur because physicians choose to preferentially treat or avoid patients who are sicker, older, or have had an illness longer,” the authors write. “In these scenarios, it is the trait (eg, dementia) that causes the adverse event (eg, a hip fracture), not the treatment itself (eg, benzodiazepine sedatives).” In other words, it wasn’t necessarily the drugs that caused the hip fractures; it was that doctors tended to prescribe the drugs to populations (such as older folks) who are more likely to suffer hip fractures, regardless of what they are taking.

To learn how to spot three other classic biases, read the full guide here.