The Atlantic features an important article about “meta-researcher” Athina Tatsioni, who has published a number of influential papers about the quality of biomedical research:

He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed. His work has been widely accepted by the medical community; it has been published in the field’s top journals, where it is heavily cited; and he is a big draw at conferences…

He chose to publish one paper, fittingly, in the online journal PLoS Medicine, which is committed to running any methodologically sound article without regard to how “interesting” the results may be. In the paper, Ioannidis laid out a detailed mathematical proof that, assuming modest levels of researcher bias, typically imperfect research techniques, and the well-known tendency to focus on exciting rather than highly plausible theories, researchers will come up with wrong findings most of the time. Simply put, if you’re attracted to ideas that have a good chance of being wrong, and if you’re motivated to prove them right, and if you have a little wiggle room in how you assemble the evidence, you’ll probably succeed in proving wrong theories right.

This raises a number of important issues from a life extension perspective. For starters, these findings reinforce that it is just not credible for mainstream researchers and medical professionals to sustain these arrogant attitudes towards serious research efforts in life extension and cryonics. Good research is hard to do, and there is little of it.  This applies particularly to research that translates into meaningful medical benefits.  It  may be hard to swallow that a lot of  what constitutes conventional medicine is based on flawed studies and interest-driven research, but there is no escaping this conclusion.

Having a strong interest in the results is a double-edged sword. On the one hand, it makes one more susceptible to bias and cherry-picking. On the other hand, it can produce a determined mindset to tackle ambitious research goals (rejuvenation, vitrification). For example, the breakthroughs in vitrification technology at a company like 21st Century Medicine would have been unthinkable if the principal researchers would not have had an enduring strong personal interest in the technologies they are researching. This phenomenon can  also throw some light on the observation that often committed amateurs have more knowledge than professional researchers. Academic researchers often move from one (grant-funded) fad to another without obtaining a wide and deep understanding of the fields they are investigating. Unfortunately, such fashionable researchers are often featured in the media as “experts.”

Athina Tatsioni’s findings should also have a sobering effect on those who think we are in a period of accelerating medical progress. Even the  more credible medical research often fails to contribute to the expected clinical breakthroughs. To those familiar with the complex biochemistry of life, and the opportunity to introduce (long term) side-effects along with beneficial interventions (including attempts to just “repair” something), this should not be a surprise.

Naturally, Athina Tatsioni does not have a high opinion on research that claims benefits for vitamins and supplements:

For starters, he explains, the odds are that in any large database of many nutritional and health factors, there will be a few apparent connections that are in fact merely flukes, not real health effects—it’s a bit like combing through long, random strings of letters and claiming there’s an important message in any words that happen to turn up. But even if a study managed to highlight a genuine health connection to some nutrient, you’re unlikely to benefit much from taking more of it, because we consume thousands of nutrients that act together as a sort of network, and changing intake of just one of them is bound to cause ripples throughout the network that are far too complex for these studies to detect, and that may be as likely to harm you as help you. Even if changing that one factor does bring on the claimed improvement, there’s still a good chance that it won’t do you much good in the long run, because these studies rarely go on long enough to track the decades-long course of disease and ultimately death.

The take-home message is that skepticism is a useful disposition when looking at all research, medical practice, and triumphant claims about accelerating technological progress. One advantage of those who have made cryonics arrangements have is that they have time and, in theory (!), should be less prone to wishful thinking and jumping on the latest bandwagon. As Michael Anissimov writes, “When I talk to older transhumanists that are into cryonics, I see people who are psychologically calmer than those who endlessly obsess over their food, questionable supplements, and other minutiae that will mean jack squat if they get into a simple car accident.” It also reinforces the approach of arguing in favor of cryonics using skeptical arguments (about our arbitrary and evolving concepts of death)  instead of making bold claims about existing and future science.

Further reading: John P. A. Ioannidis – Why Most Published Research Findings Are False.