Tag: Research

Meta-research and medical skepticism

The Atlantic features an important article about “meta-researcher” Athina Tatsioni, who has published a number of influential papers about the quality of biomedical research:

He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed. His work has been widely accepted by the medical community; it has been published in the field’s top journals, where it is heavily cited; and he is a big draw at conferences…

He chose to publish one paper, fittingly, in the online journal PLoS Medicine, which is committed to running any methodologically sound article without regard to how “interesting” the results may be. In the paper, Ioannidis laid out a detailed mathematical proof that, assuming modest levels of researcher bias, typically imperfect research techniques, and the well-known tendency to focus on exciting rather than highly plausible theories, researchers will come up with wrong findings most of the time. Simply put, if you’re attracted to ideas that have a good chance of being wrong, and if you’re motivated to prove them right, and if you have a little wiggle room in how you assemble the evidence, you’ll probably succeed in proving wrong theories right.

This raises a number of important issues from a life extension perspective. For starters, these findings reinforce that it is just not credible for mainstream researchers and medical professionals to sustain these arrogant attitudes towards serious research efforts in life extension and cryonics. Good research is hard to do, and there is little of it.  This applies particularly to research that translates into meaningful medical benefits.  It  may be hard to swallow that a lot of  what constitutes conventional medicine is based on flawed studies and interest-driven research, but there is no escaping this conclusion.

Having a strong interest in the results is a double-edged sword. On the one hand, it makes one more susceptible to bias and cherry-picking. On the other hand, it can produce a determined mindset to tackle ambitious research goals (rejuvenation, vitrification). For example, the breakthroughs in vitrification technology at a company like 21st Century Medicine would have been unthinkable if the principal researchers would not have had an enduring strong personal interest in the technologies they are researching. This phenomenon can  also throw some light on the observation that often committed amateurs have more knowledge than professional researchers. Academic researchers often move from one (grant-funded) fad to another without obtaining a wide and deep understanding of the fields they are investigating. Unfortunately, such fashionable researchers are often featured in the media as “experts.”

Athina Tatsioni’s findings should also have a sobering effect on those who think we are in a period of accelerating medical progress. Even the  more credible medical research often fails to contribute to the expected clinical breakthroughs. To those familiar with the complex biochemistry of life, and the opportunity to introduce (long term) side-effects along with beneficial interventions (including attempts to just “repair” something), this should not be a surprise.

Naturally, Athina Tatsioni does not have a high opinion on research that claims benefits for vitamins and supplements:

For starters, he explains, the odds are that in any large database of many nutritional and health factors, there will be a few apparent connections that are in fact merely flukes, not real health effects—it’s a bit like combing through long, random strings of letters and claiming there’s an important message in any words that happen to turn up. But even if a study managed to highlight a genuine health connection to some nutrient, you’re unlikely to benefit much from taking more of it, because we consume thousands of nutrients that act together as a sort of network, and changing intake of just one of them is bound to cause ripples throughout the network that are far too complex for these studies to detect, and that may be as likely to harm you as help you. Even if changing that one factor does bring on the claimed improvement, there’s still a good chance that it won’t do you much good in the long run, because these studies rarely go on long enough to track the decades-long course of disease and ultimately death.

The take-home message is that skepticism is a useful disposition when looking at all research, medical practice, and triumphant claims about accelerating technological progress. One advantage of those who have made cryonics arrangements have is that they have time and, in theory (!), should be less prone to wishful thinking and jumping on the latest bandwagon. As Michael Anissimov writes, “When I talk to older transhumanists that are into cryonics, I see people who are psychologically calmer than those who endlessly obsess over their food, questionable supplements, and other minutiae that will mean jack squat if they get into a simple car accident.” It also reinforces the approach of arguing in favor of cryonics using skeptical arguments (about our arbitrary and evolving concepts of death)  instead of making bold claims about existing and future science.

Further reading: John P. A. Ioannidis – Why Most Published Research Findings Are False.

Robert Ettinger on cryonics and research

One of the most common criticisms of cryonics is to argue that cryonics can only be a legitimate endeavor when there is (peer reviewed) demonstration of whole body suspended animation. Advocates of cryonics  point out that this is an unreasonable position because it sets a standard for rational decision making (certainty) that is rarely encountered, if ever, in real life. People make decisions under conditions of uncertainty all the time. Why should cryonics be held to higher standards?

Like many other arguments against cryonics, this line of criticism is addressed by Robert Ettinger in his book Man into Superman (1972):

There is one more foible of many scientists and physicians important enough for separate attention: the notion that we should spend our money on research, not on cryonic suspension. This is nonsense on its face, and on the record.

To begin with, as repeatedly emphasized, those now dying cannot wait for more research, but must be given the benefit of whatever chance current methods offer. Most of us, if we are in our right minds, have limited interest in abstract humanity or remote posterity; we are primarily concerned with those near us, and cannot forego their probable physical benefit and certain psychological benefit. But even on their own terms, those who complain that research should come first are wrong.

Cryonics does not divert money from research, but channels money into research, and it is the only likely source of such funds in large amounts. Those who speak of using the funds for research “instead” of cryonics are out of touch with reality: these are not the alternatives. This is scarcely even arguable; it is a matter of record. Cryobiology has always been ill-supported, and in recent years support seems actually to have dwindled, partly because of a cutback in NASA funds. And private efforts to raise research money have had very little success. In contrast, organizations growing directly out of the cryonics program have donated money to cryobiological research without the help of a single big name: these include the Cryonics Societies of America, the Harlan Lane Foundation, and the Bedford Foundation. The sums involved have so far been very modest, but they will grow with the Societies. Note, for example, that Professor James Bedford, not a very wealthy man, left $100,000 of his estate for research in cryobiology and related areas, because he was planning cryonic suspension for himself. Does it require much imagination to see how this research will fare when people are being frozen by the thousands or by the millions?

Robert Ettinger makes another important point. Cryonics does not compete with resources for cryobiology research. If anything, it makes more money available to engage in such research. What better incentive to fund research than your own life being at stake? It is not a coincidence that the field of vitrification of complex organs has received great support from cryonics advocates. This funding has culminated in the identification of the least toxic vitrification agent known in the peer reviewed cryobiology literature and the maintenance of long-term potentiation (LTP) in vitrified brain slices.

So the idea that money should be allocated to research instead of cryonics is nonsense indeed. People can have rational reasons for choosing cryonics before suspended animation has been perfected. And when they make cryonics arrangements they have a stronger incentive to contribute to research that will benefit the science and practice of human cryopreservation.

The science of personal survival

There are various competing strategies how to achieve meaningful life extension or rejuvenation, including , but not limited to, genetic manipulation, periodical elimination of damage, caloric restriction,  molecular nanotechnology and mind uploading. A useful review of these strategies has been published in the book The Scientific Conquest of Death: Essays on Infinite Lifespans (2004) by the Immortality Institute. Most people will recognize that these strategies are not mutually exclusive. Some of them can be practiced right now (e.g., caloric restriction) and others ( e.g., periodical elimination of damage) could serve as a bridge to more comprehensive interventions such as a comprehensive genetic overhaul of human biology. As has often been recognized on this website, cryonics holds a special place among life extension strategies because it enables one to benefit from progress in the biomedical sciences that may not occur during one’s lifetime. We would like to think we can escape death by jumping from one successful biomedical innovation to another and that, of course, all the good things will happen in our lifetime, but reality often interferes with such optimism.

One thing that might greatly accelerate the pace of progress in the field of longevity science is the development of an integrated framework that studies the logical and empirical relationships among all these strategies. For example, a recent blog entry on the technical challenges surrounding chemopreservation of the brain triggered a meaningful private exchange about issues concerning the perfusion of ischemic tissue, empirical criteria for information-theoretic death, and the options for histological validation of cryonics technologies.  Such overlapping areas of investigation are plentiful and it would be helpful to explicate them.

Too much focus on “the big picture” can interfere with the identification of original ideas and rapid progress. Too little attention to the adverse effects of compartmentalization risks the waste of resources, which is not a trivial concern in the still poorly funded life extension community.

Reducing compartmentalization can have other sobering effects as well. For example, it is not unusual to see a group of researchers advocating a new approach to their field that is routine in other areas of investigation. For example, the idea that anti-aging research could benefit from less emphasis on illuminating the exact molecular mechanisms of aging and simply treat the observable manifestations of aging is no news to researchers in the field of cerebral ischemia. The pathophysiology of stroke is so complex that greater progress could be achieved by identifying clear targets for pharmacological intervention. But after decades of research it has become abundantly clear that such a paradigm change is no guarantee for more rapid progress. Despite this goal-oriented approach not one single neuroprotective agent has survived clinical trials.  This does not mean that such pragmatic approaches should be abandoned. It does mean, however, that research ideas should be evaluated on their empirical success and not just on their logical merits.

There are obvious examples where the claims in one field seem to make the claims in another field redundant. The most obvious example is the case of molecular nanotechnology. The projected timescales that are envisioned for this technology are not much different from the timescales that are envisioned by some anti-aging researchers to develop meaningful rejuvenation. In that case one could argue that (exclusive) preference should be given to those research programs that allow for the most comprehensive manipulation of biology. For example, a mature nanotechnology would be able to rejuvenate people, resuscitate cryonics patients, and alter the human endoskeleton to make us far less prone to fatal accidents. Such an argument would be a logical extension of the argument against devoting too much time to find treatments for specific age-related diseases instead of tackling aging itself.  Similar reasoning can be employed against anti-aging research. If accelerated change will bring the prospect of general molecular control within reach in the next few decades it makes little sense to spend vast amounts of time agonizing over specific anti-aging interventions. Why not just launch a “Manhattan Project” to pursue the much more comprehensive vision of molecular nanotechnology?

From a logical point of view, this is a persuasive argument. The limitations of such a perspective should now be obvious too.  We do not have certainty about the future of technological progress, let alone its specifics. As a matter of fact, in such matters it is not even evident how we should think about statistical or inductive probabilities.  To some people, the progress in one field is indicative of the progress we are going to observe in other fields, including fields in which there has been little progress to date. The problem with such naive inductivism is that it can just as well be used to  make the opposite case if a different reference class is chosen.

The logical empiricist philosopher Rudolf Carnap once wrote:

The acceptance or rejection of abstract linguistic forms, just as the acceptance or rejection of any other linguistic forms in any branch of science, will finally be decided by their efficiency as instruments, the ratio of the results achieved to the amount and complexity of the efforts required. To decree dogmatic prohibitions of certain linguistic forms instead of testing them by their success or failure in practical use, is worse than futile; it is positively harmful because it may obstruct scientific progress.

A related argument can be made about the science of personal survival. We should be cautious about privileging any line of research on  “logical” grounds. The fate of competing visions should be decided through empirical investigation.  This position should not be interpreted as saying that there is no place for logic in choosing research programs.  Logic has a central place in research design and interpretation of experimental observation but it cannot be solely relied upon a guide for decision making. Empirical observation disciplines thinking and ample room should be left for the unexpected. As Nassim Nicholas Taleb has pointed out:

There is a lot more randomness in biotechnology and any form of medical discovery. The role of design is overestimated. Every time we plan on trying to find a drug we don’t because it closes our mind. How are we discovering drugs? From the side-effects of other drugs.

Many experimental researchers have had the experience of engaging in research to find a solution to one problem but to discover the solution to another problem instead. Researchers who have recognized and embraced this phenomenon by becoming less fond of their own ideas and more open to run with such unexpected discoveries have reaped great benefits.

Selection bias and dietary supplements

One problem in assessing the merits of taking a specific dietary supplement (ranging from vitamins to  exotic multi-ingredient compounds) is widespread selection bias in the documentation that is supposed to support the use of the supplement in question.  The sheer number of scientific studies combined with variation in research methodologies virtually guarantees that for every supplement a supporting study can be found. For example, the recent issue of Life Extension Magazine (August 2008) has an article on the multiple health benefits of melatonin with 81 references. All these studies discuss either the biochemical properties of melatonin or show beneficial effects. This is what is what is seen. What is not seen are the studies in which melatonin is not effective or has adverse effects.  Or the studies that never got published as a result of “publication bias.” Granted, melatonin seems to be a remarkably effective agent for a diverse number of conditions, including its use as a neuroprotective agent in stroke, but such selective presentation of biomedical research seems to be a mainstay in the marketing of dietary supplements.

Another limitation of such documentation is that the studies that are used to recommend the taking of a supplement often solely address the (short-term) effects of that compound on the medical condition in question. Although it would not be practical to report on all the studies that investigate (chronic)  administration of the compound on other systems in the body, such unrelated adverse effects should not be ruled out when considering prolonged use. It is a major leap from demonstrating beneficial effects of a compound in rodents and preliminarily studies in humans to “recommending” the use of that compound for prolonged use in humans. And it is a giant leap to go from such studies to combining different effective compounds in very high dosages in a single product.

Promoting the use of supplements with a hodgepodge of  encouraging in-vitro studies, small animal studies, and observations in humans is not necessarily wrong, nor constitutes deliberate selection bias. Human biochemistry is extremely complex, and rigorous  research would require enormous resources and longitudinal experiments.  In real life there is a need to make informed decisions based on the evidence at hand. Still, our current state of knowledge and our ignorance about how all that we know adds up for specific individuals should induce modesty and, perhaps, moderation. For those who take supplements as a means to radical life extension, making cryonics arrangements remains the irreplaceable  cornerstone of such a program because it increases the odds to reach a time where truly meaningful (molecular) life extension technologies will be available, aside from the protection cryonics offers against most “lethal” accidents.

Albert Einstein's brain and information-theoretic death

“People like you and I, though mortal of course like everyone else, do not grow old no matter how long we live…[We] never cease to stand like curious children before the great mystery into which we were born.”

Albert Einstein

One sign of the lack of faith in the future progress of technology and the poor acceptance of the neurological basis for mind is the way in which our society treats the “post-mortem” human brain.

In some cases, the brains of those whom modern medicine cannot help are removed after cardiopulmonary arrest and donated (by the permission of the patient or the family) for research. In such cases, the brains are preserved so they can be studied over a long period of time. They are also sectioned and prepared in other ways for examination. Such donated brains have helped scientists learn about the human brain, with an eye to improving methods for treating conditions such as Alzheimer’s or mental illness. However, other brains have been preserved mainly because they belonged to famous people.

One of the more famous cases is the brain of Albert Einstein, removed in 1955 and preserved apparently without his or his family’s permission, and then made available for study. According to an NPR report, Einstein’s brain was fixed, sectioned into over 200 blocks, embedded in celloidin, and then stored in formalin.

Since that time, Einstein’s brain has been further sectioned and divided among researchers. A 1985 study by Diamond et al. reported that the Einstein brain sections’ neurons were still observable, and the study’s authors even assumed the number of neurons preserved in Einstein’s brain would be the same as those in recent preserved brains.

Presumably, people have wanted to study the brains of famous people in order to learn something about what made those people special. Turning a person into a mere object of study is a questionable notion, though, and the idea that the study could yield any information about the person’s mind underscores how it is widely accepted by scientists that the brain instantiates the mind, and thus the person.

Neuroscience is still too much in its infancy to make much sense of the evidence of the brain, as the scientific reception to the Diamond study showed. We do not yet know how to “read” the brain for the specific memories and personality traits and other phenomena of mind stored in it. However, because we do know enough now to know that the mind arises from the brain, we must realize that to preserve the brain is to preserve the potential of mind, and to preserve the potential of mind is to preserve the possibility of life for the person whose brain it was.

The neural basis of personhood sits ill with older notions such as immaterial souls or spirits. The neural basis of personhood also fits poorly with existing medical and public policies such as commonly accepted definitions of death and laws related to end of life. If death is understood as irreversible damage to certain identity-critical areas of the brain, the irreversibility of such damage is put into question by every advance in the treatment of injury and disease of the brain, as well as by the brain’s mysterious ability to recover from conditions such as minimally conscious state after many years. The cardiopulmonary-arrest definition of death does not involve the condition of the brain, and the usual definitions of brain-death do not distinguish between identity-critical areas or aspects of the brain and other areas or aspects of the brain. A more rigorous definition of personal death has been developed by Ralph Merkle, who states:

“A person is dead according to the information-theoretic criterion if their memories, personality, hopes, dreams, etc. have been destroyed in the information-theoretic sense. That is, if the structures in the brain that encode memory and personality have been so disrupted that it is no longer possible in principle to restore them to an appropriate functional state then the person is dead. If the structures that encode memory and personality are sufficiently intact that inference of the memory and personality are feasible in principle, and therefore restoration to an appropriate functional state is likewise feasible in principle, then the person is not dead.”

Although there is still some lack of clarity about the “etc.” and “appropriate functional state”, this definition of death at least is founded on the neural basis of personhood. Those who believe in the future progress of technology and accept the neural basis of personhood are led inevitably to understand that preserving the brain is preserving the person, potentially for later resuscitation.

It is not impossible to imagine that, in a more advanced future time, the formalin-fixed, celloidin-embedded brain sections could be reassambled, and if the synaptic circuitry of the neurons were well preserved, any significant damage could be repaired. The brain might be able to be returned to a viable state by reversal of the fixation and removal of the celloidin embedding. Resuscitation of an isolated brain would be unacceptable, but eventually it might be possible to restore the rest of the body around the brain by cloning or regeneration of the cells or some other prosthetic embodiment.

As amazing as it may seem, a patient reduced to a preserved brain, whose mind would be in a stopped state, might be able to be healed, that is, totally restored to a healthy body and a mind which could resume the life it left off, with all the memories and personality intact.

The case of Albert Einstein’s brain is unfortunate. All the impudent cutting, handing around, and tampering with Einstein’s brains sections, and the crude preservation method, may have irreversibly damaged the neural basis of his personhood. Yet we do not know enough today about the brain to know how much of it needs to be preserved, and in what state, to be able to revive a person with future technology. The preservation of the brain, though, would provide a theoretical possibility of future resuscitation. It may not be possible to someday restore Albert Einstein from the remains of his brain, but if it were possible, those in possession of the brain sections would first have to be willing to consider whether their “specimens” might be the restorable fragments of a still potentially living person who deserves to live more than to be studied.

Incomplete assessment of experimental cytoprotectants in rodent ischemia studies

In an effort to determine why so many cytoprotective treatments for stroke that are shown to be promising in laboratory animal experiments ultimately fail in human clinical trials, DeBow et al. performed an analysis of cytoprotection studies published in several leading journals. While noting that limitations in preclinical assessments also contribute to the premature advancement of some therapies to the clinic, their primary goal was to pinpoint deficiencies in experimental methodology of rodent ischemia studies that might lead to this inability to translate positive results from the bench to the bedside.

Because the stroke therapy academic industry roundtable (STAIR) issued a report in 1999 recommending certain improvements in stroke research, the authors decided to include literature beginning in 2000 up to the time of their analysis in 2002, to be compared with the representative literature in 1990. They identified all of the rodent ischemia articles published in the Journal of Neuroscience, the Journal of Cerebral Blood Flow and Metabolism, Experimental Neurology, and Stroke, including only studies that conformed to the following criteria: (1) used adult to aged rodents to assess global or focal ischemic insults, (2) tested a putative “cytoprotective” therapy, defined as one in which a therapy was administered or a manipulation was made (e.g., knockout mouse) and (3) the effects of that therapy were assessed on histological and/or behavioral outcome.

Following those criteria, the authors identified 19 and 20 global ischemia experiments for 1990 and 2000-2002, respectively. They identified 6 and 118 focal ischemia experiments for 1990 and 2000-2002, respectively. They further categorized these studies by rodent species, sex, and age. Methods used to measure intra- and postsurgery temperature were categorized according to location (rectal, temporalis muscle/tympanic, or brain temperature) and frequency (continual, frequent, infrequent, or none) of measurement. Survival time, measured in terms of hours or days following the start of ischemia, was categorized, as was behavioral evaluation (absent, neurologic deficit scale (NDS), or additional testing with or without NDS).

Almost all studies reviewed reported positive results. The vast majority of studies used male rodents, ignoring possible gender differences, and only two recent studies used old animals (> one year old), indicating that most models do not accurately reflect the typical human clinical situation. Additionally, the authors reported that a full eighty percent of recent (2000-2002) global ischemia experiments used survival times of ≤ seven days, while sixty-six percent of recent focal ischemia studies used survival times of ≤ 48 hours. Only 8.5% of focal ischemia studies examined histological outcome after seven days, essentially the same ratio observed in 1990. Furthermore, very few of the global ischemia studies, recent (2 in 20) or old (1 in 19), assessed behavior after ischemia. The authors comment:

In the recent focal ischemia studies, 55.1% did not assess functional outcome, 33.9% used NDS alone, and 11.0% used additional testing (e.g., skilled reaching) with or without a NDS. None of the 1990 studies used behavioral assessment as an endpoint.

Concerning temperature measurement, the authors found that the majority of global and focal ischemia studies used either rectal or core temperature measurements during ischemia without any other means of predicting brain temperature. Although they noted that some studies measured temporalis muscle (skull) temperature, very few studies directly measured brain temperature. Only 15% of recent global ischemia studies and 2.5% of recent focal ischemia studies utilized telemetry probes; they were not utilized at all in the 1990 papers surveyed.

Similarly, wide variations in postsurgical temperature measurement were reported. Three of the 19 global ischemia studies surveyed from 1990 reported rectal temperatures for up to 2, 6, or 24 hours. Three of the 20 global ischemia studies surveyed from 2000-2002 measured temperature continually with telemetry probes for at least 24 hours, while three other studies from the same time period sampled rectal temperature up to one or two hours following ischemia. In general, postsurgical temperature measurement was largely not performed. The authors report that “The percentage of cytoprotection studies in focal ischemia that measured temperature following surgical anesthesia, even if once, was only 0% and 33.0% for 1990 and 2000-2002, respectively.” Several other studies of both types of ischemia and from both time periods reported placement of animals in temperature-controlled rooms without measuring the animals’ temperatures.

It is not surprising, after reviewing these results, to learn that many of the “cytoprotectants” found to be beneficial in rodent ischemia studies go on to fail human clinical trials. This survey of rodent ischemia studies clearly demonstrates that most current experimental studies do not accurately represent clinical conditions of ischemia (e.g., aged animals) and have serious methodological limitations and flaws that will continue to contribute to clinical failures.

Especially concerning is the fact that most of these studies used exceedingly limited survival times, which are not sufficient to allow injury to mature fully. Exacerbating this issue is the fact that few studies assess behavior after treatment. Such a deficiency may lead an investigator to overestimate the benefit of their treatment since not all reductions in cell death will translate into improved functional outcome. Those studies that did include behavioral assessments typically used only a NDS soon after injury, resulting in a host of other difficulties in determination of actual cytoprotectant effect.

The importance of temperature in cerebral ischemic injury is well documented. Most studies assessed temperature during ischemia, but few measured postsurgical temperature, which is also known to substantially modify ischemic brain damage. The authors state:

Given these findings and the possibility of drug interactions, it is remarkable that most studies did not assess postsurgical temperature at all, including those using drugs known to affect temperature. Furthermore, of those that did, many only took rectal probe measurements for a short period following ischemia, or sampled temperature too infrequently (e.g., one sample at 24 hr after middle cerebral artery occlusion) or not long enough following drug administration (e.g., 15 min) to rule out temperature confounds.

While success rates of cytoprotectants to treat stroke depend not only on experimental design flaws but also limitations and flaws in clinical studies (let alone the relevance of rodent studies to humans), this review does much to show that many investigators have not taken it upon themselves to improve their study designs to avoid confounds or to better represent clinical conditions. Without these improvements, treatments based on such studies will continue to fail.

Because future changes or improvements to cryonics stabilization protocol may include results obtained from rodent ischemia research, general improvements in experimental design will benefit cryonics patients. More specific benefits may be achieved by using models that better reflect a typical cryonics case (e.g., warm ischemia followed by a period of low-flow reperfusion and concurrent temperature reduction). Of course, impeccable temperature monitoring is absolutely critical to such cryonics-specific models, allowing the researcher to control for temperature-related post-surgical pathologies and to better determine the effect of cytoprotective drugs vs. induction of hypothermia.