Nothing is too wonderful to be true.

Michael Faraday (1791 – 1867)

Insight, untested and unsupported, is an insufficient guarantee of truth.

Bertrand Russell, Mysticism and Logic (1929)

Everyday we are bombarded with health claims and gadgets to “improve” our already healthy lifestyles. Statistics and facts are used to convince us to buy into the claim. For example, CFRA has played an hour-long infomercial on several Saturday afternoons that proclaims that your colon is infested with fecal material, parasites and toxins that cause almost every disease known to humans. The claimant states that John Wayne’s large abdominal girth was not due to fat but rather to 65 lbs of fecal material found on autopsy! Of course, their product will change all that for you. The interviewer does not ask even the most basic questions to challenge the claim.

The late Dr. Carl Sagan, astronomer, winner of the Pulitzer prize, author of “Contact”, the television series “Cosmos” and other award winning books wrote a book called “The Demon Haunted World —Science as a Candle in the Dark”. In one of the chapters he presents his “Baloney Detection Kit”.  Here is a brief outline.

• Wherever possible there must be independent confirmation of the facts. Never accept one source of information as proof of validity. Some of my obstetric patients are driven to distress by friends and family who proffer well-meaning but scientifically incorrect medical advice.
• Arguments from authority carry little weight. In science there are no “authorities”, only expert opinion. Do not be intimidated by the experts. Ask your questions.
• Try to come up with more than one explanation (hypothesis) for how something works. Whichever explanation stands up to rigorous testing is the one to be chosen.
• Try not to get too attached to a hypothesis because it is your own.
• Try to quantify if possible. If what you are explaining can be measured, it will allow you to compare it among competing explanations. Can you disprove the hypothesis using an unambiguous test?
• When someone proposes an explanation, every link in their argument must work not just some of it.
• Occam’s Razor: When there are several explanations for the same problem, the simpler explanation is usually correct one, e.g. Was John Wayne’s abdominal girth caused by fat or 65 lbs of fecal material?
• Always conduct “double blind” control experiments where the person taking the measurements is not aware of the test and control subjects.
• Confounding factors: Make sure that what is being measured does not interfere with another independent measurement, e.g. a study is set up to see if an acupressure bracelet or Gravol will effectively treat seasickness. If you are using both at the same time and your nausea is cured, which one is responsible for the desired effect? You have to test them separately under the same conditions to find out.

There are also common errors of logic and rhetoric made in claims.

• Ad hominem: making personal attacks against the arguer and not the argument, e.g. Father Smith is a born-again-Christian so his objection to evolution can be rejected. This is an unacceptable personal attack. It is evolution that is the issue not his belief. Ideas are to be respected and debated.
• Argument from adverse consequences: when the threat of some dire consequence is used to pressure the decision maker not to make an “unfavorable decision”, e.g.  You must take this medication to get better. If you do not, others will also refuse the treatment and die.
• Absence of evidence is not evidence of absence:  an appeal to ignorance. The claim that whatever has not been proved must be true and vice versa,  e.g. There is no evidence that homeopathic medications are bad for you therefore these medications are effective and good for you.
• Observational selection: counting the hits and forgetting the misses, e.g. an herbal medication helped 1,000 people but it did not help 30,000 with the same condition. The manufacture states that their product “helped hundreds of people” but says nary a word about the majority that it did not help.
• Statistics of small numbers: drawing conclusions from inadequate sample sizes
• Non sequitur: the logic of an argument does not follow, e.g. I will survive my cancer because I am a good person.
• Post hoc, ergo propter hoc – my favorite logical fallacy. It means “it happened after so it was caused by” There is a confusion of cause and effect, e.g. I had the flu shot and I got sick. The fact that over 100,000 people get their flu shot during cold season is overlooked. You could give an apple to the same people and some would get a cold. Would you say the apple made them sick?

It is important to have the tools to determine a claim’s validity. In fact, it can be an intellectually rewarding and satisfying experience. It is important for patients to be in control of their health care. I hope this kit will be a helpful tool for future health and life decisions.

We wait for light, but behold darkness.
Isaiah 59:9

It is better to light one candle than to curse the darkness. – Adage