British doctor, blogger and the author of the book Bad Science: Quacks, Hacks, and Big Pharma Flacks Ben Goldacre describes one of the most serious problems in medicine today: a flood of bad science. This ranges from false health and dietary claims coming from the self-proclaimed "experts" to withholding of evidence by pharma companies. Goldacre gives a few examples of bad science, in the order from easiest to hardest to spot:
- Studies that find association, but claim causation. Many poorly done observational studies fall into this category. Example: an observational study showed that people who eat more fruits and vegetables and olive oil also have fewer skin wrinkles, and claimed that there is a causative link between these. However, people who eat this kind of diet are also more likely to have indoor jobs, smoke less, etc, and this whole host of factors may influence wrinkle formation.
- Studies without a control group. Example: in one of the studies, researchers found that giving school kids fish oil pills results in higher school test scores one year later, and concluded that fish oil improves school performance. But the study was not controlled for children simply getting better over a year without any fish oil, nor was it controlled for the placebo effect.
- Studies with an inadequate control group. Example: frequently, clinical trials of new drugs are done with a placebo control. However, a proper control group in many cases would be the best currently available drug. Other examples include trials with a control group using the best currently available drug in too low or too high dosage. All of the above makes the new drug appear better. Not surprisingly, industry-funded trials are 4 times more likely to give positive results than independently-funded trials.
- Publication bias (by journals) and withholding studies (by industry), which results in under-reporting of the negative studies (studies which show that a drug is less effective or has more side effects then previously thought). Example: for many widely known drugs, such as antidepressants or the flu drug Tamiflu, a significant fraction of the studies (50-75% in many cases) have been withheld by the industry. This is a serious problem: if you withhold half of the results of a fair coin toss, you can convince people that the coin has 2 heads!