The single biggest problem with what are called "observational studies" is that you generally can't be certain about which variable or combination of variables is responsible for the positive or negative effects you're observing.
This was recently illustrated by a couple of dumb studies. The second of those links is especially egregious, implicating red meat, a staple of humans and their ancestors going back 2.5 million years while the modern grains, sugars and vegetable oils that have been in the diet only recently are given a pass.
Stephan at Whole Health Source came up with a keen observation the other day in this regard.
In other words, the reason observational studies in affluent nations haven't been able to get to the bottom of dental/orthodontic problems and chronic disease is that everyone in their study population is doing the same thing! There isn't enough variability in the diets and lifestyles of modern populations to be able to determine what's causing the problem. So we study the genetics of problems that are not genetic in origin, and overestimate genetic contributions because we're studying populations whose diet and lifestyle are homogeneous. It's a wild goose chase.
Here's another way to look at it, by means of a joke Beatrice emailed the other day.
~~~
I was shopping at the local supermarket where I selected:
A half-gallon of 2% milk
A carton of eggs
A quart of orange juice
A head of lettuce
A 2 lb. can of coffee
A 1 lb. package of bacon
As I was unloading my items on the conveyor belt to check out, a drunk standing behind me watched as I placed the items in front of the cashier. While the cashier was ringing up the purchases, the drunk calmly stated, 'You must be single.'
I was a bit startled by this proclamation, but I was intrigued by the derelict's intuition, since I indeed had never found Mr. Right. I looked at the six items on the belt and saw nothing particularly unusual about my selections that could have tipped off the drunk to my marital status.
Curiosity getting the better of me, I said, 'Yes, you are correct. But how on earth did you know that?'
The drunk replied, 'Cause you're ugly.'
~~~
Of course, that suggests another problem in addition to an unconsidered variable: you might be biased against one of more or them.






The story from the single woman was freakin hilarious! Funny you should post on the topic of hidden or missed variables, I'm sitting on a dissertation committee where the student's topic is "Mediation by unobserved variables in causal inference". It's about how people come up with causal hypotheses to explain events they witness (i.e., data they collect), and in particular how people do or fail to entertain a role for hidden (i.e., unobserved) variables. Also, I myself have published a study (and have more in the pipeline) showing that even rats entertain hidden variables under certain conditions (PDF freely available on my website: pigeonrat.psych.ucla.edu). I think I'd rather have one of my rats making decisions about what I should and should not eat than anyone on the AMA or AHA boards of VIPs (Very Incompetent Phatheads).
This is priceless! And illustrates the point beautifully!!!
Thank you.
Most everyone thinks I'm nuts in regards to how much red meat I eat. They have it as a special treat once or twice a year. Oh well…
Marc
Awesome joke!! hahaahahahahahahah! I loved it! Thank you for making me laugh!
My only concern with convincing the population that red meat is good is that it will at least initially raise the price of my ribeyes, tri-tips, etc…
I can help offset that, somewhat. Let me know next time you're in town, cousin, and I'll grill something up.
Looking forward to it!
Once again, stellar post.
[So we study the genetics of problems that are not genetic in origin, and overestimate genetic contributions because we're studying populations whose diet and lifestyle are homogeneous. It's a wild goose chase.]
That wild goose chase is tantamount to taking the blue pill over and over again and always eschewing the red pill.
If that doesn't make sense to you, see here:
http://theorytopractice.wordpress.com/2009/05/30/120-grams-of-sugar-oh-my/#comment-1918
Very funny..I almost choked to death on my water from trying to laugh and swallow. I really like the blog.
Melina
http://wishfulthinking-melina.blogspot.com/