Seeing Patterns Where None Exists
Posted on February 20, 2007 Comments (17)
Seeing Patterns Where None Exists
I call data dredge studies the “Rorschach tests” of epidemiology, because researchers can pull out characteristics about people in almost unlimited combinations to find all sorts of correlations and conclude just about anything they set out to find. Just like the Rorschach test, seeing patterns where none exists, finding connections that are there but not as strongly as believed, and seeing what one expects to see, are common.
Page 8 of Statistics for Experiments by George Box, Willliam Hunter (my father) and Stu Hunter (no relation) shows a graph of the population (of people) versus the number of storks which shows a high correlation. “Although in this example few would be led to hypothesize that the increase in the number of storks caused the observed increase in population, investigators are sometimes guilty of this kind of mistake in other contexts.” And some might make it in this context 🙂
Related: Illusion of Explanatory Depth – Illusions, Optical and Other – Theory of Knowledge – Sarah, aged 3, Learns About Soap
17 Responses to “Seeing Patterns Where None Exists”
Leave a Reply
RSS Feed
June 19th, 2007 @ 8:59 am
We have tendencies that lead us to draw faulty conclusions from data. Given that it is important to understand what common mistakes are made to help us counter the natural tendencies…
October 30th, 2007 @ 7:44 pm
When people are asked to explain random variations in data they will make up special causes (that they often even believe are special causes even when they are not) but you can improve management a great deal by just stopping the requirement to “explain” common cause variation…
February 23rd, 2008 @ 9:07 pm
I must admit, when I first read the quote I thought that it must be an wrong. But there is the math. You save 111 gallons improving from 15 mpg to 18 mpg and just 100 improving from 50 to 100 mpg…
July 13th, 2008 @ 6:50 pm
actually having a 500 year flood actually increases the odds for it happening again (because the data now includes that case which had not been included before)…
September 29th, 2008 @ 9:28 am
“We will see that a computer can correct spelling mistakes, translate from Arabic to English, and recognize celebrity faces about as well as an average human—and can do it all by learning from examples rather than by relying on programming…”
December 2nd, 2008 @ 1:30 pm
“When we look at large quantities of information, what we’re really doing is searching for patterns. And being the kind of creatures that we are, and given the nature of the laws of probability, we are going to find patterns…”
January 1st, 2009 @ 8:39 am
“People born under the astrological sign of Leo are 15% more likely to be admitted to hospital with gastric bleeding than those born under the other 11 signs. Sagittarians are 38% more likely than others to land up there because of a broken arm…”
March 6th, 2009 @ 10:00 am
[…] of Knowledge – What Makes Scientists Different – Dangers of Forgetting the Proxy Nature of Data – Seeing Patterns Where None Exists by John Hunter Tags: Data, Fun, Science, Statistics Permalink to: Friday Fun: […]
June 16th, 2009 @ 9:24 am
[…] Illusions, Optical and Other – How Our Brain Resolves Sight – Seeing Patterns Where None Exists – Magenta is a Color – posts on scientific explanations of what we experience – Computational […]
September 2nd, 2009 @ 8:13 am
[…] Edward Tufte’s: Beautiful Evidence – Simpson’s Paradox – Correlation is Not Causation – Seeing Patterns Where None Exists – Great Charts – Playing Dice and Children’s Numeracy – Theory of Knowledge by curiouscat […]
October 26th, 2009 @ 8:36 am
[…] Great Charts – Simpson’s Paradox – Seeing Patterns Where None Exists – Visible Data – Control Charts – Edward Tufte’s: Beautiful Evidence by […]
December 14th, 2009 @ 8:42 am
[…] The management system is far more important than one person. Jim Press, Toyota N. American President, Moves to Chrysler (don’t expect much – Sept 2007). We are often fooled by randomness (understanding psychology lets you know this truth and factor it into your thinking): Illusions – Optical and Other, Attributing Random Results to a Special Cause, Seeing Patterns Where None Exists. […]
March 14th, 2010 @ 11:00 am
[…] Impact: 15 to 18 mpg or 50 to 100 mpg? – Meaningful debates need clear information – Seeing Patterns Where None Exists – Fooled by Randomness – Poor Reporting and Unfounded Implications – Illusion of […]
January 23rd, 2013 @ 5:38 am
[…] Leads to Incorrect Conclusions – Managing to Test Result Instead of Customer Value – Seeing Patterns Where None Exists – Confirmation […]
August 27th, 2013 @ 9:47 am
[…] think “random variation” is much more ordered than it is and we are able to find “patterns” in “random” data – so we can see “hot streaks” where none exists, because we don’t […]
April 7th, 2015 @ 9:23 am
[…] Correlation, of course, is not causation. Moz developed this flags by analyzing the correlation. So for example, a site is flagged as raising a spam flag if there is not contact info, or link to social accounts on the site. It could well be that Google pays no attention to this. So having the contact information or social links doesn’t reduce the true chance of a site being penalized by Google. But there is a correlation in that sites that are flagged by Google have a likelihood of not including those details. […]
April 22nd, 2018 @ 7:50 pm
[…] Stratify Data to Hone in on Special Causes of Problems – Seeing Patterns Where None Exists – How to Use Data and Avoid Being Mislead by Data – Don’t Forget the Proxy Nature […]