Seeing Patterns Where None Exists

Posted on February 20, 2007  Comments (15)

Seeing Patterns Where None Exists

I call data dredge studies the “Rorschach tests” of epidemiology, because researchers can pull out characteristics about people in almost unlimited combinations to find all sorts of correlations and conclude just about anything they set out to find. Just like the Rorschach test, seeing patterns where none exists, finding connections that are there but not as strongly as believed, and seeing what one expects to see, are common.

Page 8 of Statistics for Experiments by George Box, Willliam Hunter (my father) and Stu Hunter (no relation) shows a graph of the population (of people) versus the number of storks which shows a high correlation. “Although in this example few would be led to hypothesize that the increase in the number of storks caused the observed increase in population, investigators are sometimes guilty of this kind of mistake in other contexts.” And some might make it in this context :-)

Related: Illusion of Explanatory DepthIllusions, Optical and OtherTheory of KnowledgeSarah, aged 3, Learns About Soap

15 Responses to “Seeing Patterns Where None Exists”

  1. CuriousCat: Mistakes in Experimental Design and Interpretation
    June 19th, 2007 @ 8:59 am

    We have tendencies that lead us to draw faulty conclusions from data. Given that it is important to understand what common mistakes are made to help us counter the natural tendencies…

  2. Curious Cat Management Improvement Blog » Fooled by Randomness
    October 30th, 2007 @ 7:44 pm

    When people are asked to explain random variations in data they will make up special causes (that they often even believe are special causes even when they are not) but you can improve management a great deal by just stopping the requirement to “explain” common cause variation…

  3. Curious Cat » Bigger Impact: 15 to 18 mpg or 50 to 100 mpg?
    February 23rd, 2008 @ 9:07 pm

    I must admit, when I first read the quote I thought that it must be an wrong. But there is the math. You save 111 gallons improving from 15 mpg to 18 mpg and just 100 improving from 50 to 100 mpg…

  4. CuriousCat: 500 Year Floods
    July 13th, 2008 @ 6:50 pm

    actually having a 500 year flood actually increases the odds for it happening again (because the data now includes that case which had not been included before)…

  5. Curious Cat Management Improvement Blog » Does the Data Deluge Make the Scientific Method Obsolete?
    September 29th, 2008 @ 9:28 am

    “We will see that a computer can correct spelling mistakes, translate from Arabic to English, and recognize celebrity faces about as well as an average human—and can do it all by learning from examples rather than by relying on programming…”

  6. CuriousCat: How Large Quantities of Information Change Everything
    December 2nd, 2008 @ 1:30 pm

    “When we look at large quantities of information, what we’re really doing is searching for patterns. And being the kind of creatures that we are, and given the nature of the laws of probability, we are going to find patterns…”

  7. Curious Cat Science Blog » Correlation is Not Causation
    January 1st, 2009 @ 8:39 am

    “People born under the astrological sign of Leo are 15% more likely to be admitted to hospital with gastric bleeding than those born under the other 11 signs. Sagittarians are 38% more likely than others to land up there because of a broken arm…”

  8. Curious Cat Management Improvement Blog » Friday Fun: Correlation
    March 6th, 2009 @ 10:00 am

    [...] of Knowledge – What Makes Scientists Different – Dangers of Forgetting the Proxy Nature of Data – Seeing Patterns Where None Exists by John Hunter   Tags: Data, Fun, Science, Statistics   Permalink to: Friday Fun: [...]

  9. Curious Cat Science and Engineering Blog » Albert Einstein, Marylin Monroe Hybrid Image
    June 16th, 2009 @ 9:24 am

    [...] Illusions, Optical and Other – How Our Brain Resolves Sight – Seeing Patterns Where None Exists – Magenta is a Color – posts on scientific explanations of what we experience – Computational [...]

  10. Curious Cat Science and Engineering Blog » The Value of Displaying Data Well
    September 2nd, 2009 @ 8:13 am

    [...] Edward Tufte’s: Beautiful Evidence – Simpson’s Paradox – Correlation is Not Causation – Seeing Patterns Where None Exists – Great Charts – Playing Dice and Children’s Numeracy – Theory of Knowledge by curiouscat [...]

  11. Curious Cat Management Improvement Blog » Communicating with the Visual Display of Data
    October 26th, 2009 @ 8:36 am

    [...] Great Charts – Simpson’s Paradox – Seeing Patterns Where None Exists – Visible Data – Control Charts – Edward Tufte’s: Beautiful Evidence by [...]

  12. Curious Cat Management Improvement Blog » The CEO is Only One Person
    December 14th, 2009 @ 8:42 am

    [...] The management system is far more important than one person. Jim Press, Toyota N. American President, Moves to Chrysler (don’t expect much – Sept 2007). We are often fooled by randomness (understanding psychology lets you know this truth and factor it into your thinking): Illusions – Optical and Other, Attributing Random Results to a Special Cause, Seeing Patterns Where None Exists. [...]

  13. Curious Cat Science and Engineering Blog » Statistical Errors in Medical Studies
    March 14th, 2010 @ 11:00 am

    [...] Impact: 15 to 18 mpg or 50 to 100 mpg? – Meaningful debates need clear information – Seeing Patterns Where None Exists – Fooled by Randomness – Poor Reporting and Unfounded Implications – Illusion of [...]

  14. Leanpub Podcast on My Book – Management Matters: Building Enterprise Capability » Curious Cat Management Improvement Blog
    January 23rd, 2013 @ 5:38 am

    [...] Leads to Incorrect Conclusions – Managing to Test Result Instead of Customer Value – Seeing Patterns Where None Exists – Confirmation [...]

  15. Stu Hunter Discussing Bill Hunter, Statistics for Experimenters and EVOP » Curious Cat Management Improvement Blog
    August 27th, 2013 @ 9:47 am

    […] think “random variation” is much more ordered than it is and we are able to find “patterns” in “random” data – so we can see “hot streaks” where none exists, because we don’t […]

Leave a Reply