Posts about statistics
animals appropriate technology bacteria biology Career Cats cool curiouscat economy Energy Engineering engineering education engineers evolution experiment food fun Funding government green Health Care human health innovation K-12 kids Life Science medical research nature ocean physics Products Research Robots Science science education science explained science facts science webcasts scientific inquiry Students Technology university research USA water webcasts

Statistical Errors in Medical Studies

Posted on March 14, 2010  Comments (4)

I have written about statistics, and various traps people often fall into when examining data before (Statistics Insights for Scientists and Engineers, Data Can’t Lie – But People Can be Fooled, Correlation is Not Causation, Simpson’s Paradox). And also have posted about reasons for systemic reasons for medical studies presenting misleading results (Why Most Published Research Findings Are False, How to Deal with False Research Findings, Medical Study Integrity (or Lack Thereof), Surprising New Diabetes Data). This post collects some discussion on the topic from several blogs and studies.

HIV Vaccines, p values, and Proof by David Rind

if vaccine were no better than placebo we would expect to see a difference as large or larger than the one seen in this trial only 4 in 100 times. This is distinctly different from saying that there is a 96% chance that this result is correct, which is how many people wrongly interpret such a p value.
…
So, the modestly positive result found in the trial must be weighed against our prior belief that such a vaccine would fail. Had the vaccine been dramatically protective, giving us much stronger evidence of efficacy, our prior doubts would be more likely to give way in the face of high quality evidence of benefit.
…
While the actual analysis the investigators decided to make primary would be completely appropriate had it been specified up front, it now suffers under the concern of showing marginal significance after three bites at the statistical apple; these three bites have to adversely affect our belief in the importance of that p value. And, it’s not so obvious why they would have reported this result rather than excluding those 7 patients from the per protocol analysis and making that the primary analysis; there might have been yet a fourth analysis that could have been reported had it shown that all important p value below 0.05.

How to Avoid Commonly Encountered Limitations of Published Clinical Trials by Sanjay Kaul, MD and and George A. Diamond, MD

Trials often employ composite end points that, although they enable assessment of nonfatal events and improve trial efficiency and statistical precision, entail a number of shortcomings that can potentially undermine the scientific validity of the conclusions drawn from these trials. Finally, clinical trials often employ extensive subgroup analysis. However, lack of attention to proper methods can lead to chance findings that might misinform research and result in suboptimal practice.

Why Most Published Research Findings Are False by John P. A. Ioannidis
Continue reading →

Soren Bisgaard 1951-2009

Posted on December 20, 2009  Comments (4)

photo of Soren Bisgaard

Soren Bisgaard died earlier this month of cancer. Soren was a student (Ph.D., statistics) of my father’s who shared the commitment to using applied statistics to improve people’s lives. I know this seem odd to many (I tried to describe this idea previously and read his acceptance of the 2002 William G. Hunter award).

Most recently Soren Bisgaard, Ph.D. was Professor of technology management at Eugene M. Isenberg School of Management at the University of Massachusetts – Amherst. He was an ASQ Fellow; recipient of Shewart Medal, Hunter Award, George Box Medal, among many others awards. Soren also served as the director of the Center for Quality and Productivity Improvement at the University of Wisconsin-Madison (founded by William Hunter and George Box) for several years.

I will remember the passion he brought to his work. He reminded me of my father in his desire to improve how things are done and provide people the opportunity to lead better lives. Those that bring passion to their work in management improvement are unsung heroes. It seems odd, to many, to see that you can bring improvement to people’s lives through work. But we spend huge amounts of our time at work. And by improving the systems we work in we can improve people’s lives. Soren will be missed, by those who knew him and those who didn’t (even if they never realize it).

The Future of Quality Technology: From a Manufacturing to a Knowledge Economy and From Defects to Innovations (pdf) by Soren Bisgaard. Read more articles by Søren Bisgaard.

Related: The Work of Peter Scholtes – Mistakes in Experimental Design and Interpretation – The Scientific Context of Quality Improvement by George Box and Soren Bisgaard, 1987 – William G. Hunter Award 2008: Ronald Does

Obituary Søren Bisgaard at ENBIS:
Continue reading →

Statistics Insights for Scientists and Engineers

Posted on December 5, 2009  Comments (2)

My father was a engineer and statistician. Along with George Box and Stu Hunter (no relation) they wrote Statistics for Experimenters (one of the potential titles had been Statistics for Engineers). They had an interest in bringing applied statistics to the work of scientists and engineers and I have that interest also. To me the key trait for applied statistics is to help experimenters learn quickly: it is an aid in the discovery process. It should not be a passive tool for analysis (which is how people often think of statistics).

José Ramírez studied applied and industrial statistics at the University of Wisconsin – Madison with my father and George Box. And now has a book and blog on taking statistics to engineers and scientists

The book is primarily written for engineers and scientists who need to use statistics and JMP to make sense of data and make sound decisions based on their analyses. This includes, for example, people working in semiconductor, automotive, chemical and aerospace industries. Other professionals in these industries who will find it valuable include quality engineers, reliability engineers, Six Sigma Black Belts and statisticians.
…
For those who want a reference for how to solve common problems using statistics and JMP, we walk through different case studies using a seven-step problem-solving framework, with heavy emphasis on the problem setup, interpretation, and translation of the results in the context of the problem.

For those who want to learn more about the statistical techniques and concepts, we provide a practical overview of the underpinnings and provide appropriate references. Finally, for those who want to learn how to benefit from the power of JMP, we have loaded the book with many step-by-step instructions and tips and tricks.

Related: Highlights from George Box Speech at JMP conference Nov 2009 – Controlled Experiments for Software Solutions – Mistakes in Experimental Design and Interpretation – Florence Nightingale: The passionate statistician

Stat Insights is a blog by José and Brenda Ramírez.

Analyzing and Interpreting Continuous Data Using JMP by José and Brenda Ramírez. view chapter 1 online.

[We] have focused on making statistics both accessible and effective in helping to solve common problems found in an industrial setting. Statistical techniques are introduced not as a collection of formulas to be followed, but as a catalyst to enhance and speed up the engineering and scientific problem-solving process. Each chapter uses a 7-step problem-solving framework to make sure that the right problem is being solved with an appropriate selection of tools.

Florence Nightingale: The passionate statistician

Posted on November 21, 2009  Comments (0)

Florence Nightingale: The passionate statistician

She brought about fundamental change in the British military medical system, preventing any such future calamities. To do it, she pioneered a brand-new method for bringing about social change: applied statistics.
…
he statistics changed Nightingale’s understanding of the problems in Turkey. Lack of sanitation, she realized, had been the principal reason for most of the deaths, not inadequate food and supplies as she had previously thought.
…
As impressive as her statistics were, Nightingale worried that Queen Victoria’s eyes would glaze over as she scanned the tables. So Nightingale devised clever ways of presenting the information in charts. Statistics had been presented using graphics only a few times previously, and perhaps never to persuade people of the need for social change.

Applied statistics is a tool available to all to achieve great improvement. Unfortunately it is still very underused. As George Box says: applied statistics is not about proving a theorem, it’s about being curious about things. The goal of design of experiments is to learn and refine your experiment based on the knowledge you gain and experiment again. It is a process of discovery.

Related: articles on applied statistics – The Value of Displaying Data Well – Statistics for Experimenters – Playing Dice and Children’s Numeracy – Quality, SPC and Your Career – Great Charts

Learning Design of Experiments with Paper Helicopters

Posted on October 11, 2009  Comments (3)

Paper helicopter stairwell dropPhoto showing the helicopter test track by Brad

Dr. George E.P. Box wrote a great paper on Teaching Engineers Experimental Design With a Paper Helicopter that can be used to learn principles of experimental design, including – conditions for validity of experimentation, randomization, blocking, the use of factorial and fractional factorial designs and the management of experimentation.

I ran across an interesting blog post on a class learning these principles today – Brad’s Hella-Copter:

For our statistics class, we have been working hard on a Design of Experiments project that optimizes a paper helicopter with respect to hang time an accuracy of a decent down a stairwell.
…
We were to design a helicopter that would drop 3 stories down within the 2ft gap between flights of stairs.
…
[design of experiments is] very powerful when you have lots of variables (ie. paper type, helicopter blade length, blade width, body height, body width, paperclip weights, etc) and not a lot of time to vary each one individually. If we were to individually change each variable one at a time, we would have made over 256 different helicopters. Instead we built 16, tested them, and got a feel for which variables were most important. We then focused on these important variables for design improvement through further testing and optimization.

Related: 101 Ways to Design an Experiment, or Some Ideas About Teaching Design of Experiments by William G. Hunter (my father) – posts on design of experiments – George Box on quality improvement – Designed Experiments – Autonomous Helicopters Teach Themselves to Fly – Statistics for Experimenters

William G. Hunter Award 2008: Ronald Does

Posted on October 22, 2008  Comments (1)

The recipient of the 2008 William G. Hunter Award is Ronald Does. The Statistics Division of the American Society for Quality (ASQ) uses the attributes that characterize Bill Hunter’s (my father – John Hunter) career – consultant, educator for practitioners, communicator, and integrator of statistical thinking into other disciplines to decide the recipient. In his acceptance speech Ronald Does said:

The first advice I received from my new colleagues was to read the book by Box, Hunter and Hunter. The reason was clear. Because I was not familiar with industrial statistics I had to learn this from the authors who were really practicing statisticians. It took them years to write this landmark book.
…
For the past 15 years I have been the managing director of the Institute for Business and Industrial Statistics. This is a consultancy firm owned by the University of Amsterdam. The interaction between scientific research and the application of quality technology via our consultancy work is the core operating principle of the institute. This is reflected in the type of people that work for the institute, all of whom are young professionals having strong ambitions in both the academic world and in business and industry.
…
The kickoff conference attracted approximately 80 statisticians and statistical practitioners from all over Europe. ENBIS was officially founded in June 2001 as “an autonomous Society having as its objective the development and improvement of statistical methods, and their application, throughout Europe, all this in the widest sense of the words” Since the first meeting membership has grown to about 1300 from nearly all European countries.

Related: 2007 William G. Hunter Award – The Importance of Management Improvement – Designed Experiments – Playing Dice and Children’s Numeracy

Bigger Impact: 15 to 18 mpg or 50 to 100 mpg?

Posted on December 23, 2007  Comments (9)

This is a pretty counter-intuitive statement, I believe:

You save more fuel switching from a 15 to 18 mpg car than switching from a 50 to 100 mpg car.

But some simple math shows it is true. If you drive 10,000 miles you would use: 667 gallons, 556 gallons, 200 gallons and 100 gallons. Amazing. I must admit, when I first read the quote I thought that it must be an wrong. But there is the math. You save 111 gallons improving from 15 mpg to 18 mpg and just 100 improving from 50 to 100 mpg. Other than those of you who automatically guess that whatever seems wrong must be the answer when you see a title like this I can’t believe anyone thinks 15 to 18 mpg is the change that has the bigger impact. It is great how a little understanding of math can help you see the errors in your initial beliefs. Via: 18 Is Enough.

It also illustrates that the way the data is presented makes a difference. You can also view 100 mpg as 1/100 gallon per mile, 2/100 gallons per mile, 5.6/100 gpm and 6.7 gpm. That way most everyone sees that the 6.7 to 5.6 gpm saves more fuel than 2 to 1 gpm does. Mathematics and scientific thinking are great – if you are willing to think you can learn to better understand the world we live in every day.

Related: Statistics Don’t Lie, But People Can be Fooled – Understanding Data – Seeing Patterns Where None Exists – Optical Illusions and Other Illusions – 1=2: A Proof

Playing Dice and Children’s Numeracy

Posted on December 5, 2007  Comments (10)

My father, Willaim Hunter, a professor of statistics and of Chemical Engineering at the University of Wisconsin, was a guest speaker for my second grade class (I think it was 2nd) to teach us about numbers – using dice. He gave every kid a die. I remember he asked all the kids what number do you think will show up when you roll the die. 6 was the answer from about 80% of them (which I knew was wrong – so I was feeling very smart).

Then he had the kids roll the die and he stood up at the front to create a frequency distribution of what was actually rolled. He was all ready for them to see how wrong they were and learn it was just as likely for any of the numbers on the die to be rolled. But as he asked each kid about what they rolled something like 5 out of the first 6 said they rolled a 6. He then modified the exercise a bit and had the kid come up to the front and roll the die on the teachers desk. Then my Dad read the number off the die and wrote on the chart 🙂

This nice blog post, reminded me of that story: Kids’ misconceptions about numbers — and how they fix them

in the real study, conducted by John Opfer and Rober Siegler, the kids used lines with just 0 and 1000 labeled. They were then given numbers within that range and asked to draw a vertical line through the number line where each number fell (they used a new, blank number line each time). The figure above represents (in red) the average results for a few of the numbers used in the study. As you can see, the second graders are way off, especially for lower numbers. They typically placed the number 150 almost halfway across the number line! Fourth graders perform nearly as well as adults on the task, putting all the numbers in just about the right spot.

But there’s a pattern to the second-graders’ responses. Nearly all the kids (93 were tested) understood that 750 was a larger number than 366; they just squeezed too many large numbers on the far-right side of the number line. In fact, their results show more of a logarithmic pattern than the proper linear pattern.

Designed Experiments

Posted on November 28, 2006  Comments (4)

One-Factor-at-a-Time Versus Designed Experiments by Veronica Czitrom:

The advantages of designed experiments over [One Factor at a Time] OFAT experiments are illustrated using three real engineering OFAT experiments, and showing how in each case a designed experiment would have been better. This topic is important because many scientists and engineers continue to perform OFAT experiments.

I still remember, as a child, asking what my father was going to be teaching the company he was going to consult with for a few days. He said he was going to teach them about using designed factorial experiments. I said, but you explained that to me and I am just a kid? How can you be teaching adults that? Didn’t they learn it in school? The paper provides some examples showing why OFAT experimentation is not as effective as designed multi-factor experiments.

Related: Design of Experiments articles – Statistics for Experimenters (2nd Edition) – Design of Experiments blog posts