Posted on June 10, 2013  Comments (6)

Scientific inquiry is aided by sensible application of statistical tools. I grew up around the best minds in applied statistics. My father was an eminent applied statistican, and George Box (the person in the video) was often around our house (or we were at his). Together they wrote Statistics for Experimenters (along with Stu Hunter, not related to me) the bible for design of experiments (George holds up the 1st edition in the video).

The video may be a bit confusing without at least a basic idea of factorial designed experiments. These introductory videos, by Stu Hunter, on Using Design of Experiments to Improve Results may help get you up to speed.

[the video has been removed from the internet]

This video looks at using fractional factorials to reduce the number of experiments needed when doing a multifactor experiment. I grew up understanding that the best way to experiment is by varying multiple factors at the same time. You learn much quicker than One Factor At a Time (OFAT), and you learn about interactions (which are mainly lost in OFAT). I am amazed to still hear scientists and engineers talk about OFAT as a sensible method or even as the required method, but I know many do think that way.

To capture the interactions a full factorial requires an ever larger number of experimental runs to be complete. Assessing 4 factors requires 16 runs, 6 would require 64 and 8 would require 256. This can be expensive and time consuming. Obviously one method is to reduce the number of factors to experiment with. That is done (by having those knowledgable about the process include only those factors worth the effort), but if you still have, for example, 8 very important factors using a fractional factorial design can be very helpful.

And as George Box says “What you will often find is that there will be redundant factors… and don’t forget about those redundant factors. Knowing that something doesn’t matter is almost as important as knowing what does.” If you learn a factor isn’t having an affect you may be able to save money. And you can eliminate varying that factor in future experiments.

You may notice in the example the large impact increased temperature has (and even larger impact based on the interaction with the other factor). A likely next step would be to experiment with higher temperatures using evolutionary operations methods (developed by George Box in the 1950s). Alternatively it might be that more fractional factorial designed experiments are done (likely dropping the un-important factors) and perhaps trying higher temperatures and adjustments in other variables (or even adding a new factor).

Remember that when these experiments are done in a business setting that money plays a large role. It could be the case that increasing a factor proved beneficial to the results but that the benefit was less than the cost of using that factor at that level. So the experiments are not just to find the best result but the best result factoring in costs. And those may include the need to consider that costs may vary so if a certain raw material is below x price they best option are these setting, but when it increases we need to reduce that factor and increase 2 other factors…

Related: The Art of Discovery (earlier segment of the videotape from which the segment in this post is taken) – 11 Year Old Using Design of Experiments – Multivariate experiments are a very powerful management tool to learn and improve performance – Full and Fractional Factorial Test Design – The Role of Statistics in Quality and Productivity Improvement

By the way I am pretty sure the videos were taken at George’s house at the time. This was different house than the one I visited as a child. It was quite a nice place set in the countryside outside Madison, Wisconsin.