Posts about experiment

I Just Finished Statistics for Experimenters and I Cannot Praise it Enough

Guest post by Michael Betancourt.

I just finished Box, Hunter, and Hunter (Statistics for Experimenters) and I cannot praise it enough. There were multiple passages where I literally giggled. In fact I may have been a bit too enthusiastic about tagging quotes beyond “all models are wrong but some are useful” that I can’t share them all.

photo of Statistics for Experimenters with many blue bookmarks shown

I wish someone had shared this with me when I was first learning statistics instead of the usual statistics textbooks that treat model development as an irrelevant detail. So many of the elements that make this book are extremely relevant to statistics today. Some examples:

  • The perspective of learning from data only through the lens of the statistical model. The emphasis on sequential modeling, using previous fits to direct better models, and sequential experiments, using past fits to direct better targeted experiments.
  • The fixation on checking model assumptions, especially with interpretable visual diagnostics that capture not only residuals but also meaningful scales of deviation. Proto visual predictive checks as I use them today.
  • The distinction between empirical models and mechanistic models, and the treatment of empirical linear models as Taylor expansions of mechanistic models with covariates as _deviations_ around some nominal value. Those who have taken my course know how important I think this is.
  • The emphasis that every model, even mechanistic models, are approximations and should be treated as such.
  • The reframing of frequentist statistical tests as measures of signal to noise ratios.
  • The importance of process drift and autocorrelation in data when experimental configurations are not or cannot be arbitrarily randomized.
  • The diversity of examples and exercises using real data from real applications with detailed contexts, including units everywhere.

Really the only reason why I wouldn’t recommend this as an absolute must read is that the focus on linear models and use of frequentist methods does limit the relevance of the text to contemporary Bayesian applications a bit.

Texts like these make me even more frustrated by the desire to frame movements like data science as revolutions that give people the justification to ignore the accumulated knowledge of applied statisticians.

Academic statistics has no doubt largely withdrawn into theory with increasingly smaller overlap with applications, but there is so much relevant wisdom in older applied statistics texts like these that doesn’t need to be rediscovered just reframed in a contemporary context.

Oh, I forgot perhaps the best part! BHH continuously emphasizes the importance of working with domain experts in the design and through the entire analysis with lots of anecdotal examples demonstrating how powerful that collaboration can be.

I felt so much less alone every time they talked about experimental designs not being implemented properly andthe subtle effects that can have in the data, and serious effects in the resulting inferences, if not taken into account.

Michael Betancourt, PhD, Applied Statistician – long story short, I am a once and future physicist currently masquerading as a statistician in order to expose the secrets of inference that statisticians have long kept from scientists. More seriously, my research focuses on the development of robust statistical workflows, computational tools, and pedagogical resources that bridge statistical theory and practice and enable scientists to make the most out of their data.
Twitter: @betanalpha
Website: betanalpha
Patreon: Michael Betancourt

Related: Statistics for Experimenters, Second EditionStatistics for Experimenters in SpanishStatistics for Experimenters ReviewCorrelation is Not Causation

Dogs and Wolves Share a Sense of Fair Pay

Dogs and wolves share sense of fair play

The scientists tested similarly raised dogs and wolves that lived in packs. Two animals of each species were placed in adjacent cages, equipped with a buzzer apparatus. When the dog or wolf pressed it with their paw, both animals got a reward on some occasions. Other times, the dog or wolf doing the task got nothing while the partner did.

The key finding was that when the partner got a high value treat, the animal doing the task refused to continue with it.

photo of a Gray Wold looking at the camera

Gray Wolf by Gary Kramer (USFWS), public domain

This is a similar result as that found with Capuchin monkeys that don’t like being paid less than others.

The question of social status or hierarchy also played an important role in the experiments with dogs and wolves of higher rank taking umbrage more quickly.

The human impact on dogs isn’t entirely absent though. Pet dogs are less sensitive to being treated unfairly – probably because of their experience with us!

It is fun to see these results mirror aspects of our psychology. It is fun to see how these experiments test out animal’s responses.

Related: Goats Excel at Learning and Remembering a Complex TasksRats Show Empathy-driven BehaviorInsightful Problem Solving in an Asian ElephantsHow Wolves Changed the Yellowstone Ecosystem

Elephants Learn to Cooperate to Reach Their Objective

This clip shows elephants learning to work together to achieve what they can’t achieve alone (from BBC’s Super Smart Animals). It is interesting to see what animals are capable of. See the related post links for more amazing animal behavior.

Related: Insightful Problem Solving in an Asian Elephant (2007)Crows can Perform as Well as 7 to 10-year-olds on cause-and-effect Water Displacement TasksBeehive Fence Protects Farms from ElephantsCapuchin Monkeys Don’t Like Being Paid Less Than Their PeersFriday Fun: Bird Using Bait to Fish

Using Diatom Algae to Deliver Chemotherapy Drugs Directly to Cancer Cells

I am thankful for scientists doing the time consuming and important research to find new ways to fight disease. Here is an interesting webcast discussing how chemotherapy is used to fight cancer and how scientists are looking to algae to deliver the chemotherapy drugs to better target cancer cells (while not savaging our health cells).

I am also thankful to the funding sources that pay for this research (and for cool explanations of science, like SciShow).

Read more about the genetically engineered algae kills 90% of cancer cells without harming healthy ones. The algae are a diatom and many diatoms look very cool.

Sadly the actual research paper (by government funded university professors) is published by a closed science publisher (when are we finally going to stop this practice that was outdated over a decade ago?). Thankfully those responsible for SciShow are much more interested in promoting science than maintaining outdated business models (in direct contrast to so many science journal publishers).

Related post on cool delivery methods for life saving drugs: Using Bacteria to Carry Nanoparticles Into CellsSelf-Assembling Cubes Could Deliver Medicine (2006)Nanoparticles With Scorpion Venom Slow Cancer SpreadNASA Biocapsules Deliver Medical Interventions Based Upon What They Detect in the Body

Scientific Inquiry Leads to Using Fluoride for Healthy Teeth

This webcast, from the wonderful SciShow, explores how we discovered fluoride helps prevent tooth decay and how we then used that knowledge and finally discovered why it worked.

I love stories of how we learn for observing what is happening. We don’t always need to innovate by thinking up creative new ideas. If we are observant we can pick up anomalies and then examine the situation to find possible explanations and then experiment to see if those explanations prove true.

When working this way we often are seeing correlation and then trying to figure out which part of the correlation is an actual cause. So in this dental example, a dentist noticed his patients had bad brown stains on their teeth than others populations did.

After investigation the natural fluoridation of the water in Colorado Springs, Colorado, USA seemed like it might be an explanation (though they didn’t understand the chemistry that would cause that result). They also explored the sense that the discolored teeth were resistant to decay.

Even without knowing why it is possible to test if the conditions are the cause. Scientists discovered by reducing the level of fluoridation in the water the ugly brown stains could be eliminated (these stains took a long time to develop and didn’t develop in adults). Eventually scientists ran an experiment in Grand Rapids, Michigan and found fluoridation of the water achieved amazing results for dental health. The practice of fluoridation was then adopted widely and resulted in greatly improved dental health.

In 1901, Frederick McKay, a recent dental school graduate, opened a dental practice in Colorado Springs, Colorado. He was interested in what he saw and sought out other dentists to explore the situation with him but had little success. In 1909, he found some success when renowned dental researcher Dr. G.V. Black collaborate with him.
Dr. H. Trendley Dean, head of the Dental Hygiene Unit at the National Institute of Health built on their work when he began investigating the epidemiology of fluorosis in 1931. It wasn’t until 1945 that the Grand Rapids test started. Science can take a long time to move forward.

Only later did scientists unravel why this worked. The fluoride reacts to create a stronger enamel than if the fluoride is not present. Which results in the enamal being less easily dissolved by bacteria.
Health tip: use a dental stimudent (dental picks) or floss your teeth to maintain healthy gums and prevent tooth decay. It makes a big difference.

Related: Why does orange juice taste so bad after brushing your teeth?Microbiologist Develops Mouthwash That Targets Only Harmful Cavity Causing BacteriaUsing Nanocomposites to Improve Dental Filling PerformanceFinding a Dentist in Chiang Mai, ThailandFalse Teeth For CatsWhy Does Hair Turn Grey as We Age?

MudWatt: Make Power From Mud!

Keegan Cooke and Kevin Rand created MudWatt kits as a way to engage kids/students with science. From the website:

We want to show kids this brighter side of STEM, to empower them to become the great problem solvers of tomorrow. Because let’s face it, there are plenty of problems in the world that need solving.

Unfortunately, our experience in school wasn’t unique. In 2011, less than one-third of 8th graders in the U.S. were deemed proficient in science. Today, 70% of the fastest growing careers are in STEM fields. The supply of STEM education is not meeting the demand.

Most of the world’s mud contain microbes that produce electricity when they eat. That is the engine driving the MudWatt. Colonies of special bacteria (called shewanella and geobacter) generate the electricity in a MudWatt.

The electricity output is proportional to the health and activity of that bacterial colony. By maintaining these colonies in different ways, you can use MudWatt to run all kinds of great experiments. Thus the MudWatt allows kids to engage with science, using their natural curiosity to experiment and learn. Engaging this too-often-neglected human potential will bring joy to those kids (as kids and as grown-ups) and benefit our society.

With standard topsoils, typical power levels are around 100 microWatts, which is enough to power the LED, buzzer, clock, etc..

Related: Arduino, open source hardware (Introduction Video Tutorial)Teaching Through TinkeringAwesome Gifts for the Maker in Your LifeQubits Construction Toy

Yacouba Sawadogo – The Man Who Stopped the Desert

Quote from the video

Yacouba single-handedly had more impact on the soil conservation in the Sahel than than all the national and international researchers combined.

Dr. Chris Reij, Vrije University of Amsterdam.

As is normally the case making improvements in the real world is challenging and visionaries often face setbacks. Even when they have success that success is threatened by those that want to take the rewards but ignore the lessons. The clip above is a excerpt from the documentary film on his efforts.

Meet Yacouba Sawadogo – The Man Who Stopped the Desert

The simple old farmer’s re-forestation and soil conservation techniques are so effective they’ve helped turn the tide in the fight against the desertification of the harsh lands in northern Burkina Faso.

Over-farming, over-grazing and over population have, over the years, resulted in heavy soil erosion and drying in this landlocked West African nation.

Zai is a very simple and low-cost farming technique. Using a shovel or an axe, small holes are dug into the hard ground and filled with compost. Seeds of trees, millet or sorghum are planted in the compost. The holes catch water during the rainy season, so they are able to retain moisture and nutrients during the dry season.

According to the rules of Zai, Yacouba would prepare the lands in the dry season – exactly the opposite of the local practice. Other farmers and land chiefs laughed at him, but soon realized that he is a genius. In just 20 years, he converted a completely barren area into a thriving 30-acre forest with over 60 species of trees.

Yacouba has chosen not to keep his secrets to himself. Instead, he hosts a workshop at his farm, teaching visitors and bringing people together in a spirit of friendship. “I want the training program to be the starting point for many fruitful exchanges across the region

Continue reading

Crows can Perform as Well as 7 to 10-year-olds on cause-and-effect Water Displacement Tasks

In Aesop’s fable about the crow and the pitcher, a thirsty bird happens upon a vessel of water, but when he tries to drink from it, he finds the water level out of his reach. Not strong enough to knock over the pitcher, the bird drops pebbles into it — one at a time — until the water level rises enough for him to drink his fill.

Highlighting the value of ingenuity, the fable demonstrates that cognitive ability can often be more effective than brute force. It also characterizes crows as pretty resourceful problem solvers. New research conducted by UC Santa Barbara’s Corina Logan, with her collaborators at the University of Auckland in New Zealand, proves the birds’ intellectual prowess may be more fact than fiction. Her findings, supported by the National Geographic Society/Waitt Grants Program, appear today in the scientific journal PLOS ONE: Modifications to the Aesop’s Fable Paradigm Change New Caledonian Crow Performances.

photo of Corina Logan

Researcher Corina Logan with a great-tailed grackle and a night heron at the Santa Barbara Zoo. The zoo is one of the sites where Logan is gathering data to compare and contrast the cognitive abilities of grackles and New Caledonian crows.
Photo Credit: Sonia Fernandez

Logan is lead author of the paper, which examines causal cognition using a water displacement paradigm. “We showed that crows can discriminate between different volumes of water and that they can pass a modified test that so far only 7- to 10-year-old children have been able to complete successfully. We provide the strongest evidence so far that the birds attend to cause-and-effect relationships by choosing options that displace more water.”

Logan, a junior research fellow at UCSB’s SAGE Center for the Study of the Mind, worked with New Caledonian crows in a set of small aviaries in New Caledonia run by the University of Auckland. “We caught the crows in the wild and brought them into the aviaries, where they habituated in about five days,” she said. Keeping families together, they housed the birds in separate areas of the aviaries for three to five months before releasing them back to the wild.

Continue reading

Go Slow with Genetically Modified Food

My thoughts on Genetically Modified Organisms (GMO), specifically GM foods, basically boil down to:

  • messing with genes could create problems
  • we tend to (and especially those seeking to gain an advantage tend to – even if “we” overall wouldn’t the people in the position to take aggressive measures do) ignore risks until the problems are created (often huge costs at that point)
  • I think we should reduce risk and therefore make it hard to justify using GMO techniques
  • I agree occasionally we should do so, like it seems with oranges and bananas.
  • I agree the practice can be explained in a way that makes it seem like there is no (or nearly no) risk, I don’t trust we will always refrain from stepping into an area where there is a very bad result

Basically I would suggest being very cautious with GMO. I like science and technology but I think we often implement things poorly. I think we are not being cautious enough now, and should reduce the use of GMO to critical needs to society (patents on the practices need to be carefully studied and perhaps not permitted – the whole patent system is so broken now that it should be questioned at every turn).

Antibiotic misuse and massive overuse is an obvious example. We have doctors practicing completely unjustified misuse of antibiotics and harming society and we have factory farms massively overusing antibiotics causing society harm.

The way we casually use drugs is another example of our failure to sensibly manage risks, in my opinion. This of course is greatly pushed by those making money on getting us to use more drugs – drug companies and doctors paid by those companies. The right drugs are wonderful. But powerful drugs almost always have powerful side effects (at least in a significant number of people) and those risks are multiplied the more we take (due to interactions, weakness created by one being overwhelmed by the next etc.). We should be much more cautious but again we show evidence of failing to act cautiously which adds to my concern for using GMO.

I love antibiotics, but the way we are using them is endangering millions of lives (that is a bad thing). I don’t trust us to use science wisely and safely. We need to more consciously put barriers in place to prevent us creating massively problems.

Related: Research on Wheat RustThe AvocadoOverfishing, another example of us failing to effectively cope with systemic consequences

Medical Study Findings too Often Fail to Provide Us Useful Knowledge

There are big problems with medical research, as we have posted about many times in the past. A very significant part of the problem is health care research is very hard. There are all sorts of interactions that make conclusive results much more difficult than other areas.

But failures in our practices also play a big role. Just poor statistical literacy is part of the problem (especially related to things like interactions, variability, correlation that isn’t evidence of causation…). Large incentives that encourage biased research results are a huge problem.

Lies, Damned Lies, and Medical Science

He discovered that the range of errors being committed was astonishing: from what questions researchers posed, to how they set up the studies, to which patients they recruited for the studies, to which measurements they took, to how they analyzed the data, to how they presented their results, to how particular studies came to be published in medical journals. The systemic failure to do adequate long term studies once we approve drugs, practices and devices are also a big problem.

This array suggested a bigger, underlying dysfunction, and Ioannidis thought he knew what it was. “The studies were biased,” he says. “Sometimes they were overtly biased. Sometimes it was difficult to see the bias, but it was there.” Researchers headed into their studies wanting certain results—and, lo and behold, they were getting them. We think of the scientific process as being objective, rigorous, and even ruthless in separating out what is true from what we merely wish to be true, but in fact it’s easy to manipulate results, even unintentionally or unconsciously. “At every step in the process, there is room to distort results, a way to make a stronger claim or to select what is going to be concluded,” says Ioannidis. “There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded.”

Another problem is that medical research often doesn’t get the normal scientific inquiry check of confirmation research by other scientists.

Most journal editors don’t even claim to protect against the problems that plague these studies. University and government research overseers rarely step in to directly enforce research quality, and when they do, the science community goes ballistic over the outside interference. The ultimate protection against research error and bias is supposed to come from the way scientists constantly retest each other’s results—except they don’t. Only the most prominent findings are likely to be put to the test, because there’s likely to be publication payoff in firming up the proof, or contradicting it.

Related: Statistical Errors in Medical StudiesMedical Study Integrity (or Lack Thereof)Contradictory Medical Studies (2007)Does Diet Soda Result in Weight Gain?

Tropical Lizards Can Solve Novel Problems and Remember the Solutions

Brainy Lizards Pass Tests for Birds

[Duke biologist Manuel Leal] tested the lizards using a wooden block with two wells, one that was empty and one that held a worm but was covered by a cap. Four lizards, two male and two female, passed the test by either biting the cap or bumping it out of the way.

The lizards solved the problem in three fewer attempts than birds need to flip the correct cap and pass the test, Leal said. Birds usually get up to six chances a day, but lizards only get one chance per day because they eat less. In other words, if a lizard makes a mistake, it has to remember how to correct it until the next day

Leal’s experiment “clearly demonstrates” that when faced with a situation the lizards had never experienced, most of them were able to devise a way to solve the problem. Their ability to “unlearn” a behavior, a skill that some mammalian species have difficulty in, is the mark of a cognitively advanced animal, said Jonathan Losos, a biologist at Harvard who was not involved in the study.

To see if the lizards could reverse this association, Leal next placed the worm under the other cap. At first, all the lizards bumped or bit the formerly lucrative blue cap. But after a few mistakes, two of the lizards figured out the trick. “We named these two Plato and Socrates,” Leal said.

It is very cool to see what scientists keep learning about animals.

Related: Insightful Problem Solving in an Asian ElephantBird Using Bread as Bait to Catch FishCrows Transferring Their Understanding to Novel ProblemDolphins Using Tools to Hunt