The National Assessment of Educational Progress from the United States Department of Education is the definitive report on k-12 science education based on testing 4th, 8th and 12th grade students. The report provides a huge amount of data on testing results. At first look, it seems basically things stayed the same over the last 5 years.

Various differences are shown (for example: “Most states showed no improvement at grades 4 and 8. Five of the 37 participating states, however, did improve between 2000 and 2005 - and did so at both grades.”). However, I remain a bit skeptical of reading much into such claims. Even if you changed nothing (just retest the students the next month say) and then look for differences between the two sets of data it is possible to find seemingly interesting differences. It is very easy to be fooled when you have a large pool of data and search for any differences that seem interesting.

We commented on one example of why it is important to be careful in making conclusions based on data recently (in our management improvement blog). Most often people look for the differences to highlight the differences. That creates a bias to find such differences, which leads me to be a bit skeptical of such claims without an explanation of why the data is convincing that such a difference is significant and not just variation in the data.

The data from the test does provide a resource for those interested in exploring these matters, which is good.

The Department of Education provides sample questions online. Try them yourself: they are interesting. Unfortunately, for some questions requiring written responses, they don’t actually provide what the answer should be.

Science scores up in grade four, stalled in grades 8 and 12

News stories:

• Test Shows Drop in Science Achievement for 12th Graders by Sam Dillon
• Top of the class: Virginia a model for science education
  Forty percent of fourth-grade students and 35 percent of eighth-graders in Virginia’s public schools have a solid grasp of physical and life science, the NAEP reported.

  Nationally, the proficiency percentage for fourth-grade students is 29 percent, and 30 percent for those in eighth grade.

• State pupils improving in science tests - but 4th- and 8th-graders still not doing as well as their peers across the nation
  In fourth-grade testing, only Mississippi scored below California, while California’s eighth-grade scores ranked 42nd out of 44 states. Of California’s fourth-graders, 17 percent were proficient or better in science, and half scored below the basic level. Among California eighth-graders, 18 percent were proficient or better, while 56 percent were below basic.

  Wide achievement gaps persist for California’s economically disadvantaged students, with 73 percent scoring below the basic level, and among its ethnic minorities, with 74 percent of black eighth-graders and 73 percent of Hispanic eighth-graders scoring below basic.