Superluminal neutrinos - a case study in how good science is done

As many people have now heard, the OPERA collaboration is reporting a very surprising observation.  The OPERA experiment is part of CERN, and is an experiment meant to study neutrino flavor oscillations.  The idea is, the proton beam at CERN creates a beam of neutrinos.  Since neutrinos hardly interact with normal matter, they move in a straight line right through the earth, and pass through the experimental station in Gran Sasso, Italy, where some small fraction of them are then detected.  There are (according to the Standard Model) three flavors of neutrinos, the electron neutrino, muon neutrino, and tau neutrino.  It has been determined experimentally that those flavors are not exact "mass eigenstates".  That means that if you start off with a tau neutrino of particular energy, for example, and let it propagate for a while, it will change into a muon neutrino with some probability that oscillates in time.  Anyway, OPERA wanted to study this phenomenon, and in doing so, they measured the time it takes neutrinos to go from their production point at CERN to the detector in Gran Sasso, using precisely synchronized special clocks.  They also used differential GPS to measure the distance between the production point and the detector to within 20 cm.  Dividing the distance by the time, they found much to their surprise that the neutrinos appear to traverse the distance about 60 ns faster than would be expected if they traveled at the speed of light in vacuum.

So, what could be going on here?  There are a few possibilities.  First, they could have the distance measurement wrong.  This seems unlikely, given the use of differential GPS and the sensitivity (they could clearly see the change in the distance due to a 2009 earthquake, as shown in Fig. 7 of the paper).  Second, they could have a problem in their synchronization of the clocks.  That seems more likely to me, given that the procedure is comparatively complicated.  Third, there is some other weird systematic at work that they haven't found.  Fourth, neutrinos are actually tachyons.  That would be all kinds of awesome, but given how challenging it would be to reconcile that with special relativity and causality, I'm not holding my breath.

Why is this an example of good science?  The collaboration spent three years looking hard at their data, analyzing it many different ways, checking and cross-checking.  They are keenly aware that a claim of FTL neutrinos would be the very definition of "extraordinary" in the scientific sense, and would therefore require extraordinary evidence.  Unable to find the (highly likely) flaw in their analysis and data, they are showing everything publicly, and asking for more investigation.  I want to point out, this is the diametric opposite of what happens in what I will term bad science (ahem.  Italian ecat guys, I'm looking at you.).   This is how real experimental science works - they're asking for independent reproduction or complementary investigation.  I hope science journalists emphasize this aspect of the story, rather than massively sensationalizing it or portraying the scientists as fools if and when a flaw is found.