Certain Uncertainties, Part I

When reporting scientific discoveries, the popular press hardly ever conveys the inherent uncertainties in the data or the interpretation. This seemingly innocent omission carries a subtle, misguided message: if it’s a scientific study, the results are exact and correct. These same news reports often declare that scientists, having previously thought one thing, are now forced to think something else; or are forced to return to the mythic “drawing board” in a stupor. The consequence? If you get all your science from press accounts then you might be led to believe that scientists arrogantly, yet aimlessly, bounce back and forth between one perceived truth and another without ever contributing to a base of objective knowledge.

But let’s take a closer look.

New ideas put forth by well-trained research scientists will be wrong most of the time because the frontier of discovery can be a messy place. But scientists know this and are further trained to quantify their level of ignorance with an estimate of the claim’s uncertainty. (The famous “plus-or-minus” sign in reported polling results is, perhaps, the most widely recognized example.) A scientist typically presents a tentative result based on a shaky interpretation of poor data. Six months later, different, yet equally bad data become available from somebody else’s experiment and a different interpretation emerges. During this phase, which may drag on for years or even decades, news stories implying unassailable fact are written anyway.

Eventually, excellent data become available and a consensus emerges—a long term process that does not lend itself to late-breaking news reports. Studies on environmental health risks or the effects of food consumption on diseases and longevity, are especially susceptible to being over-interpreted. The financial consequences of premature news stories, and the attendant reactions on Wall Street, can be staggering. In 1992, a law suit was brought against two cellular phone companies by a Florida man claiming that his wife’s death from brain cancer was caused by her heavy use of cellular phones. When this and several similar claims hit the news in late January 1993, the market capitalization of publicly traded cellular phone companies fell by billions of dollars in less than a week. Since you can get brain cancer without ever having used a cellular phone. And since the popularity of cellular phones was on the rise, you might expect some users to die from brain cancer just as some users might die from heart disease, or from old age. In this case, there was no definitive study to establish a cause and effect between cellular phone use and brain cancer, yet people reacted anyway.

Fortunately, most of the comings and goings of astrophysics have so little impact on how people conduct their daily lives that I can spend more time joking about the problem than crying about it.

Initial uncertainty is a natural element of the scientific method, yet the scientific method is, without question, the most powerful and successful path ever devised to understand the physical world. When a published scientific finding is confirmed and reconfirmed and re-reconfirmed and re-re-reconfirmed, further confirmation becomes less interesting than working on another problem and new nuggets of knowledge are justifiably presented with little or no uncertainty in the basic textbooks of the day. Consistency and repeatability are the hallmarks of a genuine scientific finding, for if the laws of physics and chemistry were unpredictably different from lab to lab then scientists would all just pack up and go home.

A modern astronomy textbook will discuss the well-known fact that the Sun occupies the center region of our Solar System. You can bet that five hundred years from now, we will still be saying that the Sun occupies the center region of the Solar System. Yet five hundred years ago this notion was a major source of debate—Earth was the object presumed to occupy center stage.

That same modern textbook on astronomy will, however, speak only tentatively about the formation of galaxies in the early universe, or the nature of the ubiquitous dark matter, because major uncertainties remain in these areas. Suppose one day someone discovers that dark matter in the universe is actually made of chocolate pudding. Despite all of the problems this would create in theoretical astrophysics, no previously held consensus would be overthrown because no consensus exists.

In person, scientists have been known to completely ignore their uncertainties because, for the most part, scientists are people too. There are arrogant ones, lovable ones, loud ones, soft-spoken ones, and boneheaded ones. In published research papers, however, everyone is timid because of the semi-permanence of the printed word and the overwhelming frequency of wrong ideas. Most results flow from the edge of our understanding and are therefore subject to large uncertainties.

Consider the origin of the Moon’s craters. Any modern book will describe them with certainty as being caused by high-speed collisions with rocks and other debris from interplanetary space. But a century ago, they were described as being volcanic calderas. If the scientific community changed its mind once before then why should you believe us now? Because last century, volcanic calderas were a leading idea but had not achieved consensus and were thus not presented as a certainty in responsible scientific writings. In a chapter of The Heavens Above, a popular handbook of astronomy written in 1882 by two academics (J. A. Gillet, Professor of Physics at the City College of New York, and W. J. Rolfe Headmaster, of Cambridge High School in Massachusetts) they describe the lunar surface:

The smaller saucer-shaped formations on the surface of the moon are called craters. They are of all sizes, from a mile to a hundred and fifty miles in diameter; and they are supposed to be of volcanic origin.

Short of quantifying the uncertainty, which may be inappropriate for a popular book, the words “supposed to be” make an excellent literary substitute. And as late as 1923, Sir Richard Gregory, a Professor of Astronomy, Queens College, London, wrote in the popular book The Vault of Heaven

The origin of the lunar craters is still obscure. Analogy suggests that the forces which cause volcanic eruptions on the earth have been at work on the moon… [although] some astronomers and geologists favour the view that the craters were produced by the bombardment of masses of rock when the moon was in a plastic condition.

More often than not, a scientist’s printed word presents an honest, almost humble uncertainty that goes unnoticed when people reflect on the history of scientific misconceptions. What about that 1996 research paper in the journal Science that claimed to have found life in a Martian meteorite? The nine coauthors wrote, among other things, in their abstract:

The carbonate globules [in the Martian meteorite] are similar in texture and size to some terrestrial bacterially induced carbonate precipitates. Although inorganic formation is possible, formation of the globules by biogenic processes could explain many of the observed features… and could thus be fossil remains of past martian biota.

From the over-sized newspaper headlines that followed, you would never guess that the original research paper contained such unassertive language.

At mid-nineteenth century, there was a famous disagreement between the gentle geologists and the cocky physicists over the age of the Earth and Sun. The geologists needed billions of years to account for Earth’s surface features. The physicists, led by the brilliant William Thomson (who became Lord Kelvin) asserted an age of at least ten but no more than one hundred million years, based on thermodynamic calculations that invoke Earth’s current internal temperature and the rate at which ordinary matter cools. A similar calculation for the Sun yielded similar results.

Lord Kelvin was off by a factor of fifty.

How embarrassing is this? Not bad at all. Kelvin’s actual statements (this one take from a 1871 lecture “On Geological Time”) candidly alert the reader to inherent uncertainties in his calculations:

Now, if the sun is not created a miraculous body, to shine on and give out heat forever, we must suppose it to be a body subject to the laws of matter (I do not say there may not be laws which we have not discovered) but, at all events, not violating any laws we have discovered or believe we have discovered, we should deal with the sun as we should with any large mass of molten iron, silicon or sodium.

With no concept of the heat generated by radioactivity (for the Earth’s interior) or thermonuclear fusion (for the solar interior), Kelvin had no chance of getting the right answer. But he humbly, and correctly, recognized that he might be missing yet-to-be-discovered laws of nature.

Another historical conundrum was whether the well-known force of gravity, discovered by Isaac Newton in the seventeenth century, applied to regions far beyond the solar system. As late as 1893, Charles Young, a Professor of Astronomy at the College of New Jersey (Princeton) wrote in Lessons in Astronomy: It is probable (though not certain) that gravitation operates between the stars, as indicated by the motion of binaries. Once again the scientist conveys what is known, and is candid about uncertainties.

When this humble approach is abandoned, embarrassing claims can result. The nineteenth-century astronomy popularizer Agnes M. Clerke, who wrote brilliantly in her 1890 System of the Stars about all manner of astronomical knowledge, lapsed into hyperbolic denial on the nature of the spiral nebulae in the sky:

The question whether nebulæ are external galaxies hardly any longer needs discussion. It has been answered by the progress of discovery. No competent thinker, with the whole of the available evidence before him, can now, it is safe to say, maintain any single nebula to be a star system of coordinate rank with the Milky Way. A practical certainty has been attained that the entire contents, stellar and nebular, of the sphere belong to one mighty aggregation, and stand in ordered mutual relations within the limits of one all-embracing scheme—all-embracing, that is to say, so far as our capacities of knowledge extend. With the infinite possibilities beyond, science has no concern.

This passage is simply irresponsible. The lady doth protest too much. If the facts of the case were as secure as she implies then she would not have needed 114 words to say so. While not a research scientist, Clerke should have known better. She had enough command of astronomy to write authoritatively on the subject and to become one of the leading popularizers of her day, but she left no room for uncertainty, and bet on the wrong horse.

Forty-three years later, just three years before Edwin Hubble settled the debate on the nature of the nebulae, the Astronomy Professor Sir Richard Gregory candidly confessed in The Vault of Heaven:

It can scarcely be said, however, that anything very definite is known concerning the form of the sidereal universe even at the present time.

This kind of writing does not make headlines, but it is honest and accurate.

A rare, but now-famous case of a misreported uncertainty coupled with an overconfident claim by a scientist took place in early 1998, when the Central Bureau for Astronomical Telegrams (the clearing-house for astronomers of the world who need to disseminate up-to-the-minute sky phenomena among colleagues) announced the discovery of a mile-wide asteroid whose orbit would bring it dangerously close to Earth in the year 2028. (Formerly sent around the world via telegram, these notices are now distributed instantly via e-mail.) The offending asteroid was coded 1997 XF11, which identifies when in the year 1997 the asteroid was discovered. The telegram reported on 11 March 1998:

This object, discovered by J. V. Scotti in the course of the Spacewatch program at the University of Arizona on 1997 Dec. 6… recognized as one of the 108 “potentially hazardous asteroids,” has been under observation through 1998 Mar. 4… An orbit computation from the 88-day arc…indicates that the object will pass only 0.00031 AU from the earth on 2028 Oct. 26.73 UT! Error estimates suggest that passage within 0.002 AU is virtually certain, this figure being decidedly smaller than has been reliably predicted for generally fainter potentially hazardous asteroids in the foreseeable future.

Converting to everyday language, the announcement declared that the asteroid would come within 30,000 miles of Earth (a cosmic hair’s width), but with an uncertainty in the calculation that could place the asteroid anywhere within a 200,000 mile “error-circle” that happens to enclose Earth. When the substance of this telegram was further distributed via press release from the American Astronomical Society, passing along the hair-raising words virtually certain, a media deluge followed that overnight converted your neighborhood astronomers into the most sought-after people in your community.

The telegram went on to give the best available coordinates for the object, obtained from observers who were tracking it, preceded by a scientifically sensible appeal: The following ephemeris is given in the hope that further observations will allow refinement of the 2028 miss distance. The next day, on March 12, 1998, another telegram appeared that announced the existence of what astronomers call a “prediscovery” photograph of the asteroid, obtained from archival survey images taken in 1990. This significantly extended the baseline of observations to well beyond the original eighty-eight days. (Longer baselines always provide more accurate estimates than shorter ones.) Calculations that incorporated the new data narrowed the error-circle to a skinny, ellipse that handily shifted Earth from within the range of collision uncertainty to well outside of it. Five weeks later, a telegram was issued that corrected the alarmist language of the first announcement and admitted that the original telegram’s uncertainties could have been sharpened if a more complex method of calculation been used.

The episode was widely reported as a blunder; but at worst, the original calculation was simply incomplete. At best, it was a valid scientific starting point. True, the survival of the human species was involved, but most importantly, everything worked the way it was supposed to. The early estimate, and the better estimates that followed (within a day!) were a model of how science refines itself as it approaches an objective reality.

And after what had been twenty-four hours of sensationalist journalism across the country, the New York Post, a colorfully written daily newspaper in New York City, ran the inimitable headline: KISS YOUR ASTEROID GOODBYE. And a few days later, an illustration by Jesse Gordon on the Op-Ed page of the New York Times depicted the asteroid changing its collision course over a sequence of panels. We are treated to the top nine reasons why the asteroid has decided to not hit Earth, one of them being: “No desire to spend the rest of its days in the lobby of the Museum of Natural History.“