When I was a graduate student, I was asked to review a paper submitted to a technical journal. The authors proposed a theory I didn’t agree with, and so I read the paper with anything but an open mind, selectively ignoring the good data they presented while seeking other data to support my way of thinking. In my review, I stated that I didn’t agree with the theory and presented what I thought were valid arguments as to why it was wrong. In their rebuttal, the authors pointed out a critical error in my argument. I quickly realized, to my shame, that their point was absolutely correct.
There’s a reason why I’m sharing this embarrassing story. It shows what can happen when you become attached to your own ideas, and in the world of science, how damaging the results can be.
Given this, what do we do? How can we possibly generate an idea, or hypothesis, about why a specific aspect of nature behaves the way it does without becoming attached to it? Read on.
Science begins with observation
How do we discover nature’s truths? Well, we could simply think deep thoughts. But this alone won’t work. Why not? Because what deep thoughts would we think about in the absence of outside stimulus? So what’s needed is observation. We learn about nature by first observing it—sight, sound, smell, touch, taste. This is how Aristotle and the Ancient Greeks brought structure to science. With curious inquiry, logic-based reasoning, and pure thought, they observed nature and proposed causes for what they saw, making significant contributions to many different areas in so doing.
There was one problem with this approach, however. Measurement, or more accurately, lack thereof. The Ancient Greeks came up with theories that they didn’t test. It took many years for some to question the Ancients’ theories with experiments, Galileo arguably being the first.
Sitting at the intersection of academia and practical engineering, Galileo started observing, not passively but actively. He got his hands dirty by building equipment and conducting experiments to better understand nature’s inner workings. Swinging pendulums, firing cannons, rolling balls on inclined planes. He then used mathematics to analyze the data he gathered, believing the universe to be “written in the language of mathematics.” While Aristotle sought cause, Galileo ignored it, preferring measurement and analysis instead. What Galileo discovered contradicted Aristotle and so helped launch the scientific revolution.
A former boss of mine once told me something that left a deep imprint on me. He said, “one data point is worth one thousand opinions.” Many may sit and passively think about why something is happening, but to those like Galileo who roll up their sleeves, go out into nature, and take a data point, I say, “Wonderful!”. Because in that moment when the data point arrives, one thousand “opinions” evaporate, replaced by a fact. But what do you then do with the fact? Read on.
Induction follows observation
Once you gather (accurate) observations and data from your experiences and experiments, do you stop there? Well, if you’re curious, you don’t. You can’t. Your curiosity won’t let you. You’ll want to understand the cause behind the effects you’ve observed. This step is called induction, and the cause that’s proposed is called a hypothesis.
But is this the end of the process? Will other scientists listen to your induced hypothesis and say, “Boy, that sure sounds good to us! Great job!”? No. This would be highly unlikely. The more likely response would be something like, “Prove it!” And this is entirely appropriate (although it could be stated more politely). This is how good science flows, because how do you really know if your hypothesis is correct based on observation only? Perhaps, for example, you’re witnessing correlation and not causation. Perhaps you’re only seeing a part of the story and not the whole story. The point is that you don’t really know. This was the problem with the induced hypotheses of the Ancient Greeks; they were based on observation only. They didn’t really know.
Deduction follows induction
How do you prove a hypothesis? Well, the short answer is that you can’t. You can’t prove a theory. Many thought Newton’s Theory of Universal Gravitation was “proven” until Einstein arrived with his General Theory of Relativity. And who really knows if this is the final chapter on gravity.
But while you can’t prove a hypothesis, you can still conduct experiments to test it. The approach is to assume that the induced hypothesis is true, deduce a consequence or a prediction based on it, and then experimentally validate (or not) the prediction. The more surprising and unexpected the prediction, the more powerful the test. In other words, a prediction that doesn’t separate the new hypothesis from a former way of thinking doesn’t really demonstrate anything. Ideally, the prediction should be of something that hasn’t yet been observed.
Francis Bacon and, later, Karl Popper took this deduction process to an even higher level. Recognizing that no number of experiments can prove a hypothesis and that only one experiment is required to disprove a hypothesis, they proposed testing a hypothesis by deducing ways to eliminate it. This approach, which took the required level of intellectual creativity also to a higher level, embraced the become-your-own-worst-critic mentality. If your hypothesis can withstand your best shots, then perhaps it is correct.
One of my favorite historical “best practice” examples of how this induction-deduction process works occurred when James Clerk Maxwell challenged Rudolph Clausius’s kinetic theory of gases by first assuming the theory’s correctness, then applying a higher level of mathematics involving statistical distributions to the theory, and finally using the model to predict how gas viscosity varies with density. Everyone knew that the greater the density, the greater the friction, and thus the higher the viscosity, right? It made absolute sense, even to Maxwell, for when the model told him that density has no effect on viscosity, he set to work right away to build an experimental apparatus to disprove the “absurd” conclusion. Except it didn’t end up that way. He, together with his wife, built an excellent piece of equipment and found—well, and found that the mathematical model was absolutely correct. This was a wonderful moment in the history of thermodynamics, for this single data point greatly increased confidence among the world’s scientists in both the kinetic theory of gases and the atomic theory of matter on which it was based.
The Scientific Method
Maxwell’s example demonstrated the power of what became known as the scientific method: observe nature, induce a hypothesis, deduce a consequence, experimentally support or refute the consequence. While this description is admittedly brief, it captures the essence of the method. The top two panels in the figure below (from my book Block by Block – The Historical and Theoretical Foundations of Thermodynamics) illustrate this sequential process. Rudolf Clausius induced his hypothesis of the 1st Law of Thermodynamics from the data of, among others, Sadi Carnot and James Joule. J. Willard Gibbs then deduced 300 pages worth of consequences of Clausius’s induced hypothesis. Their combined work helped establish the new field of thermodynamics.
This approach sounds pretty good. What’s wrong with it?
It’s hard to find the Achilles’ Heel in the scientific method as stated above, isn’t it? But it’s there, implicit in its use of the singular “hypothesis.” The way science evolved often rested on the proposal and validation of a single hypothesis. And this approach chalked up some major successes. But it also chalked up some major failures. Which brings me back to the opening of this post. When you propose a single hypothesis, you become attached to it, even in the face of mounting evidence that it’s wrong. Worse yet, you run the risk of becoming guilty of what’s known as confirmation bias, the act of seeking only those data that support your hypothesis and ignoring those that don’t. You embrace and defend your hypothesis because it’s yours.
Consider some incorrect hypotheses that enjoyed long lives: the Earth-centered universe, Aristotle’s theories on motion, the phlogiston theory of fire, the caloric theory of heat, the anti-atom theory of matter. While those who held on tightly to such theories may not have come up with them, they believed in and even staked their reputations on them, resisting contrary evidence and unfortunately hindering progress. They embraced their chosen hypotheses to the grave, which led to Max Planck’s famous quote, “Science advances one funeral at a time.”
Why am I bringing this up? In the world of science, our job is to discover nature’s truths. When we bring our egos into the mix, it sets us on a collision course with learning the truth. The challenge we face is, how do we neutralize our instinct to love our own ideas?
How do you neutralize the ego? Multiple hypotheses!
T.C. Chamberlain, a late-1800s geologist and educator, proposed a path out of this dilemma. Acknowledging the attachment problem inherent to the single hypothesis, Chamberlain recommended proposing many hypotheses. Observe nature, take measurements, and then propose as many hypotheses as you possibly can that are consistent with the data. In this way, you shift the focus from a negative conflict between scientists, each embracing his or her own individual hypothesis, to a positive, exciting, and team-based conflict between ideas in which technical debate among those with differing perspectives is encouraged in order to learn and not to win. The creativity needed to propose and then attempt to eliminate many hypotheses can be very liberating, releasing you from the instinct to protect your own single hypothesis and energizing you toward true discovery.
Combining Bacon, Popper, and Chamberlain into the strong inference method of scientific discovery
John R. Platt wove the approaches of Bacon, Popper, and Chamberlain into his strong inference method (Science 1964). As illustrated in the bottom panel of the below illustration, strong inference embraces the creation of multiple hypotheses followed by a strategically designed attack on each and every one until only one—the most likely hypothesis—is left standing.
Once seen, strong inference is hard to un-see. It makes so much sense. And it opens your mind toward a critical assessment of how science is being carried out today. Look around you at the many science-related problems that need solving. How are we going about doing this? Do you hear multiple hypotheses being proposed and then attacked until one remains? This is what strong inference is all about.
Strong inference is a powerful place for science to live. By becoming our own critics, leaving our egos on the sidelines, and embracing and attacking multiple hypotheses, we will arrive at an answer that has withstood the best attacks we could offer. This answer then becomes that which guides us forward, toward whatever goal it is that we’re trying to achieve. Why would we choose to conduct science any other way than this? Why would we accept any other answer than the one that results from such a process?
I end this post with a quote by Louis Pasteur that embodies the essence of strong inference.
What I am here asking of you, and what you in turn will ask of those whom you will train, is the most difficult thing the inventor has to learn. To believe that one has found an important scientific fact and to be consumed by desire to announce it, and yet to be constrained to combat this impulse for days, weeks, sometimes years, to endeavor to ruin one’s own experiments, and to announce one’s discovery only after one has laid to rest all the contrary hypotheses, yes, that is indeed an arduous task. But when after all these efforts one finally achieves certainty, one feels one of the deepest joys it is given to the human soul to experience. Louis Pasteur, Nov. 14, 1888, in a speech given at the inauguration of the Pasteur Institute in Paris.
Illustration below from Block by Block – The Historical and Theoretical Foundations of Thermodynamics
Many thanks to Jim Faler and Brian Stutts for their helpful contributions to this post.
Identifying the Micro behind the Macro 