The Scientific Method, Redefined

Introduction

In this blog post, I criticize two of my idols in science, physicists Neil Degrasse Tyson and David Deutsch.  Both of them have written or spoken about the scientific method, and both of them have missed the mark.  Dr. Tyson says that science is all about prediction, and not about explanation.  Dr. Deutsch says that science is about explanations, but gives an unsatisfactory definition of what qualifies as an explanation.  In this post, I set out my own definition for the scientific method. 

The scientific method is a search for truth through objective reasoning.  We use the scientific method to solve scientific problems, and step-by-step, we improve our understanding of reality.  When we apply the scientific method, the end product should be a good explanation for the phenomenon we are investigating.  The idea of "a good explanation" was put forward by David Deutsch, but without clear criteria for what makes a good explanation.  I think a good explanation has the following attributes:  

A good explanation must define a process which changes some aspect of reality.

The process must be observed in action.

The process must be measured and quantified.

The explanation must reconcile theory and observation.

The work must meet standards of objectivity for scientific research. 

The explanation must be verified through successful prediction of experimental results or observations of real-world changes.

The explanation will often explain other phenomena in areas unrelated to the initial inquiry.
The explanation must be subjected to peer review, and published in a reputable journal.

The Scientific Enlightenment

In general, science is the modern way that we search for truth and develop useful technologies for civilization.  The deliberate practice of scientific investigation began in the mid-1600s.  Science greatly accelerated human progress in terms of technology, understanding of the earth and the cosmos, literacy, health, prosperity, government and all other aspects of civilization.  We are living in an age of scientific enlightenment, the longest continuous period of enlightenment in history. 

The scientific method is critical to that enlightenment.  The process of objective reasoning is essential not only to science, but to most other aspects of civilization, including government, law, economics, journalism, education and medicine.  Objective reasoning is a social process, involving not just individuals, but represents how society comes to conclusions about various issues.  Objective reasoning requires academic and political freedom, free discourse and argument, unrestricted access to data and information, and equality in public debate.

We were all taught the basics of the scientific method in middle school.  According to the Oxford Online Dictionary, the scientific method consists of “systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses”.  That sounds pretty close to what I remember from middle school. 

But I now think that this description is incomplete.  What place is there in this process for human intuitions, guesses, and thought experiments?  How do we choose what to observe and measure?  What does it mean when we form a hypothesis, and where does it come from?  Where is the part about free access to primary data?   Where is the part about replicability of experiments?  Where is criticism and peer review?  Where is the part about absence of conflict of interest?  Where is publication and dissemination of results? 

I would like to convene a team of the greatest scientists to rewrite the scientific method.  I would like to enlist not only giants of scientific accomplishments, but those who have the gift of explanation of complex issues.  I want to enlist polymaths, those who have a sense of the broader meaning of objective reasoning for all humanity.  I want to ask Richard Feynman, Jacob Bronowski, Stephen J. Gould, Carl Sagan and David Attenborough.  But Feynman, Bronowski, Gould, and Sagan are all gone.  We will have to do our best without them. 

Looking further in the past, it would have been interesting to hear the opinions of Albert Einstein, Nicola Tesla, Neils Bohr and Enrico Fermi; but especially those who did not receive equality in research or recognition:  Lise Meitner, Marie Curie, Barbara McClintock, and Rosalind Franklin, but who made civilization-altering discoveries anyway.  We will also have to imagine what they would have said.

Finding Truth

In a sense, the scientific method began with the ancient Athenians.  Socrates and others advocated a process of discourse and argument to find truth.  Finding relevant truth is the goal of science. 

From that premise alone, we see that science is intrinsically democratic and egalitarian.  Conclusions must always be subject to challenge and debate.  For that there must be freedom to replicate experiments and calculations.  There must be unrestricted access to primary data, in order to review others’ work. 

Very early in my career as a geologist, I made a presentation in which I gave my interpretation of a geologic problem.  After the presentation, a wise mentor corrected me.  I learned that I must never give my conclusions in the first person.  Instead of “I say that the conclusion is X”, one should say, “The data says that the conclusion is X.”  The best analysis is always when the data speaks for itself.

The premise is that objective truth exists, and that it is accessible by everyone.  The scientific method is the process by which we analyze the world around us to illuminate objective truth for ourselves and others.

Explanations, Empiricism, and Mathematics

Physicist David Deutsch writes about the scientific method in his books “The Fabric of Reality” and “The Beginning of Infinity”.  I like Deutsch’s view, that the purpose of science is to make good explanations.  I agree with Deutsch that explanations matter. 

Empiricism is one way to make successful predictions about future events, but it is conceptually empty.  Empirical results are limited, by nature, to the specific conditions under which precedents have occurred.  As Deutsch explains, Empiricism lacks reach.  Empiricism will never take us outside the envelope of prior experience.  By extension, all forms of science which rely on empirical results are also limited.  Without explanation, we cannot know the bounds of our models, and we cannot transfer our understanding into new realms, with new insights. 

I was horrified to read an interview of celebrity physicist Neil Degrasse Tyson, in the book “But What If We’re Wrong”, by Chuck Kloosterman.  The passage is as follows:

“In physics, when we say we know something, it’s very simple.  Can we predict the outcome?  If we can predict the outcome, we’re good to go, and we’re on to the next problem.  There are philosophers who care about the understanding of why that was the outcome.  Isaac Newton [essentially] said, ‘I have an equation that says why the moon is in orbit.  I have no fucking idea how the Earth talks to the moon.  It is empty space – there’s no hand reaching out.’  He was uncomfortable about this idea of action at a distance.  And he was criticized for having such ideas, because it was preposterous that one physical object could talk to another physical object.  Now, you can certainly have that conversation.  But an equation properly predicts what it does.  That other conversation is for people having a beer.” 

NO, Dr. Tyson.  Just no.  Explanations matter.  Equations without explanations are empty, and their predictions limited.  It matters whether the unseen force causing action at a distance is made of gravitons, or if the action is caused solely by the curvature of space-time. It is precisely because Newton was unable to provide an explanation for gravity's action at a distance that the science was incomplete.  Einstein's gravity is an improvement in providing an explanation for how gravity works, but is probably not the final word on the matter.

 In college hydrology, we empirically derived equations for the flow of water through a pipe at different velocities.  Were we finished, and “on to the next problem”?   Of course not.  The equations represented processes within the pipe with physical meaning.  Only after we had observed the flow of liquid with dye tracers in transparent pipes could we assign meaning to the equations.  We could assign names to processes we observed, such as laminar flow and turbulent flow.  And those explanations give rise to practical new predictions, such as the erosion rate in the production tubing of an oil well, depending upon the velocity of the flow, and the flow regime of the fluid. 

As David Deutsch puts it, good explanations have reach.  “Reach is the ability of some explanations to solve problems beyond those for which they were created to solve.”  Thus, Newton’s theory of gravity solved the rate at which objects on earth fall to the ground, and it also solved the problem of the orbital paths of the planets. 

Observation, Theory and Experiment

There is a duality in scientific work that has existed throughout history.  Scientists come in two types: theorists and experimenters.  We can contrast Plato, for whom truth is found in the mind and imagination, with Aristotle, for whom truth is found through objective observation.  Theorists include Copernicus, Newton, Einstein ,Tesla and Feynman; observers/experimenters include Darwin, Tycho, Galileo, Edison and Fermi.  The two sides are mutually dependent, neither can make progress without the other. 

Our textbook definition of the scientific method is “systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses”. 

But where do we begin?  What informs our first observations?  Some scientists, including Deutsch and Stephen J. Gould, have emphasized the importance of human intuition and inspired guesswork.  These authors suggest that theory precedes observation.  I am skeptical of this claim.  It seems to me that theory must always address a problem posed by an observation.  Einstein never acknowledged the importance of the Michelson-Morey experiment as inspiration for the theory of special relativity.  But he must have been aware of this landmark experiment, which proved, against all expectations, that the speed of light is constant in all directions.  From that fact, it is possible to develop the theory of relativity.  Without that experiment, however, there is no problem to solve, and no reason to develop the explanation.

However observation begins, the scientific process must alternate between theory and experiment.  Theory shows what observations are necessary and informs experimental design.  Experimental results pose problems which new theories must solve. 

Ancillary Elements of the Scientific Method

• Data must be freely available, for re-calculation and verification by others.
• The process of gathering the data must be clearly identified, so the data may be critiqued as well as the analysis.  Data gathering must be replicable by other researchers.
• Sources of funding must be identified.  Potential conflicts of interest identified, and if possible, eliminated.
• Results should be subject to formal peer review before general release to the press or public.
• Results should be published in reputable journals to make the information available to other researchers.
• Problems encountered in the research should be clearly revealed. 
• Hypotheses should be framed in a way that allows refutation through further work.  

These ancillary processes of the modern scientific method are important, and must be observed if the results of a study are to be considered “scientific”.

Process, Observation, Measurement, Quantification, Validation

David Deutsch makes a wonderfully clear case about how the scientific method is about finding good explanations for scientific problems. The explanation of phenomena is the end goal, and it supersedes empiricism and mathematical modeling as the goal of science. 

But Deutsch falls flat when defining what makes a good explanation.  According to Deutsh, a good explanation is: “hard to vary, while still accounting for what it purports to account for.”  That’s it.   That’s the best definition given.  Deutsch gives several good examples of good explanations, and contrasts those with bad explanations, but still, the definition is weak.  There is no measure for what makes a definition hard to vary, and what the explanation should account for is left undefined.

But let’s pursue the idea of a good explanation as the goal of the scientific method.   A good explanation should answer a scientific problem.  That explanation should identify a process which transforms or changes some aspect of reality.  The process must be observable.  We should be able to measure and quantify the process.  And finally, we should be able to validate the process through experiment, or through verified predictions of some other behavior of the natural world, or both.  That’s it.

I have a former colleague who does not believe in human-caused climate change.  In his view, the world has been warming since the end of the last ice age, and the current warming of the globe is just a continuation of that trend.  Is this a good explanation?

Of course not.  This explanation has not identified any process for warming the planet, only the fact that some warming has occurred at some time in the past.  There is no observation of a current process, which is a continuation of the former process, and no measurement, quantification, or validation that the process which ended the last ice age is still operative. 

In contrast, the process of how greenhouse gases warm the earth is observed.  The buildup of greenhouse gases is observed, and the amount of heat retained by these concentrations of gases has been measured.   The total heat retained by greenhouse gases has been quantified, and can be compared to expected changes in air temperature, water temperature, volumes of melted ice and rising sea level.  Additional predictions have been made and confirmed regarding the behavior of trade winds and the frequency of extreme weather events.  This explanation has reach, and accounts for large-scale changes in the biosphere, in terms of gardening zones, movement of microclimates and timing of animal migrations.

The explanation of climate change by human emissions of greenhouse gases is a good explanation.  It is a good explanation because it identifies a process.  The process can be observed, measured, and quantified.  The measured process can be compared to predictions from theory, and verified by successful prediction of changes in the atmosphere and ocean.  And the explanation has reach in terms of explanation of seemingly unrelated phenomena. 

Conclusion

The scientific method is a search for truth through objective reasoning.  We use the scientific method to investigate problems in our understanding of reality.  Our goal is to develop good explanations which resolve these problems. 

The scientific method includes a number of ancillary processes to ensure the integrity of the investigation and the results.  We have made progress in understanding reality when we find a good explanation for our problem.  A good explanation meets the following criteria. 

A good explanation:

• Must define a process which changes some aspect of reality.
• The process must be observed in action.
• The process must be measured and quantified.
• The explanation must reconcile theory and observation.
• The work must meet the standards of objectivity listed above as ancillary elements of the scientific method.
• The explanation must be verified through successful prediction of experimental results or observations of real-world changes.
• The explanation will often explain other phenomena in areas unrelated to the initial inquiry.
• The explanation must be subjected to peer review, and published in a reputable journal.

--

References

Isaac Asimov, 1988, The Relativity of Wrong, 225 p.

David Deutsch, 2011, The Beginning of Infinity, 487p.

David Deutsch, 1997, The Fabric of Reality, 390 p.

Richard Feynman, 1998, The Meaning of It All, 288 p.

Richard Feynman, 1999, The Pleasure of Finding Things Out, 144 p.

Stephen J. Gould, ??, in an essay I can no longer locate.

Chuck Kloosterman, 2016, But What if We’re Wrong, 272 p.