This week I’m at the biennial meeting of the International Society of Microbial Ecologists (ISME) in Seoul. During lunch yesterday there was a special “bird’s eye view” talk given by Dr. James Prosser, a preeminent microbiologist from the University of Aberdeen. the subject of the talk was the perceived over-indulgence of the community in observational, rather than hypothesis-testing, studies. The organizers asked him to be provocative and he certainly was. The talk was stimulating and well thought-out, but begged for a counter-point to balance his strong views on the subject. In defense of observational microbiology here’s my attempt at a counter-point to the talk.
Dr. Prosser’s thesis was that microbial ecology has lost it’s way a bit, with too many researchers relying on “observational” studies that simply report the state or composition of a microbial community without really testing any hypotheses. The classic example here is a 16S gene phylotyping or metagenomic study that just explore an environment for the sake of exploration. The community often refers to this as stamp collecting, and while I agree that it’s not the right way to “do science”, I disagree strongly with Dr. Prosser about why this is and what we should do about it. A second, but related, complaint was that the community is spending too much time and effort developing new tools and methods.
Point 1: Why we need new observations
Observation is the first, and most essential, step in the scientific method that we all learned in elementary school. Before anyone can formulate a question and develop a hypothesis they must observe something that they cannot explain. I think that Dr. Prosser might be missing two essential points here; it is difficult to observe microbial communities with sufficient detail to develop interesting questions, and, microbially, the word is a really, really big place that requires a lot of observation. I would not be surprised if Antoni van Leeuwenhoek, the amateur scientist who made the first biological observations with a microscope in the 17th century, got a lot of flack from contemporary philosophers for tinkering with instruments and wasting time using them to look at water drops instead of debating the “real” questions about life.
The “omics” tools that Dr. Prosser is quick to dismiss are today’s microscope; vastly more complex but capable of producing magnitudes more information. Time and money spent learning these tools and improving their application is time and money well spent. So is time and money spent applying these tools in what might appear to be idle contemplation of the world. I’m not saying we shouldn’t be efficient and sensible with scientific studies, but sometimes the most obvious questions aren’t the most interesting. It can take a lot of observations to observe something worthy of a testable hypothesis.
Point 2: Other fields are way better than us, and they invest heavily in observation
Biology in general and microbial ecology in particular is often viewed as a “softer” science by those more aligned with the fields of math, physics, and chemistry. I had a bit of a laugh when Dr. Prosser suggested (and my sincere apologies if I’ve misinterpreted his statement – here or elsewhere) that this was in part because of our obsession with methods development. I think this couldn’t be farther from the truth. The methods that have become prevalent since the onset of the genomics era have taken microbial ecology a long way toward becoming a quantitative, process-driven field of study. Big data, statistics, and modeling are all skills that the more traditionally quantitative sciences mastered a long time ago. It’s time for microbial ecology to join the 21st century.
The value that these other fields place on observation can be seen in the way those communities invest in observational programs. I doubt that physical oceanographers are lectured at their meetings about the need to reduce the highly successful Argo program, a global network of observational floats that doesn’t seek to address any particular hypothesis (though individual floats or sets of floats may be deployed for this purpose). They understand that carefully and systematically collected and curated data is enormously valuable because it is the feedstock of future hypotheses and the means to test them. Microbial ecology is decades behind the power curve here. Astronomy (e.g. Kepler) and physics (e.g. Ice Cube) are additional examples. The latter field is particularly interesting, as it has evolved to the point where it recognizes a need for two distinct subcultures; observationalists and theorists.
Point 3: Observations are difficult, so cut those that do it a break
The reason (I’m guessing) that physics makes a distinction between observationalists and theorists is that both of these subfields are difficult to master and require different skill sets. I don’t think we’re at that point with microbial ecology, but it might be time for some recognition of how difficult it is to make meaningful observations. During his talk Dr. Prosser lamented at how eager new graduate students are to begin their programs of study mastering new skills instead of developing interesting hypotheses. I think that’s a deeply flawed line of thinking. Yes, one should always be developing hypotheses as they go about their work, but if they never take the time to master technical skills they’ll never be in a position to do anything about it! Molecular work at the lab bench, coding skills, and advanced statistics are essential tools in the field. One should learn them as early as possible so that they know how to frame testable hypotheses and design realistic experiments!
Naturally, having spent the first 2 or 3 years of graduate school learning some lab techniques and developing proficiency with a programming language a graduate student would like to have something to show for their efforts. I think that many of the “observational” papers that Dr. Prosser referred to reflect the work of junior scientists in this position or more senior scientists bold enough to try and master new methods (a particularly rare breed!). It’s doing no harm to have these papers out there. One will not get a postdoc, faculty position, or tenure on them alone, so what’s the issue? At worst they pad the CV of a scientist who will go on to develop hypotheses and do more interesting work, at best they add to a critical institutional knowledge base from which the most interesting hypotheses will eventually come.
Point 4: Not all hypotheses are worth pursuing – and some observations are worth more than some hypotheses
As part of his (extensive) preparations for the talk Dr. Prosser went to the effort to catalog recent papers as purely observational or hypothesis testing, finding 62 % to be purely descriptive and 70 % producing, but not testing, a new hypothesis. I’m not sure how big a problem this really is. As mentioned before it takes time to make meaningful observations in microbial ecology. At the end of it you might not have anything worth pursuing, or it may not be logistically feasible to pursue (a common problem in marine and polar systems, for example), but you’d still like a paper for your efforts. I think the fundable, testable, interesting hypotheses are getting tested. The less interesting and currently not testable ones are not, and that’s okay. In the meantime however, and lacking any systematic sampling program (Argo, Kepler…), all these observations are slowly building a knowledge base. Dr. Prosser has perhaps never used the genomes in Genbank to test a hypothesis. I have. I’m glad for all the purely observational studies that have produced the data I can use.
I had some other things to say, but I think I better end it here and go find some Korean BBQ instead. As a parting thought, I think the field – to some degree split along generational lines – is afraid of data and observations. People don’t know what to do with it and they don’t know how to handle it. That’s beginning to change. We need more observations now, not fewer. But they need to be better observations, collected carefully and systematically, and curated and shared. I’m deeply concerned that the continued grumbling about too many observations and too much methods development is ruining our ability to do this.
Time for BBQ.
