Calling Bullshit: The Art of Skepticism in a Data-Driven World

by Carl T. Bergstrom

Another issue to be on the lookout for is whether the claims in a paper are commensurate with the venue in which it is published. As we mentioned, journals occupy different positions in a hierarchy of prestige. All else equal, papers published in top journals will represent the largest advances and have the highest credibility. Less interesting or less credible findings will be relegated to less prestigious outlets. Be wary of extraordinary claims appearing in lower-tier venues. You can think of this as the scientist’s version of “if you’re so smart, why aren’t you rich?” Thus if a paper called “Some Weights of Riparian Frogs” lists the weights of some frogs in the little-known Tasmanian Journal of Australian Herpetology, there is relatively little cause for concern. A table of frogs’ weights, while possibly useful to some specialists in the area, is far from a scientific breakthrough and is well suited for the journal in question. If, however, a paper called “Evidence That Neanderthals Went Extinct during the Hundred Years’ War” appears in the equally low-profile Journal of Westphalian Historical Geography, this is serious cause for concern. That finding, if true, would revolutionize our understanding of hominin history, and shake our notion of what it means to be human. Such a finding, if true, would appear in a high-profile journal.