The problem with overqualified research

Not all research findings turn out to be true. Of those that are tested, some will need to be amplified, others refined or circumscribed, and some even rejected. Practicing researchers learn quickly to qualify their claims, taking into account the possibility of improved measurements, more stringent analyses, new interpretations, and, in the extreme, experimental or theoretical error.

Qualification is routinely achieved by changing the form of the verb. Take this example from a physics article: “This lack of correlation may be due to the increased inhomogeneity expected for the MG method.” The authors’ use of “may” allows for the lack of correlation to be due to something else.

The softening of any statement with a word or words is called hedging. It can be accomplished in many ways. One has already been indicated—inserting an auxiliary modal verb, “may,” “might,” “can,” or “could.” Another common way is to add a mitigating phrase or clause such as “It is possible that” or “This suggests that.”

Unhappily, the temptation for an author is to keep accumulating hedges to secure progressively more protection against potential criticism. As a result, it becomes correspondingly harder for the reader to see what the author is actually  claiming.

In these two sentences from a molecular biology article, there’s a doubling of hedges, here in italics: “it is possible that … G9a-mediated H3K9 and H3K27 monomethylation might serve as stimulating factors” and “Possible candidates might comprise Jarid2 and Snail1.” Both instances of “might” can be eliminated without affecting the provisional status of the claims.

Altogether this article contained 44 instances of “might,” “can,” “could,” “likely,” and varieties of “possibly,” but none of “may.” Because of its potential for ambiguity, some authorities advise authors not to use “may” when what’s being signalled is possibility instead of permission.

This next example is from an oncology article: It has five hedges in one sentence: “Although, on balance, testosterone might possibly be beneficial to the cardiovascular system, testosterone might also have some detrimental effects.” Two of the hedges can be safely eliminated; for example, “Although testosterone might be beneficial to the cardiovascular system, testosterone might also have some detrimental effects.” Yet with two opposing uses of “might,” the sense of equivocation remains, despite the restrictive “some.” Given that “might” is seen as being more tentative than “may,” a less equivocal version is “Although testosterone may be beneficial to the cardiovascular system, it might also have some detrimental effects.” Whether this edit is valid, though, needs expert judgement. In all, this article contained 54 instances of “might.”

There can be very good reasons for using “might” so often—but it’s unusual.  I estimated its general frequency of occurrence from an analysis of a sample of about 1000 articles from the physical, life, and behavioural sciences, mathematics, engineering, and medicine. Across the sample, 50% of articles containing “might” used it just once or twice and only 1% used it as many as 21 times or more.

In a more detailed analysis, I also found that as the frequency of the modals “may,” “might,” “can,” and “could”  increased in an article, the frequency of pairs of hedges such as “could indicate” and “possibly suggests” increased still more. The implication is that multiple hedges within a sentence are a response to the prevailing rate of protection within the article—rather like having to talk more loudly in a noisy restaurant.

Whatever the cause, hedging can clearly escalate. But there’s a countervailing force: the need to deliver impact, usually in the abstract or conclusion of an article—something to be remembered by the reader, a take-home message. Since impact is lessened by hedging, summary statements may succumb to under-qualification and go beyond what the data can safely support. At worst, they can lead to public correction and retraction of the article.

How, then, to choose the right amount of hedging?  Every claim is unique to its particular context and it’s difficult to generalize. Even so, there’s some strategic advice, popularly attributed to Albert Einstein, that may be helpful. Applied to hedging, it’s this: Hedging should be as simple as possible—but never simpler.  Although finding the right balance is another challenge, it at least encourages restraint.

Featured Image Credits:  Julian Schöll on Unsplash 

The post The problem with overqualified research appeared first on OUPblog.