Policy-based evidence making

My column in the Times, with post-scripts:

 

As somebody who has championed science all his
career, carrying a lot of water for the profession against its
critics on many issues, I am losing faith. Recent examples of bias
and corruption in science are bad enough. What’s worse is the
reluctance of scientific leaders to criticise the bad apples.
Science as a philosophy is in good health; science as an
institution increasingly stinks.

The Nuffield Council on Bioethics published a report last week that found
evidence of scientists increasingly “employing less rigorous
research methods” in response to funding pressures. A 2009 survey
found that almost 2 per cent of scientists
admitting that they have fabricated results; 14 per cent say that
their colleagues have done so.

This month has seen three egregious examples of poor scientific
practice. The most recent was the revelation in The
Times last week that scientists appeared to scheme to get
neonicotinoid pesticides banned, rather than open-mindedly
assessing all the evidence. These were supposedly “independent”
scientists, yet they were hand in glove with environmental
activists who were receiving huge grants from the European Union to
lobby it via supposedly independent reports, and they apparently
had their conclusions in mind before they gathered the evidence. Documents that have recently come to light
show them blatantly setting out to make policy-based evidence,
rather than evidence-based policy.

Second example: last week, the World Meteorological Organization
(WMO), a supposedly scientific body, issued a press release stating that this is likely to
be the warmest year in a century or more, based on surface
temperatures. Yet this predicted record would be only one hundredth of a degree above
2010 and two hundredths of a degree above 2005 — with an error
range of one tenth of a degree. True scientists would have said:
this year is unlikely to be significantly warmer than 2010 or 2005
and left it at that.

In any case, the year is not over, so why the announcement now?
Oh yes, there’s a political climate summit in Lima this week. The
scientists of WMO allowed themselves to be used politically. Not
that they were reluctant. To squeeze and cajole the data until they
just crossed the line, the WMO “reanalysed” a merger of five data
sets. Maybe that was legitimate but, given how the institutions
that gather temperature data have twice this year been caught
red-handed making poorly justified adjustments to “homogenise” and
“in-fill” thermometer records in such a way as to cool down old
records and warm up new ones, I have my doubts.

In one case, in Rutherglen, a town in Victoria, a recorded
cooling trend of minus 0.35C became a reported warming trend of plus 1.73C
after “homogenisation” by the Australian Bureau of Meteorology. It
claimed the adjustment was necessary because the thermometer had
moved between two fields, but could provide no evidence for this,
or for why it necessitated such a drastic adjustment.

Most of the people in charge of collating temperature data are
vocal in their views on climate policy, which hardly reassures the
rest of us that they leave those prejudices at the laboratory door.
Imagine if bankers were in charge of measuring inflation.

Third example: the Royal Society used to be the gold standard of
scientific objectivity. Yet this month it issued a report on resilience to extreme
weather that, in its 100-plus pages, could find room for not a single graph to show
recent trends in extreme weather. That is because no such graph
shows an upward trend in global frequency of droughts, storms or
floods. The report did find room for a graph showing the rising
cost of damage by extreme weather, which is a function of the
increased value of insured property, not a measure of weather.

The Royal Society report also carefully omitted what is perhaps
the most telling of all statistics about extreme weather: the
plummeting death toll. The global probability of being killed by a
drought, flood or storm is down by 98 per cent since the 1920s and
has never been lower — not because weather is less dangerous but
because of improvements in transport, trade, infrastructure, aid
and communication.

The Royal Society’s decision to cherry-pick its way past such
data would be less worrying if its president, Sir Paul Nurse, had
not gone on the record as highly partisan on the subject of climate
science. He called for those who disagree with him to be
“crushed and buried”, hardly the language of Galileo.

Three months ago Sir Paul said: “We need to be aware of those who mix up
science, based on evidence and rationality, with politics and
ideology, where opinion, rhetoric and tradition hold more sway. We
need to be aware of political or ideological lobbyists who do not
respect science, cherry-picking data or argument, to support their
predetermined positions.”

If he wishes to be consistent, he will therefore condemn the
behaviour of the scientists over neonicotinoids and the WMO over
temperature records, and chastise his colleagues’ report, for these
are prime examples of his point.

I am not hopeful. When a similar scandal blew up in 2009 over
the hiding of inconvenient data that appeared to discredit the
validity of proxies for past global temperatures based on tree
rings (part of “Climategate”), the scientific establishment closed
ranks and tried to pretend it did not matter. Last week a further instalment of that story came to light,
showing that yet more inconvenient data (which discredit
bristlecone pine tree rings as temperature proxies) had
emerged.

The overwhelming majority of scientists do excellent, objective
work, following the evidence wherever it leads. Science remains (in
my view) our most treasured cultural achievement, bar none. Most of
its astonishing insights into life, the universe and everything are
beyond reproach and beyond compare. All the more reason to be less
tolerant of those who let their motivated reasoning distort data or
the presentation of data. It’s hard for champions of science like
me to make our case against creationists, homeopaths and other
merchants of mysticism if some of those within science also
practise pseudo-science.

In all the millions of scientific careers in Britain over the
past few decades, outside medical science there has never been a
case of a scientist convicted of malpractice. Not one. Maybe that
is because — unlike the police, the church and politics —
scientists are all pure as the driven snow. Or maybe it is because
science as an institution, like so many other institutions, does
not police itself properly.

 

Postscript:

 

For those interested in further details of some of the incidents
mentioned in this artice, here follow some links and quotes.

1. On neonicotinoids, here is what the scientists wrote:

"Based on the results of the meeting
in Paris the following was agreed that the will be published in
peer-reviewed journals. Building on these papers a research paper
will be submitted to Science (first choice) or Nature (second
choice) which would introduce new analyses and findings across the
scientific disciplines to demonstrate as convincingly as possible
the impact of neonicotionoides on insects, birds, other species,
ecosystem functions, and human livelihoods. This high-impact paper
would have a carefully selected first author, a core author team of
7 people or fewer (including the authors of the initial four
papers), and a broader set of authors to give global and
interdisciplinary coverage. A significant amount of the supporting
evidence will be in the official Supporting Online Material
accompanying the paper. A parallel « sister » paper (this would be
a shorter Policy Forum paper) could be submitted to Science
simultaneously drawing attention to the policy implications of the
other paper, and calling for a moratorium in the use and sale of
neonicotinoid pestcides. We would try to pull together some major
names in the scientific world to be authors of this paper. If we
are successful in getting these two papers published, there will be
enormous impact, and a campaign led by WWF etc could be launched
right away. It will be much harder for politicians to ignore a
research paper and a Policy Forum paper in Science The most urgent
thing is to obtain the necessary policy change to have these
pesticides banned, not to start a campaign. A stronger scientific
basis for the campaign will hopefully mean a shorter campaign. In
any case, this is going to take time, because the chemical industry
will throw millions into a lobbying exercise."

2. On Rutherglen, an Australian weather station, the Bureau of
Meteorlogy added a page of explanation after the scandal was drawn
to their attention. Unfortunately this made things worse by
admitting that the record "does not list a site move" and contains
"no firm evidence exists [of] exact location". Moreover, the page
reveals precisely nothing about how the adjustment was done to
account for this unknown site move. As Jo Nova commented at the time:

"The BOM has added a page listing
“Adjustments”. It’s two years late, inadequate and incomplete.
Skeptics shouldn’t have had to ask for it in the first place, and
we still don’t have the algorithms and codes, or rational answers
to most questions.  No one can replicate the mystery black box
homogenisation methods of the BOM — and without replication, it
isn’t science. There is still no explanation of why an excellent
station like Rutherglen should change from cooling to
warming, except for vague “statistics”, or why any station
should be adjusted without documentary evidence, based on
thermometers that might be 300km away."

The new “adjustments”
page doesn’t resolve much at all. There are still blatant errors —
The changes to long term trends in minima are not “neutral”, 
but increase the trend by nearly 50% (See 
Ken Stewart’s site here and 
the finished set here). The hottest day in Australia was almost
certainly not in Albany in 1933 (which remains
uncorrected at 51C). Many 
maximums have been adjusted and become lower than minimums.
Those mistakes did not exist in the raw data. The homogenisation
has created them, like the new 
discontinuity in Deniliquin.

The adjustments page is just a glorified rehash of the
same old excuses

Effectively the bureau
is saying “we need large mysterious transformations of data to make
Australian trends look like international trends”.What serious
climate scientist thinks Australia is supposed to get hotter,
colder, wetter, drier, or cloudier with the exact same timing and
patterns to the rest of the world? Even high schoolers know that
when it rains on the East Coast with El Ninos, it’s not raining on
the other side of the Pacific. Just because other homogenizations
have produced the same trends by blending data to the point where
it is unrecognisable does not make it “good” science.

Lots of international bankers were
marketing the same overrated mortage bundles. Anyone
want to buy subprime science — I have a collateralized trend for
sale

Three years ago the
independent audit team, with Senator Cory Bernardi,
asked for an ANAO audit of the BOM’s “High Quality” HQ data
set. The BOM was not enthused. They dumped the HQ set that they had
previously lauded and set up a new one called “ACORN”. We listed
some of the 
errors in June 2012. Two years on, nothing much appears to have
changed. They still haven’t released the algorithms used in the
homogenization process. They are still using stations 
more than 100km away, some 600 km away, to “adjust”
temperatures.  The mystery black box adjustments are still
producing inexplicable nonsense, and the BOM still can’t explain
why — on individual stations 
like Rutherglen and 
Bourke — anyone should find their adjustments necessary and
scientifically justified. There is 
no documentation showing Rutherglen has moved. But there is
documentation suggesting perhaps 
Bourke’s deleted “hottest” day really might have been 125F in
1909.

The BOM’s active silence on
the 
long hot recordsof the late
1880s and 1890s suggests they are more interested in promoting
one message — “it’s warming” — rather than being custodians of the
real and more complicated history of the Australian climate.

 

3. On tree-ring proxies, the new data concerns "out of sample"
data on tree rings from bristlecone pines on Sheep Mountain in
California. This data set gives a sharp hockey stick up till 1980,
implying rapidly increasing growth of trees as 1980 approached. But
critics have alleged that this is unlikely to be down to
temperature, and more iikely a "strip-bark" phenomenon, whereby the
tree regrows rapidly after damage by goats and sheep. The absence
of data from after 1980 has always been puzzling. Well, as Steve
McIntyre now reports, the data have now been collected and they
show low growth rates during the warm years of 1980-2000. Once
again, Jo Nova summarises the story especially well:

The  obvious message is that
these particular proxies don’t work now and probably never did, and
that this hockeystick shape  depends
on not using tree rings after 1980.

More important than the details of
one proxy, is the message that the modern bureaucratized
monopolistic version of “science” doesn’t work. Real scientists,
who were really interested in the climate, would have published
updates years ago. (Indeed, would never have published the
hockeystick graph in the first place. Its dysfunctional combination
of temperatures and truncated proxies is mashed through a maths
process so bad it produces a hockey stick most of the time even if
the data is replaced by red noise.)

The screaming absence of this
obvious update for so long is an example of what I call the “rachet
effect” in science — where only the right experiments, or the right
data, gets published. It’s not that there is a conspiracy, it’s
just that no one is paid to find the holes in the theory and the
awkward results sit buried at the bottom of a drawer for a
decade.  The cortex soaked in confirmation-bias couldn’t
figure out how to explain them.

See Climate Audit for McIntyre’s
view on Salzer et al 2014.

"The new results of Salzer et al 2014 (though not candid on the
topic) fully demonstrate this point in respect to Sheep Mountain.
 In the warm 1990s and 2000s, the proxy not only doesn’t
respond linearly to higher temperatures, it actually goes the wrong
way.   This will result in very negative RE values for
MBH-style reconstructions from its AD1000 and AD1400  networks
when brought up to date, further demonstrating these networks have
no real “skill” out of sample.

We’ve also heard over and over about how “divergence” is limited
to high-latitude tree ring series and about how the Mann
reconstruction was supposedly immune from the problem.
 However, these claims mostly relied on stripbark chronologies
(such as Sheep Mountain) and the validity of such claims is very
much in question.

As previously discussed on many occasions, stripbark
chronologies have been used over and over in the canonical IPCC
reconstructions, with the result that divergence problems at Sheep
Mountain and other sites do not merely impact Mann et al 1998-99,
but numerous other reconstructions.  Even the recent PAGES2K
North America reconstruction uses non-updated Graybill stripbark
chronologies.  It also ludicrously ends in 1974.  So
rather than bringing the Mann et al network up-to-date, it is even
less up-to-date."

Note that bristlecone pines were
never supposed to be used as climate proxies anyway. They are a
rather unusual species — their growth was thought to be CO2-limited
rather than limited by temperature or moisture, so they responded
well at first to the increase in CO2 in the 20th century, though
obviously something else is going on after 1980. This graph and
these results apply only to one situation — not all tree rings. But
the  failure of review applies to the whole scientific
community.

The IPCC adopted the hockeystick for
their logo shortly after Mann produced it, but long since dropped
it. Where was the all-marvelous, hallowed, IPCC “expert”
review?

 

PPS After this article was published an extraordinary series of
tweets appeared under the name of Richard Betts, a scientist at the
UK Met Office and somebody who is normally polite even when
critical. He called me “paranoid and rude” and made a series of
assertions about what I had written that were either inaccurate or
stretched interpretations to say the least. He then advanced the
doctrine that politicians should not criticize civil servants. The
particular sentence he objected to was:

Most of the people in charge of
collating temperature data are vocal in their views on climate
policy, which hardly reassures the rest of us that they leave those
prejudices at the laboratory door.

He thought this was an unjustified attack on civil servants.
However, if you read what I said in that sentence, it is that (1)
people in charge of collating temperature data are vocal in support
of certain policies – which is not a criticism, just a statement;
and (2) that we need reassurance that they do not let that
consciously or unconsciously influence their work, which again is
not a criticism, let alone an attack, merely a request for
reassurance. Certainly there is no mention of civil servants, let
alone by name, and nothing to compare with an attack on me by name
calling me paranoid and rude.

Is the first assertion true? I had in mind Jim Hansen, who was
in charge of GISS, a data set for which serious questions have been
raised about adjustments made that warm the present or cool the
past, and who is prepared to get himself arrested in protest
against fossil fuels. I also had in mind Phil Jones, partly in
charge of HADCRUT, who also is not shy with his views. I was not
thinking of Julia Slingo of the Met Office, because I do not think
of the Met Office as a collater of temperature data, but perhaps I
should have been. And then there’s Australia’s BoM. And indeed the
RSS data, whose collater, Dr Carl Mears, fumes at the way
“denialists” talk about his data. Hardly objective language.

Is my request for reassurance reasonable? In view of the
Australian episodes, the GISS adjustments, the USHCN story from
earlier this year (see here) – all of which raised doubts about the
legitimacy of adjustments being made to the temperature data – then
yes, I think I am. Do I think the data are fatally flawed? No, I
don’t. I happily accept that all the data sets show some warming in
the 1980s and 1990s and not much since and that this fits with the
satellite data. But do I think such data can be used to assert that
this is the warmest year, by 0.01 degrees, a month before the year
ends? No, I don’t. I think people like Dr Betts should say as
much.

As of this writing, Dr Betts’s latest tweet is:

If ‪@mattwridley wants to
criticise climate policy then he's got every right, but attacking
scientists is wrong.

Well, if by attacking he means physically or verbally abusing,
then yes, I agree, but I don’t do it. I don’t call people by name
“paranoid”, for example. But criticizing scientists should be
allowed surely? And asking for reassurance? Come on, Richard.

The WMO “re-analysed” a data set to get its 0.01 degree warmest
year. What was that reanalysis and has it been independently
checked? I would genuinely like to know. I stopped taking these
things on trust after the hockey stick scandal.

The thrust of my article was that the reputation of the whole of
science is at risk if bad practices and biases are allowed to
infect data collection and presentation, and that science like
other institutions can no longer take public trust for granted. A
reaction of bluster and invective hardly reassures me that science
takes my point on board. For the moment, I remain of the view
that

The overwhelming majority of
scientists do excellent, objective work, following the evidence
wherever it leads. Science remains (in my view) our most treasured
cultural achievement, bar none. Most of its astonishing insights
into life, the universe and everything are beyond reproach and
beyond compare.

But Dr Betts’s reaction has weakened my confidence in this
view.