Tom Stafford's Blog, page 10

3 salvoes in the reproducibility crisis

The reproducibility crisis in Psychology rumbles on. For the uninitiated, this is the general brouhaha we’re having over how reliable published psychological research is. I wrote a piece on this in 2013, which now sounds a little complacent, and unnecessarily focussed on just one area of psychology, given the extent of the problems since uncovered in the way research is manufactured (or maybe not, see below). Anyway, in the last week or so there have been three interesting developments

Despair

Michael Inzlicht blogged his ruminations on the state of the field of social psychology, and they’re not rosy : “We erred, and we erred badly“, he writes. It is a profound testament to the depth of the current concerns about the reliability of psychology when such a senior scientist begins to doubt the reality of some of the phenomenon upon which he has built his career investigating.

As someone who has been doing research for nearly twenty years, I now can’t help but wonder if the topics I chose to study are in fact real and robust. Have I been chasing puffs of smoke for all these years?

Don’t panic!

But not everyone is worried. A team of Harvard A-listers, including Timothy Wilson and Daniel Gilbert, have released press release announcing a commentary on the “Reproducibility Project: Psychology”. This was an attempt to estimate the reliability of a large sample of phenomena from the psychology literature (Short introduction in Nature here). The paper from this project was picked as one of the most important of 2015 by the journal Science.

There project is a huge effort, which is open to multiple interpretations. The Harvard team’s press release is headlined “No evidence of a replicability crisis in psychological science” and claimed “reproducibility of psychological science is indistinguishable from 100%”, as well as calling from the project to put effort into repairing the damage done to the reputation of psychological research. I’d link to the press release, but it looks like between me learning of it yesterday and coming to write about it today this material has been pulled from the internet. The commentary announced was due to be released on March the 4th, so we wait with baited breath for the good news about why we don’t need to worry about the reliability of psychology research. Come on boys, we need some good news.

…But whatever you do, optimally weight evidence

Speaking of the Reproducibility Project, Alexander Etz produced a great Bayesian reanalysis of the data from that project (possible because it is all open access, via the Open Science Framework). This take on the project is a great example of how open science allows people to more easily build on your results, as well as being a vital complement to the original report – not least because it stops you naively accepting any simple statistical report of the what the reproducibility project ‘means’ (e.g. “30% of studies do not replicate” etc). Etz and Joachim Vandekerckhove have now upgraded the analysis to a paper, which is available (open access, natch) in PLoS One : “A Bayesian Perspective on the Reproducibility Project: Psychology“. And their interpretation of the reliability of psychology, as informed by the reproducibility project?

Overall, 75% of studies gave qualitatively similar results in terms of the amount of evidence provided. However, the evidence was often weak …The majority of the studies (64%) did not provide strong evidence for either the null or the alternative hypothesis in either the original or the replication…We conclude that the apparent failure of the Reproducibility Project to replicate many target effects can be adequately explained by overestimation of effect sizes (or overestimation of evidence against the null hypothesis) due to small sample sizes and publication bias in the psychological literature

The death of the soul has been greatly exaggerated

I’ve got a piece in today’s Observer looking back on 20 years since novelist Tom Wolfe wrote a landmark article that threw open the doors on how the new science of cognitive neuroscience was challenging the notion of the self.

Exactly 20 years ago, Tom Wolfe wrote one of the most influential articles in neuroscience. Titled Sorry, But Your Soul Just Died, the 1996 article explores how ideas from brain science were beginning to transform our understanding of human nature and extend the horizons of our scientific imagination. It was published in a mainstream magazine, written by an outsider, and seemed to throw open the doors to an exhilarating revolution in science and self-understanding. Looking at the state of neuroscience and society two decades later, Wolfe turned out to be an insightful but uneven prophet to the brain’s future.

Wolfe’s article has been cited widely by both neuroscientists and the popular press.

It’s not entirely clear whether it shaped our popular understanding of brain science or whether it just predicted a future trend but it’s notable that before 1996 most press articles on fMRI were focused on technical details but subsequently they tended to be much more about ‘the brain reason for’ some aspect of human thought or behaviour.

Either way, it was clearly an important moment for neuroscience and my piece in The Observer looks back on Wolfe’s take on our changing view of human nature with the benefit of 20 years of brain science behind us.

And just to say, I occasionally have a bit of a grumble about the headlines written for my articles but hats off to however came up with “Neuroscience and the premature death of the soul” for this piece.

 

Link to ‘Neuroscience and the premature death of the soul’ in The Observer

Link to Tom Wolfe’s epic ‘Sorry, But Your Soul Just Died’.

Spike activity 12-02-2016

Quick links from the past week in mind and brain news:

Don’t tase me bro! Because it’ll cause short term cognitive impairment which may affect my ability to respond correctly under police interview. Important research from Drexel.

Mosaic has an interesting piece on hacking the placebo response and associative learning to improve medical treatments.

Your Next New Best Friend Might Be a Robot. Might be already for all I know. Nautilus on social robotics.

Science reports that sleep deprivation markedly increases false confessions.

The microcephaly brain changes apparently linked to the Zika virus are puzzling science. Good piece from NBC News.

The Atlantic covers the bitter fight over the benefits of bilingualism.

Good sceptical Gary Marcus talk on the current state of artificial intelligence and a useful tonic to those who think deep learning will lead to strong AI.

The Economist has an excellent in-depth article on the social effects of legalising cannabis.

There’s an excellent interview with pioneering neuroengineer Ed Boyden in Edge. Really, go read it.

Science News reports that the rise of human civilization was tied to belief in punitive gods. And also reality TV, you’ll notice.

A quartet of complementary brain books

Last night I taught a two hour class called ‘Navigating Neuroscience’ for the Guardian Masterclass series and I had the interesting challenge of coming up with a two hour course on some key concepts to help people make better sense of brain science, how it’s discussed, and its changing place in society.

As part of that, I recommended some books to give interested non-specialists a good critical introduction. I added a book after hearing some of the questions and I’ve included the list below.

I’ve mentioned some of them before on Mind Hacks in their own right, but I thought they’re worth mentioning as a set.

The books have been chosen to complement each other and the idea is that if you read all four, you should have a solid grounding in modern cognitive neuroscience and beyond. In no particular order:

Brainwashed: The Seductive Appeal of Mindless Neuroscience

by Sally Satel and Scott O. Lilienfeld

This is a great book for understanding common fallacies in conclusions drawn from cognitive neuroscience studies and what conclusions can reasonably be drawn from this evidence. It tackles several areas as examples of where these fallacies are having a significant effect: neuromarketing, neurolaw, lie detection, addiction and the brain-disease fallacy.

50 Ideas You Really Need to Know About the Human Brain

by Moheb Costandi

It’s a book of 50 small chapters each of which contains an essential idea on which the foundation of modern neuroscience rests. It’s very accessibly and accurately written and gets across some key subtleties that many academic textbooks miss. The great thing about this book is that it’s not just a ‘nuts and bolts’ guide to the brain and isn’t afraid to go into quite technical areas (‘Default Mode’, ‘Prediction Error’) while making sure they’re described in straight-forward language.

Great Myths of the Brain

by Christian Jarrett

This is especially good for listing and dispelling commonly cited but erroneous brain ‘facts’. It starts with some historical ones (‘Drilling a Hole in the Head Releases Evil Spirits’), move on to more obvious contemporary myths (‘We Only Use Ten Percent of Our Brains’) but then includes a range of common myths that may be well understood by neuroscientists but which pervade popular discourse and the media (‘Mirror Neurons Make Us Human’, ‘The Brain Receives Information from Five Senses’).

Neuro: The New Brain Sciences and the Management of the Mind

by Nikolas Rose and Joelle M. Abi-Rached

This is a great book for understanding how neuroscience is understood and used in society. It’s actually an academic book and Rose and Abi-Rached are sociologists but it’s technically accurate without being densely written. I genuinely think it’s one of the most important neuroscience books of the last decade. It is a brilliant analysis of how brain science and the practice of brain science have become associated with changing ideas of what it means to be human and their reciprocal relationship between politics and social influence in the world.

Spike activity 22-01-2016

Quick links from the past week in mind and brain news:

The New Yorker covers the shifting sands of autism in light of recent books that have rethought the history of the condition.

Brian Resnick at Vox asked twenty psych researchers: What do you hate about science journalism? Lots of good stuff.

Science reports big and welcome news: the Montreal Neurological Institute, one of the world’s leading brain research centres, is going entirely open science.

Why does the brain use so much energy? asks Wired UK.

The Independent has a piece on the history of the drug amyl nitrate, sold widely as ‘poppers’, and its place in gay culture, clubbing and sex.

I get interviewed by the Spanish-language blog Neuromexico – text in Spanish but audio largely in English.

The New York Times has a subtle first-person piece on prison psychotherapy.

A brief history of decapitation. Over at Inverse.

World’s stupidest drugs laws enacted by Britain

Yesterday, the UK Parliament approved the Psychoactive Drugs Bill which will become law in April. New Scientist pulls no punches in an uncharacteristically direct article and tells it like it is:

It’s official – the UK ban on legal highs that will begin in April is going to be one of the stupidest, most dangerous and unscientific pieces of drugs legislation ever conceived.

Watching MPs debate the Psychoactive Substances Bill yesterday, it was clear most of them hadn’t a clue. They misunderstood medical evidence, mispronounced drug names, and generally floundered as they debated the choices and lifestyles of people who are in most cases decades younger than themselves.

It would have been funny except the decisions made will harm people’s lives and liberty.

Parliament has just demonstrated you can invent nonsensical bullshit in place of science and get it passed as law as long as you claim it’s to ‘protect people’ from drugs.

Quite frankly, it’s an embarrassment.

 

Link to New Sci piece on “one of the worst laws ever passed”.

Spike activity 15-01-2016

Quick links from the past week in mind and brain news:

The New York Times has a brilliant piece on the non-scandal around sociologist Alice Goffman that’s also a reflection on sociology itself.

There’s a fascinating piece on ‘super forecasters‘ – people who seem to have an exceptional ability to judge the outcome of future events – in the Washington Post.

Knowing Neurons is a relatively new neuro blog that just keeps getting better.

Applying a knowledge of cognitive biases to add reality to virtual reality. Aeon covers an interesting area of applied psychology.

National Geographic has a fantastic piece on the evolution of the eye.

A mathematician is using computers to manufacture award-winning illusions. Fantastic piece in Nautilus.

Where Are We Now? – David Bowie and Psychosis

The mercurial David Bowie has left the capsule and the world is a poorer place. His circuit is dead, and there definitely is something wrong, at least for those of us still on Planet Earth.

There have been many tributes, noting Bowie’s impact on music, art and cinema, and the extent of his eclectic tastes. But one significant part of Bowie’s life has barely merited a mention – his experiences with psychosis – despite the fact that they had a major impact on his life and was featured in some of his most important work.

Bowie was familiar with psychosis from an early age, not least because it affected his close family. Two of his aunts were reportedly diagnosed with schizophrenia and third was confined to an asylum.

One of Bowie’s most influential early role models, his half-brother Terry, was diagnosed with schizophrenia and reportedly had marked periods of psychosis.

Here is Bowie, discussing one of his brother’s psychotic episodes, in a 1998 documentary for VH1:

Bowie’s brother was admitted to now defunct Cane Hill psychiatric hospital in South London and the experience heavily influenced 1970’s The Man Who Sold the World album with a drawing of the hospital appearing on the original sleeve art.

One of the songs on that album, All the Madmen, vividly describes madness and treatment in the old asylums, and was discussed in a 2010 article for the British Medical Journal:

“All the Madmen” was inspired by the mental health problems of David Bowie’s brother and was released 39 years ago (before Bowie achieved major fame), on the album The Man Who Sold the World. It recognises the separation from society of mentally ill people, who are sent to “mansions cold and grey.” In a lucid interval, spoken instead of sung, the national shame of mental illness and policies of alienation and institution are questioned with sadness: “Where can the horizon lie / When a nation hides / Its organic minds in a cellar.”

Faced with the prospect of discharge, the patient protagonist recognises his comfort in Librium, considers his ability to cope outside, and pushes the risk buttons with, “I can fly, I will scream, I will break my arm / I will do me harm.” He adopts a catatonic posture, standing with a foot in his hand, talking to the wall. He is accepting of electric shock treatment. When he asks, “I’m not quite right at all . . . am I?” is this a cryptic taunt that he knows he is putting it on, pushing the psychiatrist to keep his place in the institution? Or, more worryingly, is he questioning his own sanity and certainty?

Perhaps unsurprisingly, the themes of madness pervade Bowie’s work. The title track for the Aladdin Sane album (a play on “A lad insane”) was inspired by his brother, as was the song Jump They Say. Some other references are more obvious, such as in the song I’m Deranged, while some only allude to altered states and psychological alienation, as in The Man Who Sold the World.

Little known is that most famous character, Ziggy Stardust, was based on someone who experienced striking periods of psychosis. In a 1996 , Bowie recounted how Ziggy was based on the obscure rock star Vince Taylor who Bowie met several times, presumably between the periods Taylor spent in psychiatric hospital.

Bowie himself was widely thought to have experienced an episode of psychosis himself, some years later, largely due to a period when he was taking very large amounts of cocaine while working on the album Station to Station.

Several biographies describe how he feared evil entities floating past his window, thought The Rolling Stones were sending message to him through their music and believed witches were stealing his semen.

But the semantic traffic between madness and Bowie’s work was not solely one way. The medical literature has reports of Bowie featuring in the delusions of people with psychosis. One case report described a “32-year-old divorced white female with a long history of affective and behavioral problems”:

She believed she was secretly married to the rock star, David Bowie, after supposedly meeting in a church camp several years previously. She described seeing him “wait for her” outside her hospital window. The onset of this delusion coincided with a local tour by Bowie.

As Bowie was the master of looping cultural expression, making his art reference himself reacting to cultural responses to his work, it’s a return acknowledgement he may have appreciated.

Spike activity 08-01-2016

Quick links from the past week in mind and brain news:

The State of Texas now allows guns in state-run psychiatric hospitals, according to the Statesman. I am genuinely lost for words.

Sifting the Evidence has an excellent piece on the science behind the UK’s new lowered alcohol intake recommendations.

Scale Invariance: A Cautionary Tale Against Reductionism in Neuroscience. Thought-provoking piece from Knowing Neurons.

The New York Times has an excellent piece on the psychology of the con.

Human clinical trials planned for revolutionary neuroscience technique optogenetics, reports Scientific American.

The Atlantic has a wonderful piece on how we coordinate conversations between us with the most precise synchronisation.

There’s a good piece on the psychopharmacology of new psychoactive substances in Cerebrum.

The New York Times reports how psychologists have now been quietly but officially withdrawn from working with Guantanamo detainees.

We need more pieces like this on male mental health: Ex-editor of lad’s mags Loaded and GQ talks on “How therapy saved my life” in The Telegraph.

A radio station run by patients that broadcasts from inside an Argentinean psychiatric hospital. Al Jazeera with an excellent documentary.

Geek medal to this man: Neurosurgeon criticises latest Bond movie over anatomically inaccurate depiction of how to drill out the fusiform gyrus. Gizmodo has the story.

How to formulate a good resolution

We could spend all year living healthier, more productive lives, so why do we only decide to make the change at the start of the year? BBC Future’s psychologist Tom Stafford explains.

Many of us will start 2016 with resolutions – to get fit, learn a new skill, eat differently. If we really want to do these things, why did we wait until an arbitrary date which marks nothing more important than a timekeeping convention? The answer tells us something important about the psychology of motivation, and about what popular theories of self-control miss out.

What we want isn’t straightforward. At bedtime you might want to get up early and go for a run, but when your alarm goes off you find you actually want a lie-in. When exam day comes around you might want to be the kind of person who spent the afternoons studying, but on each of those afternoons you instead wanted to hang out with your friends.

You could see these contradictions as failures of our self-control: impulses for temporary pleasures manage to somehow override our longer-term interests. One fashionable theory of self-control, proposed by Roy Baumeister at Florida State University, is the ‘ego-depletion’ account. This theory states that self-control is like a muscle. This means you can exhaust it in the short-term – meaning that every temptation you resist makes it more likely that you’ll yield to the next temptation, even if it is a temptation to do something entirely different.

Some lab experiments appear to support this limited resource model of willpower. People who had to resist the temptation to eat chocolates were subsequently less successful at solving difficult puzzles which required the willpower to muster up enough concentration to complete them, for instance. Studies of court records, meanwhile, found that the more decisions a parole board judge makes without a meal break, the less lenient they become. Perhaps at the end of a long morning, the self-control necessary for a more deliberated judgement has sapped away, causing them to rely on a harsher “keep them locked up” policy.

A corollary of the ‘like a muscle’ theory is that in the long term, you can strengthen your willpower with practice. So, for example, Baumeister found that people who were assigned two weeks of trying to keep their back straight whenever possible showed improved willpower when asked back into the lab.

Yet the ‘ego-depletion’ theory has critics. My issue with it is that it reduces our willpower to something akin to oil in a tank. Not only does this seem too simplistic, but it sidesteps the core problem of self-control: who or what is controlling who or what? Why is it even the case that we can want both to yield to a temptation, and want to resist it at the same time?

Also, and more importantly, that theory also doesn’t give an explanation why we wait for New Year’s Day to begin exerting our self-control. If your willpower is a muscle, you should start building it up as soon as possible, rather than wait for an arbitrary date.

A battle of wills

Another explanation may answer these questions, although it isn’t as fashionable as ego-depletion. George Ainslie’s book ‘Breakdown of Will‘ puts forward a theory of the self and self-control which uses game theory to explain why we have trouble with our impulses, and why our attempts to control them take the form they do.

Ainslie’s account begins with the idea that we have, within us, a myriad of competing impulses, which exist on different time-scales: the you that wants to stay in bed five more minutes, the you that wants to start the day with a run, the you that wants to be fit for the half-marathon in April. Importantly, the relative power of these impulses changes as they get nearer in time: the early start wins against the lie-in the day before, but it is a different matter at 5am. Ainslie has a detailed account of why this is, and it has some important implications for our self-control.

According to this theory, our preferences are unstable and inconsistent, the product of a war between our competing impulses, good and bad, short and long-term. A New Year’s resolution could therefore be seen as an alliance between these competing motivations, and like any alliance, it can easily fall apart. Addictions are a good example, because the long-term goal (“not to be an alcoholic”) requires the coordination of many small goals (“not to have a drink at 4pm;” “not at 5pm;” “not at 6pm,” and so on), none of which is essential. You can have a drink at 4pm and still be a moderate drinker. You can even have a drink also at 5pm, but somewhere along the line all these small choices add up to a failure to keep to the wider goal. Similarly, if you want to get fit in 2016, you don’t have to go for a jog on 1 January, or even on 2 January, but if you don’t start doing exercise on one particular day then you will never meet your larger goal.

From Ainslie’s perspective willpower is a bargaining game played by the forces within ourselves, and like any conflict of interest, if the boundary between acceptable and unacceptable isn’t clearly defined then small infractions can quickly escalate. For this reason, Ainslie says, resolutions cluster around ‘clean lines’, sharp distinctions around which no quibble is brooked. The line between moderate and problem drinking isn’t clear (and liable to be even less clear around your fourth glass), but the line between teetotal and drinker is crystal.

This is why advice on good habits is often of the form “Do X every day”, and why diets tend to absolutes: “No gluten;” “No dessert;” “Fasting on Tuesdays and Thursdays”. We know that if we leave the interpretation open to doubt, although our intentions are good, we’ll undermine our resolutions when we’re under the influence of our more immediate impulses.

And, so, Ainslie gives us an answer to why our resolutions start on 1 January. The date is completely arbitrary, but it provides a clean line between our old and new selves.

The practical upshot of the theory is that if you make a resolution, you should formulate it so that at every point in time it is absolutely clear whether you are sticking to it or not. The clear lines are arbitrary, but they help the truce between our competing interests hold.

Good luck for your 2016 resolutions!