Goodreads helps you keep track of books you want to read.

Start by marking “Science Fictions: The Epidemic of Fraud, Bias, Negligence and Hype in Science” as Want to Read:

Other editions

Enlarge cover

Rate this book

Clear rating

1 of 5 stars 2 of 5 stars 3 of 5 stars 4 of 5 stars 5 of 5 stars

Open Preview

Science Fictions: The Epidemic of Fraud, Bias, Negligence and Hype in Science

Stuart Ritchie

4.47 · Rating details · 565 ratings · 95 reviews

A major exposé that reveals the absurd and shocking problems that pervade and undermine contemporary science.

So much relies on science. But what if science itself can’t be relied on?

Medicine, education, psychology, health, parenting – wherever it really matters, we look to science for advice. Science Fictions reveals the disturbing flaws that undermine our understanding of ...more

Get A Copy

Paperback, 368 pages

Published July 21st 2020 by Bodley Head

More Details... Edit Details

Friend Reviews

To see what your friends thought of this book, please sign up.

Reader Q&A

To ask other readers questions about Science Fictions, please sign up.

Lists with This Book

Too Much and Never Enough by Mary L. Trump

The Splendid and the Vile by Erik Larson

Non-Fiction Published in Year: 2020

686 books — 118 voters

The Frontlines of Peace by Severine Autesserre

The Blindspots Between Us by Gleb Tsipursky

Speaking for Myself by Sarah Huckabee Sanders

Non-Fiction Published in Decade: 2020s

418 books — 56 voters

More lists with this book...

Community Reviews

Showing 1-30

Average rating 4.47 ·

Rating details

· 565 ratings · 95 reviews

More filters

Sort order

Start your review of Science Fictions: The Epidemic of Fraud, Bias, Negligence and Hype in Science

Jul 25, 2020 Gavin rated it really liked it

Shelves: very-useful, pop-sci

Wonderful introduction to meta-science. I've been obsessively tracking bad science since I was a teen, and I still learned loads of new examples. (Remember that time NASA falsely declared the discovery of an unprecedented lifeform? Remember that time the best university in Sweden completely cleared their murderously fraudulent surgeon?)

Science has gotten a bit fucked up. But at least we know about it, and at least it's the one institution that has a means and a track record of unfucking itself.

R ...more

flag 17 likes · Like · see review

Sep 24, 2020 Andy rated it liked it · review of another edition

This is an important topic, and the author does an excellent job explaining problems like p-hacking. But these issues are nothing new to scientists, so the main value of this book is if it engages and clearly explains things for the general public. And there, I’m afraid the author may end up just increasing confusion by trying to turn everyone into a scientist. In terms of solutions to bad science, I wonder if we don’t need to start by addressing the underlying culture of corruption and incompet ...more

flag 14 likes · Like · see review

Jul 27, 2020 Julia rated it it was amazing

This is one of the most important books I’ve read in the past few years. Ritchie skillfully examines the problems plaguing modern science, looks at the motivations that cause them, and posits solutions. Science Fictions drives home the importance for skepticism in all things, even science.

flag 13 likes · Like · see review

Jul 24, 2020 Alvaro de Menard rated it really liked it · review of another edition

In 1945, Robert Merton wrote:

There is only this to be said: the sociology of knowledge is fast outgrowing a prior tendency to confuse provisional hypothesis with unimpeachable dogma; the plenitude of speculative insights which marked its early stages are now being subjected to increasingly rigorous test.

Then, 16 years later:

After enjoying more than two generations of scholarly interest, the sociology of knowledge remains largely a subject for meditation rather than a field of sustained and metho

...more

flag 10 likes · Like · see review

Jan 23, 2021 J.J. rated it really liked it · review of another edition

First of all the title slaps, this is the kind of word play you want in a popular science book title.

Ritchie grabs your attention with some spicy cases of scientific fraud, but follows up with other pernicious problems that lead science astray. He goes on to suggest changes to the way research is conducted, funded, reviewed and published to right some of these wrongs. A worthwhile read (or listen) for researchers or mere muggles like myself.

flag 6 likes · Like · see review

Jan 06, 2021 muthuvel rated it really liked it

Shelves: psychology, politics

Nobel Laureate Economist Daniel Kahneman, in his work targeted to public audience 'Thinking, Fast and Slow (2011)' talks about the certainty of Priming effects through citing various psychological studies and thereby claimed certain stimulus can be produced without conscious guidance or intention and that which can be patterned. It was one of the widely read popular bestsellers in the genre but things of uncertainty were likely after a few years when the studies he cited were failed to replicate Nobel Laureate Economist Daniel Kahneman, in his work targeted to public audience 'Thinking, Fast and Slow (2011)' talks about the certainty of Priming effects through citing various psychological studies and thereby claimed certain stimulus can be produced without conscious guidance or intention and that which can be patterned. It was one of the widely read popular bestsellers in the genre but things of uncertainty were likely after a few years when the studies he cited were failed to replicate or published with inadequate data. He even acknowledged that the fact that he was wrong about his certainty. What happened here?

"The books we’ve just discussed were by professors at Stanford, Yale and Berkeley, respectively. If even top-ranking scientists don’t mind exaggerating the evidence, why should anyone else?"

Following Kahneman, we have similar claimed by NASA, pop science books like Why We Sleep, studies of austerity, mediterranean diets, publication biases and issues of hacking p-values, cherry picking, salami slicing, self citations, self plagiarism, creating ghost citations and review from ghost peers, coercive citations from accepting journals.

Most of the people who's already in the field would know most of the replication crises discussed in the book but I guess mostly their guides would have calmed them down that it's okay to not being able to replicate scientific study due human error among other factors. It's a conditioning that's been practiced contradictory of the objectives set by the founding figures of scientific publishing community like Boyle.

Afterall the practitioners of science in the end has the susceptibility of becoming more of an organized cult working for their incentives of various kinds from academic survival, personal fame and status to achieve the bureaucratic standards forgetting the basic tenets of what scientific research is all about.

The last book I read was a work of a Wittgenstein student showcasing how Social Science was massacred by Social Scientists (Sociologists, Social Psychologists, Economists, Political Scientists to name a few..) where as this one does the same in the Natural Sciences.

But rather not going philosophical, it's limited to how science is practiced today than what science actually is. Maybe there are no better methods to understanding the world but as Winch said it's better to stay vigil and question 'the extra scientific pretensions' of scientific communities which creates its own norms and beliefs in its culture of practicing Science.

Science Fictions: The Epidemic of Fraud, Bias, Negligence and Hype in Science (2020) ~ Stuart Ritchie ...more

flag 4 likes · Like · see review

Oct 18, 2020 Sophia rated it really liked it

Shelves: pop-science, academic-stuff

I highly recommend this book for anyone planning (or considering) to do science, either a bachelors, masters or more. It's a great overview of how science is actually practiced, and how it can so easily go wrong. I also recommend this to current scientists, because it's a humbling reminder of what we're doing wrong, and also a quick update on things we might have been taught as facts has actually been disproven in the meantime.
The book is exceptionally well structured, very clear writing, very I highly recommend this book for anyone planning (or considering) to do science, either a bachelors, masters or more. It's a great overview of how science is actually practiced, and how it can so easily go wrong. I also recommend this to current scientists, because it's a humbling reminder of what we're doing wrong, and also a quick update on things we might have been taught as facts has actually been disproven in the meantime.
The book is exceptionally well structured, very clear writing, very engaging, switching between as much information as needed to understand a given concept, then compelling examples, and discussion as to why it matters, what people might object to, etc. Really really good.
However, the author fails to give proper due to the main strength of science: it's ability to self-correct. This book is described as an "expose", but in reality all of what he mentions has been known for decades, and in fact every single example he gives of fraud, negligence, bias, or unwarranted hype was not uncovered by external journalists but rather other scientists. It was the peers who read papers that looked suspicious and did more digging, or whole careers built around developing software and tools for automatically detecting plagiarism, statistical errors, etc. It was psychology itself that "wrote the book" on bias that was fundamental to exposing the biases of scientists themselves. And more often than not, it was just a future study that tried something better that should have worked but didn't that disproved a flimsy hypothesis. Sure; fraud, hype, bias, and negligence are dragging science down, but science isn't "broken", it's just inefficient. Wasting a lot of money on bad experiments and scientists needs to be avoided, but in the end, a better truth tends to bubble up regardless. Anyone who has had to defend science against religious diehards will be particularly aware of this.

Also missing is proper consideration as to why these seemingly blindingly obvious problems have been going on for so long. As an insider, here are some of my answers:
- All this p-hacking (trying different analyses until something is significant). Scientists are not neatly divided into those that immediately find their results because of how fantastically well they planned their study, and those that desperately try to make their rotten data significant. Every. Single. Study. has to fine tune their analysis once they get the data, not before. Unless you are in fact replicating something, you have absolutely no idea what the data will look like, and what's the most meaningful way to look at it. This means you can’t just tell scientists "stop p-hacking!", you need an approach that acknowledges this critical step. Fortunately, an honest one exists that can be borrowed from machine learning: splitting your data into a "training" and "testing" dataset, where you fine-tune your analysis pipeline on a small subset, then you actually rely on the results applied to a larger one, using only and exactly the pipeline you previously developed, without further tweaking.
- The file drawer problem (null results not getting published). I think especially in the field of psychology, statistics courses are to blame for this; we don't reeeally understand how the stats work, so we rely on Important Things To Remember that we're taught by statisticians, and one of these is that "you can't prove a null hypothesis". This ends up getting interpreted in practice in "null results are not real results, because nothing was proven". We are actively discouraged from interpreting "absence of evidence as evidence of absence", but sometimes that is in fact exactly what we should be doing; for sure not with the same confidence and in the same way with which we interpret statistically significant positive results, but at some point, a study that should have found something but didn't is a meaningful indication that that thing might not in fact be there. A useful tool to help break through this narrow-minded focus on only positive results is a new statistical tool called similarity testing, where you test not whether two groups are different but whether they are statistically significantly "the same". This is a huge shift in mindset for many psychologists, who suddenly learn that you can in fact have a legitimate result that there was no difference to be found. Knowing this I suspect will make people less wary of null results in general.
- Proper randomization (and generally the practicalities of data collection). The author at some point calls it a mistake that a trial on the Mediterranean Diet had assigned to the same family unit the same diet, thus breaking the randomization. For the love of God, does he not know how families work? You cannot honestly ask members of the same family to eat differently! Sure, the authors should have implemented proper statistical corrections for this, but sometimes you have to design experiments for reality, not a spherical world.
- Reviewers nudging authors to cite them. This may seem like a form of blatant self-promotion, but it's worth mentioning that in reality, the peer reviewers were specifically selected as members of the EXACT SAME FIELD, and so odds are good that they have in fact published relevant work, and odds are even better that they are familiar with it enough to recommend it. That is not to say that some of it is for racking up citations, but this is not true unless proven otherwise, because legitimate alternative explanations exist.

Other little detail not mentioned by the author is that good science is f*cking hard. For my current experiment, I need a passing understanding of electrical engineering to run the recording equipment, a basic understanding of signal processing and matrix mathematics to clean and analyze the data, a good understanding of psychology for experimental design, a deep understanding of neuroscience for the actual field I'm experimenting in, a solid grasp of statistics, sufficient English writing skills, separate coding skills for both experimental tasks and data analysis in two different languages, and suddenly a passing understanding of hospital-grade hygiene practices to deal with COVID! There's just SO MUCH that can go wrong, and a failure at any point is going to ruin everything else. It's exhausting to juggle all that, and honestly, it's amazing that we have any valid results coming out at all. The only real solution to this is to have larger teams; focus less on individual achievements. The more eyes you have on scripts, the fewer bugs there will be; the more assistants available to collect data, the fewer mishaps. The more people reading the paper beforehand, the fewer mistakes slip through. We need publications from labs, not author lists; it can be specified somewhere the exact contribution of each, but science needs to move away from this model of venerating the individual, because this is not the 19th century anymore: the best science comes from groups. On CVs, we shouldn’t write lists of publications, we should write project descriptions (and cite the paper as “further reading”, not as an end in and of itself).
~~~
Scientists need the wakeup call from this book. Journalists and interested laymen also greatly benefit from understanding why a healthy dose of scepticism is needed towards any single scientific result, and how scientists are humans too. But the take-home message that can transpire from this book and is not actually true, is that scientists are either incompetent or dishonest or both. The author repeatedly bashes poor science and science communication that has eroded public trust in science, but ironically this book is essentially highlighting this with neon letters and making sure trust in science is eroded. To some extent it is warranted, but the author could have done more to defend the institution where it is deserved, and as an insider, could have done more to talk about the realities an individual scientist faces when they make these poor decisions. It's worth mentioning that science has not gotten worse, we're still making discoveries, still disproving our colleagues, and still improving quality of life. We could just be doing it more efficiently. ...more

flag 3 likes · Like · see review

Dec 05, 2020 Agne rated it it was amazing

Shelves: favorites

Extremely informative and well argued. I would suggest it to anyone who has any contact with science in their daily life (so... everyone). I loved the examples and statistics and that it's at the same time really approachable. For the layperson, it's pretty shocking to hear how null results and replication studies have been treated by even reputable journals.

There are a bunch of solutions offered at the end.

The only downside is that if an aspiring scientist were to read this book, they might t ...more

flag 3 likes · Like · see review

Jan 11, 2021 Pete rated it it was amazing

Shelves: nonfiction, science

Science Fictions : The Epidemic of Fraud, Bias, Negligence and Hype in Science (2020) by Stuart Ritchie is an excellent book that looks at the many problems in science and what can be done to improve the situation. Ritchie is a Psychologist at King’s College London.

Science Fictions goes through how science currently works and then details the replication crisis, where the replication of studies, particularly in psychology but also in other fields demonstrated serious problems with science as it ...more

flag 3 likes · Like · see review

Aug 14, 2020 Steve rated it really liked it

Really enjoyed this description of some of the big problems in science today. This is not in any way an anti-science book, Ritchie makes clear that he wants to improve science, not to dispense with it. Along with describing problems he also describes much of the process of science which I enjoyed.

He spends a lot of time on the reproducibility crises, p-hacking and other statistical cheating, and many other issues that one hears about when science problems get in the news.

This book has been wel ...more

flag 3 likes · Like · see review

Jul 21, 2020 Chris Boutté rated it it was amazing · review of another edition

Incredible book that I binged in a day. As an influencer who often references psychological studies but also knows how much bad science is out there, I’m always trying to learn more about this subject.

This author did a great job not just giving examples of bad science, but he explains WHY it’s happening and offers solutions. Absolutely loved this book and hope some journalists read it as well before they keep reporting on hyped up science.

flag 3 likes · Like · see review

Oct 30, 2020 Cam rated it really liked it

Shelves: non-fiction, psychology

Essential reading for graduate science researchers. Although, much of the material will hopefully be familiar to them.

Ritchie writes clearly. He's likeable and scientifically and statistically literate, but doesn't take himself too seriously. He's a great science populariser even when he is denigrating science!

Ritchie helped kick off the well-publicised replication crisis in social science in 2012 when he attempted and failed to replicate a para-psychology paper. The original paper by Bem purpo ...more

flag 2 likes · Like · see review

Dec 17, 2020 Fin Moorhouse rated it really liked it

Shelves: 2020

As other reviewers have said: sober, balanced, hopeful cataloguing of perverse incentives in present-day science, plus some potential fixes. A highlight was learning about this actual paper, accepted for publication in the 'International Journal of Advanced Computer Technology'.

What stood out for me was how superbly well-written this was. It's friendly and funny and crystal clear. Infuriating for anyone who's tried and failed hard to find that voice in their own writing! ...more

flag 2 likes · Like · see review

Aug 08, 2020 Tristan Eagling rated it it was amazing

Goodharts's law states "When a measure becomes a target, it ceases to be a good measure", which sums up the premise of the book perfectly. For centuries, science has tried to give value to subjective knowledge and academia relies on these often arbitrary metrics. But all we have done is create a system which can be gamed, and populated that system with clever (mostly) people who are heavily incentivized to game the system.

As someone who has published scientific research, peer reviewed others and ...more

flag 2 likes · Like · see review

Nov 30, 2020 Anthony D'Ambrosio rated it it was amazing

Pleasantly surprised by how often the phrase "salami slicing" appears ...more

flag 2 likes · Like · see review

Aug 29, 2020 MIKE Watkins Jr. rated it really liked it · review of another edition

This book reminds me of another book I read, the color of law. Like the Color of Law, this is a book that is very informative and will transform the way you think about the topic discussed in this book, but it lacks a few key ingredients that prevent it from being a 5/5 read.

This book starts out by introducing you to how science works and how that science is publicized.

1. A scientist comes up with a scientific theory/question to look into and comes up with an hypothesis.

2. The scientist attempts ...more

flag 1 like · Like · see review

Oct 13, 2020 Mbogo J rated it it was amazing

Science is the new god, a deity that has gotten its power too fast and too much, it has behaved the same way a human would;it became power drunk. The hostile takeover it did on religion as the preeminent field of knowledge that guides human affairs left it feeling smug and thought itself immune to the same pitfalls that had befallen religion. The end result? The current state of an epidemic of fraud, negligence, bias and hype.

Stuart Ritchie took us across the current landscape of science as prac ...more

flag 1 like · Like · see review

Oct 18, 2020 Kyle rated it it was amazing · review of another edition

This was a good read, in the sense that it clarified the problems that science faces (at least, for me). Ritchie does a good job of bringing in a lot of material to tell the story of the troubling findings of recent years. Ritchie focuses mostly on psychology, his area of expertise, and so while I found his thoughts very interesting for psychological and medical research, I did wonder about how some of it might translate to mathematics or physics. Reproducibility does not appear to be as much of ...more

flag 1 like · Like · see review

Sep 16, 2020 William Schram rated it it was amazing

Shelves: science, criticism

Science is an ancient and vaunted establishment. It has done so much good for the world that it is sometimes easy to forget that science is a human construct. Scientists are human beings and are prone to making mistakes. This issue already surfaced for me in nutritional science. It seemed that every week a new food was discovered to cause cancer or to extend your lifespan. It sickens me to no end.

Science Fictions is by Stuart Ritchie. It discusses the various issues that scientists have to deal ...more

flag 1 like · Like · see review

Sep 02, 2020 Mizuki rated it it was amazing · review of another edition

This book is not usual 'science fiction', do not expect that. It is about scientific fraud, full of well-known cases. I'm quite familiar with these cases as a scientist, and even as a reader of many scientific literatures.
Most of the points that the author says are painfully true - I know that is true, especially about the publication bias. Most scientists just want to publish their "best" data even if it is not reproducible well enough.
On the other hand, I somewhat want to believe that most sci ...more

flag 1 like · Like · see review

Dec 26, 2020 Ryan rated it it was amazing · review of another edition

This book covers the "replication crisis" in science and some of the causes of it -- not merely "how bad science is done" but also the motivations and "why". There has been some progress in the past few years in fighting this, but there are a lot of motivations for why people will continue to do things which are bad or counterproductive (including unintentionally). I'm somewhat optimistic science will ultimately figure out better was to communicate and value research which will help solve the pr ...more

flag 1 like · Like · see review

Aug 12, 2020 Ulrich Schroeders rated it it was amazing

Shelves: psych, science

This book is an important contribution to the so-called replication crisis in science. Ritchie neatly and clearly sums up the current state of affairs and unmasks the crisis as a social phenomenon that is strongly driven by communicative processes. The bigger picture across different scientific disciplines was very instructive. I also found a lot of new and useful references. I definitely recommend this book to every (psychology) student and researcher.

flag 1 like · Like · see review

Jan 17, 2021 Morgan rated it really liked it · review of another edition

Shelves: non-fiction, per-and-prof-dev

Presents critical issues in science in a layperson-friendly way and provides some interesting ideas for solutions — and is humble enough to recognize the possible problems with those proposed solutions. My only gripe is that the book felt somewhat repetitive at times and could have been a little shorter, but overall this is a good read, especially for someone newer to the issues it discusses.

flag 1 like · Like · see review

Oct 31, 2020 Ryan rated it it was amazing · review of another edition

Science Fictions is an important book, which is aimed primarily at a non-scientific audience. It builds upon a number of other books about scientific practices, and covers some of the same ground as for work by Ben Goldacre [1, 2] who the author cites. The author is a lecturer in Psychology, and most of the examples are drawn from this area, medicine, and social sciences in general - you can expect to read some discussion about for e.g. the Andrew Wakefield scandal, Cochrane reviews, pre-study r ...more

flag Like · see review

Oct 21, 2020 Scott Lupo rated it it was ok

Shelves: non-fiction

AVOID. Here are my reasons:
-From the beginning, this author lost my trust. In the preface, the author mentions how him and some colleagues wrote a null paper on the psychic experiments by Daryl Bem and was "unceremoniously rejected from the journal that published the original." The leaves the reader thinking that he never got that study published and moves on to the next subject. WRONG. Read the notes and you will find it did get published, just not to his liking.

-Read the notes. It's another w AVOID. Here are my reasons:
-From the beginning, this author lost my trust. In the preface, the author mentions how him and some colleagues wrote a null paper on the psychic experiments by Daryl Bem and was "unceremoniously rejected from the journal that published the original." The leaves the reader thinking that he never got that study published and moves on to the next subject. WRONG. Read the notes and you will find it did get published, just not to his liking.

-Read the notes. It's another whole book back there with some of them paragraphs long. Many of them either refute what he originally said or alters the original meaning just enough to realize he's trying to pull something. It would be interesting to know how many people actually read citations or notes at the end of books (couldn't find anything on google). I would venture to say not many, which I think he purposefully relied on for his narrative.

-Do you enjoy abusive relationships? Me neither. However, that is what this book is like. "I come to praise science, not to bury it; this book is anything but an attack on science itself, or its methods." The next paragraph then explains the only "fragile scrap of hope and reassurance that emerges from the Pandora's box of fraud, bias, negligence and hype" is that scientists have uncovered these things themselves. Throughout the book it's 'science is great but science also sucks really, really bad because...'.

-He has a hard on for Daniel Kahneman. He really doesn't like him.

-He conflates social sciences with ALL SCIENCE. Yes, social sciences are muddy and gray because it deals with human beings, who are muddy and gray in just about everything they do. Creating experiments is very difficult and interpreting results even more difficult. But to lump all of science into this category is foolish and leans towards trickery. Throughout the book he switches, sometimes within the same paragraph, from a social science to other sciences.

-Fraudsters, charlatans, flimflammers, and hustlers all use certain phrases in their toolkit of shams. Some of those are things like "you know what I mean", "let's face it", "it should be noted", "that being said". These are the phrases of all those psychics we used to see on tv (John Edward, Miss Cleo, Sylvia Brown). These phrases purposefully leave the door open for interpretation and let the listener/reader fill in the blanks themselves. Yeah, that is fraudster 101 class right there.

-He constantly brings up the oldest cases of science fraud and then tries to compare them to today's frauds. Every case he brings to light ends in one way: they were caught! Because that is what science does. Incredible claims require incredible evidence. He acts like it is the worst thing in the world that science actually caught these things. Apparently, it is never fast enough for the author or he thinks science should be absolutely free of any errors, full stop. I am unsure whether he truly understands the scientific process.

-His conclusions on how to fix these things is paltry at best. In fact, many of his suggestions are already in use today! Others he admits would be impossible to do.

My only conclusion to this book is that it is a thinly veiled hit piece on science. Every fraudster knows that if you include nuggets of truth in your parable, then it will seem like everything is truthful. That is exactly this book. He even talks about this in the book. That scientists have gotten so good at faking their results so that it doesn't look 'perfect' and people will buy it. The irony!! The author grandiosely overstates his hypothesis that there is an epidemic of fraud, bias, negligence, and hype in science. Many times I thought I was reading an Onion article made into a book because he uses all those things in this book.

Okay, I have laid out my reasons but I also want to give credit when it is due. Science is not perfect and the process is not always efficient and it does not always incentivise the proper way. Welcome to the problem with scaling up and money. Sure, it would be great to have science run without any thought to money or resources. Science for the sake of science. Cool with me. Let's shoot for that and do what we can to get as close as we can to that ideal. But this is not the message in this book. I truly believe this author has problems with two things: social sciences and the philosophy of science (epistemology). He should consider writing on those subjects instead of attacking the whole of science. Especially in a dishonest way like this book. It gets a star for actually writing a book and a star for at least shedding light on some of the issues with scientific research sometimes. That's two stars. The same I gave to Michelle Malkin. Enough said.

...more

flag 2 likes · Like · see review

Jul 23, 2020 Liam Crismani rated it it was amazing · review of another edition

Brilliant. Comprehensive but easy to understand exploration of the problems in the ways we currently do science.

flag 1 like · Like · see review

Feb 17, 2021 Jesse Field rated it really liked it

This book certainly reminds me of the work of Nassin Nicholas Taleb as injunctions toward skepticism. Stuart Ritchie is sort of the Taleb of science. Both even make mention of Daniel Kahneman’s work, though Ritchie uses it as illustration of problems in science (‘priming’ effects were found not to be defended as well as Kahnman found in his original work) where Taleb makes extensive use of the survivorship bias concept. Ritchie is a better writer than Taleb, I find, giving us clear accounts and This book certainly reminds me of the work of Nassin Nicholas Taleb as injunctions toward skepticism. Stuart Ritchie is sort of the Taleb of science. Both even make mention of Daniel Kahneman’s work, though Ritchie uses it as illustration of problems in science (‘priming’ effects were found not to be defended as well as Kahnman found in his original work) where Taleb makes extensive use of the survivorship bias concept. Ritchie is a better writer than Taleb, I find, giving us clear accounts and arguments where Taleb relies too much on fast-and-loose anecdote and analogy. While it’s illuminating to see the problems faced in contemporary science, I suppose it's not that surprising, given the drive toward profit and prestige of the professional fields. And the situation is better in science than in finance. (And just think of popular social and political discourse! If knowledge production is fouled up, no wonder the hoi polloi are even more confused. I wonder if anyone out there is bringing all of this together as one meta-epistemic crisis.)

Anyway, this book is divided by sections into expositions on the problems of fraud, bias, negligence and hype, all of which contain cases outstanding and horrid enough to make the reader suspect that Ritchie could be guilty of hyping himself. I was not aware that Philip Zimbardo's Stanford Prison Experiment has been thoroughly discredited, which just goes to show that retracted science often lives on, in zombie form. But he does anticipate this response and qualifies his narrative with assurances that he is only concentrating on the most salient examples, and he covers measures currently being undertaken to combat fraud, bias, negligence and hype, so I don’t think he is over-hyping his case here. The social nature of science led us to the replication crisis of the last decade, affecting all fields of science and decimating some, like psychology. (A chart of fields affected by inability to replicate results would have been nice here. Ritchie is potentially guilty of overgeneralizing.)

Fraud is what seems to have drawn the most attention in the book so far, and it does occupy a front-and-center position. Science is a social construct, Ritchie begins, with many players eager to impress each other with salient and positive results of clear originality, so it’s natural that some voices would use false data to gain positive attention, even as the larger community of thinkers has been collegial and trusting, giving fraudsters an environment in which to thrive. I really don’t think there is any disputing this reasoning, and more stringent skepticism toward results will have to evolve along with reforms to paper publishing, as Ritchie argues in the final portions of the book. Websites like Retraction watch profile dramatic, even hair-raising cases, but also serve as evidence that the community of knowledge production is taking steps to address the problem. (One wonders what steps meta-analysts in China might be taking. And it’s important to note that the argument that political conditions in China encourage fraud is a conjecture without data of any kind to support it, though it sort of makes sense.)

As Ritchie says, bias is even more disturbing than fraud, because the former is not based on malevolent intent, is clearly more widespread, and will be much tougher to fix. Publication bias against null results masks hypotheses proven wrong from view, which means they might be tried out again, and again. (I wonder if there is solid evidence that that happens. I notice that some critics still affirm that it’s better not to publish negative results.) “P-hacking” is a form of confirmation bias that has scientists of all stripes massaging their data to strengthen positive results, with tremendous implications for anyone looking to use or build on such work — refer here studies of the effect of priming, as well as of certain medical treatments. String theory is susceptible to a similar form of confirmation hypothesis, according to the book Lost in Math: How Beauty Leads Physics Astray. (Again, one would like to see a chart of the hypothetical prevalance of such bias across fields.)

Negligence was a particularly shocking chapter. First there is the fact that the data sets of many labs full of errors as revealed by GRIM testing, which is bad enough, but the more dramatic episode of negligence involves ‘cell lines,’ maintained living tissue samples used in medical research. These are often simply mislabeled, with the result that work on bone cancer might accidentally use a lung cancer line, or else the very species the tissue was taken from is mistaken. This is such a common problem, estimates are that it affects tens of thousands of papers worldwide, and maybe as many as 46 percent of cell lines in China. Science is obstructed by simple book-keeping mistakes! The level of negligence here is mind-boggling, and really invites deeper inquiry, I think. And the chapter ends with a consideration of ‘low powered’ research, meaning sample sizes are too small. The basic logic is that graduate students and lower-level scholars need to get research published, and so must do smaller studies, but the results might include ‘mirage’ effects such as those observed in the case of candidate gene studies, which hoped to link traits to specific genes, an approach thought promising until larger-scale studies showed effects in smaller studies were imaginary. (Interesting that Ronald Fisher theorized the polygenic nature of traits in 1918; that’s a case where scientists should have known the history of their own fields.)

This last type of negligence brushes up against hype in science. Science must report and draw attention to its results, so inevitably some will take up language that makes the findings dramatic and important. Results in nutrition and psychology that feed into the self-help industry seem particularly vulnerable, as Ritchie reports on these discredited books: Mindset: The New Psychology of Success], Before You Know It: The Unconscious Reasons We Do What We Do, and Why We Sleep: Unlocking the Power of Sleep and Dreams, all of which hype data that is of too small scale or even incorrect and retracted. NASA has gotten in trouble, of course, and one of the biggest data sets from a study of the effects of the Mediterranean diet deserves skepticism even after corrections thanks fact-checking by John Carlisle, a data hound who emerges as a kind of Lone Ranger against fraud, bias, negligence and hype in this book.

Ritchie’s proposed solutions to upgrade science will not surprise readers of behavioral economics cases: it’s about setting up defaults and incentives so that quality is rewarded, where too often today quantity is rewarded. To improve data quality, more diverse statistical analysis is called for, but just which and how to implement them is a more difficult question. The distinction between p-values and Bayesian statistics, for example, was revealing but subtle to me — apparently p-values are “almost always calculated independently of any prior evidence.” In that case, how are p-values calculated? Looking over the book, it’s not surprising that Ritchie skims over the question, referring us to The Art of Statistics: How to Learn from Data and Understanding Psychology as a Science: An Introduction to Scientific and Statistical Inference.

All in all, this is a fantastic addition to what I think we may call a new literature off critical thinking. Precisely because the world is so full of thorny problems, we need to step up our game when it comes to establishing good data and sound analysis based on un-hyped reasoning. ...more

flag Like · see review

Aug 11, 2020 Dylan O'Connell rated it it was amazing

Almost certain to be the book I annoyingly recommend to everyone I meet for the next few months.

“Science Fictions” is a vitally needed introductory text to the current crisis (crises?) in science, and I wish it was getting more “buzz” (podcast interview circuit, take note). Most readers will be individual with different individual parts–the reproducibility crisis in psychology, the dangers of hype in scientific journalism, or examples of egregious image-manipulation-fraud in prominent papers. Bu Almost certain to be the book I annoyingly recommend to everyone I meet for the next few months.

“Science Fictions” is a vitally needed introductory text to the current crisis (crises?) in science, and I wish it was getting more “buzz” (podcast interview circuit, take note). Most readers will be individual with different individual parts–the reproducibility crisis in psychology, the dangers of hype in scientific journalism, or examples of egregious image-manipulation-fraud in prominent papers. But this is the first work I’ve found that properly synthesizes all these disjoint strands (it’s rather tedious to try and point folks to a bunch of different blogs to get up to date).

To be crystal clear, this is not a book about the fundamental shortcomings of science, or our need for alternatives. I’ve seen books like those, and none of them have really worked for me, but maybe I just need a more open mind. But this is a step-by-step breakdown of the current failures of science, and the need for reform. The tagline of this book might as well be “Science is really, really, hard, and it’s time we take that seriously”.

Of course, that means nothing in here is revolutionary, but it does the important work of introducing these ideas in an accessible manner, and in particular showing the ways that these issues are connected. My background is statistics, so my version of this book would basically be the analysis of statistical significance, drawn out into a full length book. I never gave ideas like fraud and “hype” much thought, because those just seemed fundamental to any field. But Ritchie makes a compelling case for how these are all part of the same whole. One of my favorite moments is his early line about the fundamental nature of “trust” as the foundation of science. Research might be built on natural principles, but the way that research is integrated into scientific progress is ultimately a social exercise.

If the book has a weakness, it’s the use of certain meta analyses as ground truth, when they are obviously subject to those very same flaws the book so carefully outlines. But I don’t want to harp on that point too hard, because 1. Ritchie is obviously aware of this (and does mention it), and 2. there’s not really an alternative… a more comprehensive book would have included a more thorough discussion of the meta-analysis-of-meta-analyses, but this is meant as an introduction, not a deeply technical tome.

Best of all, this is a genuinely entertaining page turner. Ok, at least large sections are. Learning about the difficulty of science not sound as exciting as tales of its success… but so often the failures and frauds are so spectacularly grand, the anecdotes are hard to put down. That is, most of the troubles that ails science are banal, but with millions of scientists in the world, the best anecdotes are extremely spicy indeed.

Highly recommend this book, but it might make you trust your doctor quite a bit less, so be warned. ...more

flag Like · see review

Jan 07, 2021 Attila rated it really liked it

Shelves: speed-reading, science

Review:
This was an interesting book about the corruption in the publication system. It was fascinating to learn about this world, how a paper gets published and what are the scientific values.
I was definitely not the target audience. It was interesting to hear about the problems and the possible solutions.
Main points:
(view spoiler) Review:
This was an interesting book about the corruption in the publication system. It was fascinating to learn about this world, how a paper gets published and what are the scientific values.
I was definitely not the target audience. It was interesting to hear about the problems and the possible solutions.
Main points:
(view spoiler) ...more

flag Like · see review

Aug 12, 2020 Ben Chugg rated it really liked it

A sober analysis of the ways in which the current scientific institution incentivizes poor research practices. Useful for graduate students and the general public alike, the book catalogues the systemic flaws which result in unreliable results.

Most importantly, you will come away knowing what you always suspected: Nutritional science rests on a firm foundation of bull****.

While the book will most likely be received as an indictment of science, it should actually inspire awe and optimism. For i A sober analysis of the ways in which the current scientific institution incentivizes poor research practices. Useful for graduate students and the general public alike, the book catalogues the systemic flaws which result in unreliable results.

Most importantly, you will come away knowing what you always suspected: Nutritional science rests on a firm foundation of bull****.

While the book will most likely be received as an indictment of science, it should actually inspire awe and optimism. For in leveling criticism against current malpractice, Ritchie has reminded us of the capability of our scientific tradition to error-correct; to be criticized using its own tools and admit its own faults. Indeed, much of Ritchie’s analysis itself relies on scientific studies which shine light on all the (admittedly, often exceptionally fucked up …. Looking at you Paolo) ways in which the current system is sub-optimal. Contrast this with many other traditions, whether religious or political, which don’t have the ability for, or actively discourage, such self-awareness. Typical religious traditions explicitly denounce such self-criticism as heresy, for example. The fact that Ritchie can publish this book without fear of being ostracized (or stoned, or hung, or burned, or imprisoned, or eaten by a whale …) and probably contribute to bettering scientific practice should be cause for celebration. We’ve done it: we’ve created a perpetual progress machine.

Ritchie ends by giving some suggestions for how to improve the reliability of scientific findings. These include:

Multiverse analysis: looking at the average effect over many (possibly all valid) methodological choices;

Pre-registration: committing to looking a single effect, ideally using a single type of analysis;

Pre-print grading system: all papers are first distributed as pre-prints. They are then given a grade by independent investigatory groups. Journals then select the papers they’d like to publish;

Data integrity tests by journals: requiring that journals submit their papers to automated tests which pinpoint errors in the analysis;

Enforced Transparency: Requiring that all data, analysis, and initial drafts of a paper be made available for scrutiny.

All-in-all, an excellent read. ...more

flag Like · see review

« previous 1 2 3 next »

new topic
Discuss This Book

There are no discussion topics on this book yet. Be the first to start one »

Recommend It | Stats | Recent Status Updates

Readers also enjoyed

See similar books…

Genres

Science

107 users

Nonfiction

59 users

Psychology

22 users

Philosophy

11 users

Politics

7 users

Audiobook

4 users

Science > Popular Science

3 users

Sociology

3 users

Health > Medicine

2 users

Social Science

2 users

See top shelves…

About Stuart Ritchie

Stuart Ritchie

29 followers

Books by Stuart Ritchie

A Satisfying Career: Starting and Running Your Own Resume Writing Business

Fiduciary Duties: Directors And Employees

More…

News & Interviews

Check Out Speculative Fiction's Rising Stars

Kerine Wint is a software engineering graduate with more love for books than for computers. As an avid reader, writer, and fan of all things...

Trivia About Science Fictions:...

No trivia or quizzes yet. Add some now »

Quotes from Science Fictions:...

“Science, the discipline in which we should find the harshest scepticism, the most pin-sharp rationality and the hardest-headed empiricism, has become home to a dizzying array of incompetence, delusion, lies and self-deception.” — 1 likes

“To give Kahneman his due, he later admitted that he’d made a mistake in overemphasising the scientific certainty of priming effects. ‘The experimental evidence for the ideas I presented in that chapter was significantly weaker than I believed when I wrote it,’ he commented six years after the publication of Thinking, Fast and Slow. ‘This was simply an error: I knew all I needed to know to moderate my enthusiasm … but I did not think it through.’14 But the damage had already been done: millions of people had been informed by a Nobel Laureate that they had ‘no choice’ but to believe in those studies.” — 0 likes

More quotes…

See a Problem?

Preview — Science Fictions by Stuart Ritchie

Science Fictions: The Epidemic of Fraud, Bias, Negligence and Hype in Science

Get A Copy

Friend Reviews

Reader Q&A

Lists with This Book

Community Reviews

new topic
Discuss This Book

Readers also enjoyed

Genres

About Stuart Ritchie

Books by Stuart Ritchie

News & Interviews

Trivia About Science Fictions:...

Quotes from Science Fictions:...

See a Problem?

Preview — Science Fictions by Stuart Ritchie

Science Fictions: The Epidemic of Fraud, Bias, Negligence and Hype in Science

Get A Copy

Friend Reviews

Reader Q&A

Lists with This Book

Community Reviews

new topicDiscuss This Book

Readers also enjoyed

Genres

About Stuart Ritchie

Books by Stuart Ritchie

News & Interviews

Trivia About Science Fictions:...

Quotes from Science Fictions:...

new topic
Discuss This Book