When is good data impossible?
Was thinking recently about skepticism in small sample sizes. How I always doubt big claims based on scant data.
But what things in life can never have large samples? Some ideas or beliefs will never be shared by many people no matter how useful or right they are.
The scientific method is based in part of repeatability. That an experiment should produce consistent results and that's how we know a discovery is true. But what about things that are true, but are simply hard to repeat? To be provocative, what if perpetual motion is possible, but only once every 100 years? Or if UFOs exist, but they have equipment to ensure they only appear when crazy people with bad cameras are around? Sure, these things are very unlikely, but are impossible to prove.
The lack of data about a premise does not guarantee it isn't true, it only guarantees it hasn't been proven to be true. And my point is, some things that are true will always fall in that gap.
More specific to my half-baked line of inquiry: What situations in life have no possibility for good data, and mandate we make decisions anyway? I think there are more of these situations than we realize.
Related posts:New essay: How to survive a bad manager
This week in uxclinic: Death by comparison
The data death spiral
Help decide the title of a book
Experience is overrated
