When even a scientific magazine like Wetenschap in Beeld (Science in the Picture) makes huge errors like this, errors in their thinking as much as in their facts, it’s no wonder that less and less people really understand science, and more and more people lose faith in it.
In their April issue, WiB printed an article making the case that there may be 500 habitable planets among the 4,000 known exoplanets. And there may even be as many as 2,000 habitable moons circling those exoplanets. Their reasoning, or rather, their statistical method, is embarassingly flawed.
Look, they say, in our Solar system, we have eight planets, right, and one of them supports life. So, in any given solar system, the chance of finding a habitable planet is 1:8. With 4,000 known exoplanets, that means 500 might support life! Also, they continue, there are four moons in our Solar system that might support life. So the ratio of habitable planets to habitable moons is 1:4. And we’ve already concluded that there are 500 habitable planets, so there are 2,000 habitable moons!
The numbers check out, you may say, so what’s wrong with their reasoning?
Everything.
For starters: it’s ridiculous and sheer bad thinking to start a chain of reasoning with an actual inhabited planet, and make the jump to moons that might conceivably be capable of supporting life. That’s like starting your argument for the prevalence of cake by counting actual cakes, and expanding your argument by including any room where flour, milk, and eggs* are in proximity .
But that’s not even what bothers me most. What bothers me most is their statistical method, or rather, their utter lack of one.
Earth is the only planet with life of the eight in our Solar system, so the ratio of habitable planets in any solar system is 1:8.
This conclusion is drawn from a single sample: our Solar system. And as any student with even a basic understanding of statistics can tell you, you cannot draw conclusions of any kind based on sample size 1. Our Solar system might be average, in which case their conclusion might be correct. Or life on Earth might be a freak accident, in which case the chance of life would be much, much smaller. Or our Solar system might be singularly inimical to life, in which case the chance of life might be much larger.
Point is: our N=1 Solar system does not provide any grounds whatsoever to support the 1:8 conclusion.
And looked at it from a purely statistical standpoint: their sample measurement may be a chance of 12,5%, but their standard deviation is infinity (because N-1 is the denominator), so statistically, they can only conclude that the chance of life on a planet is 12,5%, plus or minus infinity. In fact, the only thing our Solar system proves with any certainty is that the chance of life is greater than 0.
And of course, they make the same mistake two more times in the same piece. Once, when they conclude from the same N=1 sample that the ratio of habitable moons to planets is 4:1. And once more, when they take that already flawed ratio, sourced from an N=1 sample, into the rest of the universe.
This is a science magazine, and not a single thing it states in the article is supported by evidence.
* Or whatever goes into cakes; I’m no baker.
