Fake Science

Fake Science is a beautiful term. It perfectly captures the art of defining a metric, taking measurements, analysing those data points, and continually generating a complex output before chaining it to other inputs for yet more bullshittery.

This is something I see almost on a daily basis, and it gets to the core of what triggers my skepticism when ingesting any media:

1. What’s the source of the data?
2. What is the precision of your medium of measurement?
3. What is the power of your measurement mechanism?
4. What’s the proximity between the data and the inference?

Note that the source here is a special factor (and not just because it doesn’t start with a P.) The source of an inference is the primary heuristic that we use to save ourselves from having to apply maximum discernment to everything we hear. If you trust someone you’re more likely to accept their inference.

If an aeronautical engineer with 3 decades of experience at the most successful companies in the field lets me know that garden gnomes are not optimal for travelling through space efficiently, I’m likely to take it on face value.

Fake Science rolls in when we take low precision or power data and make inferences that are sufficiently distant from the data itself or the more common mistake – we chain inferences using bad outputs and therefore inputs.

1. Observation: I glance out the window and infer that my friend likes gardening
2. Observation: when asked “how are you doing?” my friend replies “yeah not bad”
3. Observation: I remember they were having a hard time last time we spoke
4. Inference: gardening was the cause of the improvement
5. Inference: I’m going to start gardening
6. Inference: large numbers of gnomes drive an increase in happiness

At every point here we’re making bad choices if we want really high quality inferences that we can rely on. At step 2 both the question and the answer are vague. Step 3 relies on my own memory and interpretation of the situation. Step 4 is a classic correlation vs. causation problem. But steps 5 and 6 are those proximity issues leaking in – we’re layering inferences and driving unwarranted conclusions.

I think most people with critical thinking faculties beyond those of a gnome are comfortable isolating clear correlation/causation issues, but when our things go awry when our we lack the domain knowledge to actually know if a source, precision, or proximity are high quality.

This is the classic example peddled every day. Happiness is vague because there isn’t a shared definition across all people. And “says study” or the phrase “you know studies have shown” immediately lets you know that the claim is most likely entirely wrong (sorry Scunthorpians.)