What's the problem with false certainty?

What's the problem with false certainty?

Building a model to run your company is good for all sorts of reasons that we’ve discussed previously, but sometimes it can lead you to be overconfident about what the future will be like: you think the future will reflect the model.

And in some sense, building a model makes the future more certain in that it directs what you’ll be working on more closely; but the relative certainty of the future hasn’t changed just by virtue of the model’s existence.

It’s worth keeping in mind that a model is only as certain as the assumptions you’ve plugged in, and so at its worst, a model can be a house of cards built on top of faulty predictions. 

This is just one example of how when we’re staring into the abyss of an uncertain future, we have a natural tendency to seek greater certainty. This can have healthy effects; like pushing us to do things which reduce the likelihood of tail events, like spreading cash-on-hand across multiple checking accounts. 

Where does false certainty come from?

But sometimes this desire for certainty can spillover in unhealthy ways. We swap from doing things which actually reduce the range of outcomes (usually mitigating the downside); and instead do things that make the future more certain on paper. This is false certainty.

For example, Mike Bracken (co-founder of the UK’s Government Digital Service) mentioned on his blog on “false certainty” that:

I recently saw one budget spreadsheet of a leading public organisation that forecasted the cost of one day’s writing of unknown code for an unknown piece of a currently unknowable public service. In May, 2026. (Written in 2022.)

How could anyone be expected to reason about this in a useful way? 

This kind of thinking is driven by ERP processes, where the finance department will ask the rest of the organisation something like, “how many resources will you need to do X {thing} over the next budget cycle?”, and then require the rest of the organisation to say something other than “we’re not sure yet”. 

The finance department has good intentions: to help the leadership to coordinate the organisation’s resources, and to create plans for the future, but it leads to “false certainty”. They’re requiring answers to questions that aren’t answerable yet. The pernicious aspect of thinking in these terms, like believing that it’s possible to say anything about (e.g.) per-day output on an unknown project in four years time, is that any confidence you can derive from an answer is artificial. It’s confidence on paper; not about the state of how things really are.

How can we avoid imposing false certainty on the future?

Avoiding false certainty begins with how you create your assumptions. When you have to come up with an assumption, it’s better to use a confidence interval, rather than a single number. For example, if you’re asked: “how many demos can we get booked in a month?”, it’s more helpful to say: “I’m 90% confident that we’ll get between 3 and 7,” rather than, “probably 5”.

Why is this more helpful?

  1. The first statement includes the variance, meaning how far you might miss the target by, either on the upside or the downside.
  2. It avoids getting a distorted answer because someone wants to compensate on one side. If you ask someone: “How much is this going to cost?”, they might tell you $100k, when they’re expecting $80k, because they want to have some wiggle room. Instead if they can tell you they’re expecting it to be between $75k and $110k, you’re able to keep greater flexibility.

By using these confidence intervals, you’re not forced into being artificially confident about the future.

With Causal, this is a really easy change to make, because using confidence intervals is built into the software. When you have a value, you’re able to input “$75k to $110k” rather than a single number, and it will dynamically run a monte carlo simulation on your graphs to show you the range of outcomes. (You can also update your confidence intervals in the graphs editor.)

Live Example


The difficulty with beginning to make these changes is that it requires a cultural shift alongside, because the finance department will need to be comfortable with hearing a range of outcomes when they ask: “How much will this save/cost?”. Most companies are fundamentally conservative, and will select for projects which have more certain outcomes. But that can mean lots of projects which have uncertain outcomes, but potentially high payoffs can be left on the table. These projects shouldn’t be forced, by virtue of false certainty, to claim that can be confident about the future. 

If you’re looking to avoid false certainty in your planning process, using Causal can change how you interact with an uncertain future.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
PERSONAL FINANCE
Buy vs Rent
Should you buy a house or rent?
StartuP
B2B SaaS Revenue
Forecast your inbound and outbound leads to determine revenue, and understand what kind of sales funnel you need to hit your revenue targets.
Finance
Detailed Headcount Model
Understand the breakdown of your headcount and payroll costs by Department (Sales, Engineering, etc.) and plan your future hires.

What's the problem with false certainty?

Apr 4, 2023
By 
Taimur Abdaal
5-Min Read
Table of Contents
Heading 2
Heading 3

Building a model to run your company is good for all sorts of reasons that we’ve discussed previously, but sometimes it can lead you to be overconfident about what the future will be like: you think the future will reflect the model.

And in some sense, building a model makes the future more certain in that it directs what you’ll be working on more closely; but the relative certainty of the future hasn’t changed just by virtue of the model’s existence.

It’s worth keeping in mind that a model is only as certain as the assumptions you’ve plugged in, and so at its worst, a model can be a house of cards built on top of faulty predictions. 

This is just one example of how when we’re staring into the abyss of an uncertain future, we have a natural tendency to seek greater certainty. This can have healthy effects; like pushing us to do things which reduce the likelihood of tail events, like spreading cash-on-hand across multiple checking accounts. 

Where does false certainty come from?

But sometimes this desire for certainty can spillover in unhealthy ways. We swap from doing things which actually reduce the range of outcomes (usually mitigating the downside); and instead do things that make the future more certain on paper. This is false certainty.

For example, Mike Bracken (co-founder of the UK’s Government Digital Service) mentioned on his blog on “false certainty” that:

I recently saw one budget spreadsheet of a leading public organisation that forecasted the cost of one day’s writing of unknown code for an unknown piece of a currently unknowable public service. In May, 2026. (Written in 2022.)

How could anyone be expected to reason about this in a useful way? 

This kind of thinking is driven by ERP processes, where the finance department will ask the rest of the organisation something like, “how many resources will you need to do X {thing} over the next budget cycle?”, and then require the rest of the organisation to say something other than “we’re not sure yet”. 

The finance department has good intentions: to help the leadership to coordinate the organisation’s resources, and to create plans for the future, but it leads to “false certainty”. They’re requiring answers to questions that aren’t answerable yet. The pernicious aspect of thinking in these terms, like believing that it’s possible to say anything about (e.g.) per-day output on an unknown project in four years time, is that any confidence you can derive from an answer is artificial. It’s confidence on paper; not about the state of how things really are.

How can we avoid imposing false certainty on the future?

Avoiding false certainty begins with how you create your assumptions. When you have to come up with an assumption, it’s better to use a confidence interval, rather than a single number. For example, if you’re asked: “how many demos can we get booked in a month?”, it’s more helpful to say: “I’m 90% confident that we’ll get between 3 and 7,” rather than, “probably 5”.

Why is this more helpful?

  1. The first statement includes the variance, meaning how far you might miss the target by, either on the upside or the downside.
  2. It avoids getting a distorted answer because someone wants to compensate on one side. If you ask someone: “How much is this going to cost?”, they might tell you $100k, when they’re expecting $80k, because they want to have some wiggle room. Instead if they can tell you they’re expecting it to be between $75k and $110k, you’re able to keep greater flexibility.

By using these confidence intervals, you’re not forced into being artificially confident about the future.

With Causal, this is a really easy change to make, because using confidence intervals is built into the software. When you have a value, you’re able to input “$75k to $110k” rather than a single number, and it will dynamically run a monte carlo simulation on your graphs to show you the range of outcomes. (You can also update your confidence intervals in the graphs editor.)

Live Example


The difficulty with beginning to make these changes is that it requires a cultural shift alongside, because the finance department will need to be comfortable with hearing a range of outcomes when they ask: “How much will this save/cost?”. Most companies are fundamentally conservative, and will select for projects which have more certain outcomes. But that can mean lots of projects which have uncertain outcomes, but potentially high payoffs can be left on the table. These projects shouldn’t be forced, by virtue of false certainty, to claim that can be confident about the future. 

If you’re looking to avoid false certainty in your planning process, using Causal can change how you interact with an uncertain future.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.