Risk Analysis in Estimating: do-it-yourself (DIY) Monte-Carlo Simulation

5–7 minutes

No matter what class of estimate it is, every estimator makes his or her slightly different assumptions based on their own experience. An estimate is finally an opinion of a particular estimator. Some estimators tend to be pessimistic in every assumption, factor, historic data they use, ending up with a very conservative (i.e. high) overall estimate. For the same technical inputs, which might themselves not be perfect, another estimator might make optimistic assumptions for individual elements of the project, which might make the overall estimate very low. In both cases, the estimator is likely to add a contingency to their estimate.

Most often, the assumptions made by individual estimators are not transparent as they are generally not properly recorded or documented. Besides this, the decision-maker is usually given only one estimate from an individual estimator to look at, which does not give them the option to question or critique the possible range of the overall estimate.

If somehow the same estimate could be done by multiple estimators, and all those estimates were tabulated together, we would get a range of estimates, with a maximum and a minimum value, which would allow everyone to understand the possible range of cost outcomes of a future project. But it would be very expensive or nearly impossible to get the project estimated by several estimators at the same time.

But there is a workaround to this! We could instead use MS Excel to do multiple estimates. And not only that, but we could also get Excel to run 5,000 or even 10,000 estimates almost instantaneously (for free!!).

  • This is like predicting 10,000 possible scenarios out of the infinite possibilities (simulating reality by trying to analyse many possibilities).
  • If we could analyse and simply present these 10,000 results, we could in a way say what the likelihood (probability) of success for a particular point estimate would be.
  • This would be better (and more transparent) than trying to present a single number.

Nobody knows the future, so whatever we predict is likely to be wrong; the point is to reduce the amount of error in our predictions. So, instead of trying to produce a correct single number (Point Estimate), I suggest that we generate a reasonable range instead (Range Estimate).

This is called Monte Carlo simulation, and it is a mathematical model of probability analysis. There are various companies selling risk analysis (Monte Carlo simulation) software in the market, but the cost for those tools (toys!) are sometimes not justifiable even within large organisations. But a range estimate would be really useful by allowing decision-makers to understand what contingency would be suitable for a particular estimate and at what risk level are they pitching the overall number.

I have developed something easy and simple to help bring the team together and run these simulations to see what the possible range of the overall estimate could be:

  • Let us say that you come up with a point estimate for the project – say the total is 100
  • And the estimate is built up by adding, say, 20 different items (or more)
  • But each of those different items could have been assigned a different number (within a particular range)
  • The team can come together to understand the assumptions made by the estimators for those individual items, and can together decide an appropriate range for each item

Once those ranges are defined, we could ask Excel to randomly pick any number between those ranges for each of those items (by using the Excel formula “Randbetween”) and add them up to come up with a new estimate, as shown below:

We could do this 10,000 times to come up with 10,000 new estimates. This is how a simple DIY Monte-Carlo simulation model could be made.

Typical results will look like the following:

Interpretation:

  • The first graph plots all the 10,000 estimates (simulations) generated by this method [using a simple scatter graph]
  • Based on these 10,000 simulations, the second graph shows what are the chances of completing a project within the “Point Estimate” value. [this simply plots the Percentiles of all the 10,000 values using a scatter graph]
  • Do note that in a real life situation there is no chance of all possible worst cases happening simultaneously, nor all the best cases can happen for a project. Therefor the total of all the maximums and all the minimums does not make any sense, and the Monte-Carlo results also show that they never occurred within the 10,000 simulations.

The file attached below is the model I have developed as explained here and is a free estimating resource for anybody who wishes to download and use.

This example template file is designed to take up to 50 items in the inputs tab (this is the breakup of the point estimate), with their estimated values and the max-min ranges for each value. The inputs tab is then linked to the Monte-Carlo Results tab where the 10,000 simulations are run by simply pressing “Fn&F9” together on the keyboard. This tab also presents the results in the graphical form presented earlier. If needed, this file can be modified easily to increase the number of items adding up to a total point estimate.