Forecast Error Metrics to Assess Performance

If something is not measured, it will never improve. To improve forecast performance, we must have metrics in place to measure and monitor them. There will be inherent volatility and variability in what we forecast and if there is any human judgement, there is the possibility of bias in our consensus Demand Plan.

Why Measuring Forecast Error Is Important

If we can measure and recognize the error, we can do something about it. Plus, with the knowledge of the error, we will be in a better position to make better decisions as a business and manage the risk associated with it. Because of this, every step of the Demand Planning process should have two fundamental objectives:

• To create the most accurate demand forecast (best prediction of what is actually going to happen) with the lowest error.
• To alleviate bias whenever possible.

It is clear that forecast error and bias - in either direction - can be problematic not only to a supply chain, but to the entire organization. The over-forecasting error increases cost of inventory, transshipment, shrinkage, and obsolescence. The under-forecasting error increases production, procurement and transportation costs, as well as the loss of sales because of stock-outs.

Popular Forecast Error Metrics

To measure and mitigate, this we use some key performance indicators for forecasting:

• Mean percentage error (MPE). Average percent of error, a measure of variation. Forecast accuracy and sometimes as an average MPE used for proxy on bias.

• Mean Absolute percentage error (MAPE). To account for both positive and negative errors, we compute the average of percentage errors with signs ignored, that is, average of absolute percentage error. Measure of forecast phasing and/or mix error

• Weighted Mean Absolute percentage error (WMAPE). Sum of the product of errors over a set of parameters. Measures mix error and phasing.

• Forecast Value Added (FVA). Measures the change in a performance metric that can be attributed to a particular step or participant in the forecasting process. Adds visibility into the inputs and provides a better understanding of the sources that contributed to the forecast, so one can manage their impact on the forecast properly.