Forecasting Methods and Forecast Modeling

Sources of Forecasting Errors
Although larger samples improve forecasting precision, samples may be limited if older data are unavailable or not comparable. Data collected more frequently increases sample size but may not add much information.

Because forecast inferences cannot be based on future data, extrapolation of past relationships results in an unknown amount of bias, especially if (1) explanatory variables move outside their historical range (2) some variables no longer are important while others become important (3) variable coefficients change substantially or switch signs

Conditional forecasting occurs when one or more explanatory variables must be guessed because their values not known with certainty for the period forecast.

Unconditional forecasting occurs when the future values of all explanatory variables are known with certainty. In conditional forecasting, errors may be huge because we first must forecast values of the explanatory variables. Only unconditional forecasts are free of these errors.

Contingency forecasting involves generating several forecasts, one for each alternative set of circumstances, or "scenario," that is likely to arise.

The estimation period is the time series data used to fit a forecasting model. Ex post forecasting involves "forecasting" the most recent observations after withholding them from the estimation period. By contrast, ex ante forecasting uses an estimation period that includes the most recent observations.

Ex post forecasting is a valuable method for evaluating performance of time series models. Before making a judgment, however, forecast several observations, examine model performance under different number of periods forecast, consider only models defensible on prior grounds, and include conditional forecasting errors in your analysis.

Average Forecast Error, Forecast Bias, and Turning Points
A positive bias describes forecasts that are too high on average. Negative bias results if average forecasts are too low. Bias near zero occurs when all errors are small, but zero bias also happens if positive and negative forecast errors cancel out.

The root mean square error, RMSE, is the square root of the mean square forecasting error. The mean absolute error, MAE, is the mean magnitude of the forecasting errors. Unless all forecast errors have identical magnitudes, RMSE will be larger than MAE. This difference is greatest if one error is much larger than the rest. Also, the magnitude of the bias B will equal MAE if all forecast errors have the same sign.

The mean absolute percent error, MAPE, is the mean absolute error expressed as a percent of the actual value being forecast.

A turning point is a single time period or narrow range of periods that precedes a sharp (and usually long-term) change in direction of in a time series variable.

If MAPE is tolerably small for all models, then model selection may be based on criteria other than MAE and RMSE. The ability to forecast turning points and avoid conditional forecasting errors are factors that also should be considered.