INCREASING FORECASTING ACCURACY: A SHOWCASE OF APPLIED EXPERIENCE

GEMSEEK WHITE PAPER SERIES

FORECASTING AT GEMSEEK: TRADITION MEETS INNOVATION

Forecasting is an integral part of GemSeek’s activities. We have gained a lot of experience by analyzing the Healthcare and medical equipment markets in numerous destinations around the globe and in dealing with various types of data sets. One of the most important objectives of these projects in general was to provide a forward looking outlook of how the market size of different markets (a.k.a.modalities) would develop in the future. The model provides three different techniques each bringing a choice in finding more accurate forecasts with regards to various factors, local market specifics, data distribution etc. In summary, these are:

• Linear Trend Regression (a.k.a Time extrapolation) captures the linear trend of the Time Series Data on Imports per Segment & Total Market

• Exponential Smoothing uses the linear trend and fluctuations of time of the Time Series Data on Imports and applies different weights to actual values so the data looks more smoothed. Also, uses smoothed values to extrapolate future levels

• Multi Regression binds the actual values of Imports to a set of external predictors. Uses forecasted data on selected macroeconomic indicators to estimate future values of the market size.

The above list is by no means exhaustive. By choosing these particular methods we strive to provide the simplest and the most robust ways of analyzing data and get insights about the future developments of the markets. At the same time we aim to facilitate the Client in terms of availability of dedicated software, easiness to use, interactivity and other features that make the models comprehensive and valuable tools. A different approach in forecasting provides an alternative viewpoint on market developments aligning with industry peculiarities.

INCREASING FORECASTING ACCURACY: A SHOWCASE OF APPLIED EXPERIENCE

GEMSEEK WHITE PAPER SERIES

We leave the final decision to final users, accounting for their topmost expertise, knowledge of the market and awareness of specific, occasional events and ad-hocs that could potentially affect industry trends. Having this optional choice proved to play pivotal role in the decision making process regarding adoption of company strategies and future policies.

CHALLENGES IN FORECASTING ACCURACY

Inevitably some questions arise regarding forecasting accuracy. Theoretically, a statistical forecast could yield accurate results, provided that data sets meet strict rules. In reality, however, things prove otherwise. In many cases an analyst has to deal with fragmented information on markets e.g. only partial information is available, some periods of data are either missing completely or numbers are unreliable. Another issue that significantly hampers output results lies in limited time series and small number of observations in historical terms.

Even the most trusted data libraries have started gathering data not so while ago. Hence, time series cannot provide enough information on market trends and no valid forecasts are therefore possible. Also, we have faced no regular (granular) statistical data on macro drivers, which hampered additionally forecasting possibilities and limited the usage of regressions and other exploratory methods relying on relationship between market fluctuations and possible predictors.

ACHIEVING FORECASTING ACCURACY: THE GEMSEEK APPROACH

Dealing with these and other issues working alongside with Clients’ demands, GemSeek elaborated an entire process of accounting and controlling the forecasting estimates which could be summarized in three basic steps:

1. Determining the actual level of forecast deviation.

Statistical theory provides helpful tools in measuring how actual values of the markets deviate from predictions already made.

INCREASING FORECASTING ACCURACY: A SHOWCASE OF APPLIED EXPERIENCE

GEMSEEK WHITE PAPER SERIES

Again, choosing the right ones determines the operational, practical and theoretical integrity of final outcomes. The most common measures of accuracy are the Mean Square Error (M.S.E.), Theil's U-coefficient (Theil, 1966) and the Mean Absolute Percentage Error (M.A.P.E.). All of them are based on quadratic loss function, therefore prove helpful when determining parameters based on GLS. All of these methods provide a general overview on how predicted values derived from one of the forecasting techniques mentioned earlier actually differs from real readings of the market for the periods that we already have data on.

In other words, each method is compared to naïve one-step-ahead forecast i.e. assuming that current period value would be the same as the one from the previous.

This actually underpins the usage of historical type of forecasting instead of extending the trend line and extrapolating it with the growth from the previous period.

Despite some shortcomings, these methods are relatively easy to explain and execute, balancing for their fairly high level of accuracy.

INCREASING FORECASTING ACCURACY: A SHOWCASE OF APPLIED EXPERIENCE

GEMSEEK WHITE PAPER SERIES

2. Adjusting future outlooks with estimated errors

At this stage, the forecast accuracy is calculated by comparing the predicted (forecasted) values with the actual numbers of the market. Since the total error is calculated based on the difference, its validity is higher when more actual vs. predicted periods are observed and integrated in the model. Since GemSeek started the whole forecasting process, we’ve collected quite a significant data set that we could use for adjusting forecasting errors. The next step was adjusting next forecasted periods using calculated deviation value as a reference point either by correcting the-next-in-line predicted value itself or setting a confidence range around it, this depicting how far the market could span across. The whole process, also known as evaluation procedure between ex-post and ex-ante forecasts, could be explained using a slightly ameliorated pictogram suggested by Small & Wong 2002. See Figure 1.

3. Dynamic integration of actual data into error estimation.

Since all models are ongoing and time periods are observed over time, we apply a rolling forecast values adjustment. The process assumes new estimations every time new actual data is available. Each error estimator is recalculated including real values from the last period and the respective coefficient is then corrected. This ensures the respective measure is fine-tuned by increasing the base on which coefficients are estimated.

Figure 1. Explaining the forecasting adjustment process*

INCREASING FORECASTING ACCURACY: A SHOWCASE OF APPLIED EXPERIENCE

GEMSEEK WHITE PAPER SERIES

The above described method should be interpreted with caution as it accounts only for stochastic error i.e. average fluctuation of estimated values caused by random factors and specific competitive conditions also known as “white noise”. Estimating the actual error, on the other hand, could be quite challenging. The reason for that is the necessity to anticipate various non-market and sometimes accidental events, sudden turns in public policies, economic collapses etc. that could have radical impact on the markets and could significantly bias projections. Even if such conditions were known prior to actual forecasting, proper quantification of their effects could also pose problems. Hence, the unbiased estimations would be compromised as in most of the cases “real” adjustment indicators are based on soft data, subjective assumptions or expert evaluations. All these, taken individually or in as a whole have their impact on forecasting accuracy. Depending on market’s specifics these could have significant impact on future outlooks regardless of level of sophistication of actual forecasting techniques used.

Therefore, forecasting in the sense of historical estimations of trend and aggregation of past behavior and all forecasts thereafter should be regarded as estimators of market mainstream i.e. the general direction where the specific market is heading rather than concentrating on specific values for each time period. The next stage, which inevitably leads to more precision, is applying post estimations adjustments on these individual levels based on proper measurement of external factors and events which are not directly involved into the model.

Regardless of the forecasting technique used, controlling the forecast accuracy is a vital process in decision making especially in markets expressing high volatility. Despite some flaws of existing methodologies regarding proper estimations of forecasting errors having market fluctuations under control could, by no means, bring more value to future estimations of the market.

INCREASING FORECASTING ACCURACY: A SHOWCASE OF APPLIED EXPERIENCE

GEMSEEK WHITE PAPER SERIES

We, at GemSeek believe that these should be achieved by using an entire process of well-defined multistage mechanism which starts with time investment in primary research on specific peculiarity of the respective market and going through selection of the proper statistical time series analysis. At a later stage actual estimations on errors should be calculated and then used as post estimation adjustments of forecasts. Lastly, individual levels should be controlled, accounting for external factors, apart from those already in the model. Only by using these logically justified steps, some of the problems of uncertainty accompanying the forecasting process could be tackled and kept within controllable boundaries. In this respect, applying those techniques would render increasing the accuracy of forecast a bit closer to desired levels of business sense and practicality.

Martin Dimov is a Senior Statistician and Data Modeler at GemSeek. He helps businesses make sense of data by suggesting the most appropriate statistical and modeling approaches and analyses to inform strategic decision making.To learn more about forecasting in business context, contact us at office@gemseek.com

LIKED THIS WHITE PAPER?

Like GemSeek in social media to receive the latest updates.