Tracking SignalTracking SignalA Measure of Forecast AccuracyA Measure of Forecast Accuracy

Prepared by:

Tyler Hedin

• Tracking Signal Defined– Tracking Signal and Forecasting

• Application, Advantages & Disadvantages• How it works

– Step by step formula• Company XYZ Example• Exercise• Summary• Readings list• Useful websites• Appendix A

AgendaAgenda

• A measure that indicates whether the forecast average is keeping pace with any genuine upward or downward changes in demand*

What is Tracking Signal?What is Tracking Signal?

Tracking Signal and ForecastingTracking Signal and Forecasting

• Continuous control indicator

• Monitor effectiveness of forecasting method

• Provide control limits

ApplicationApplication

• Evaluates forecasting method

• Indicator of change in demand patterns

• Used in conjunction with anything dependent on future demand – Sales– Inventory

AdvantagesAdvantages

• Unbiased

• Versatile– Can be used with any type of forecasting

method (time series, regression line, etc.)

• Could wrongfully flag perfect forecasts– Unlikely

• Small differences in the same direction could cause signal to go outside of control limits

DisadvantagesDisadvantages

How it Works – Forecast ErrorHow it Works – Forecast Error

• Difference between actual demand and forecast

Week Actual Demand

Forecasted Demand

Forecast Error

1 21 19 2

2 25 22 3

3 22 24 -2

How it Works – Absolute ValuesHow it Works – Absolute Values

• Express the forecast errors as absolute values

WeekActual

DemandForecasted

DemandForecast

ErrorAbsolute

Value

1 21 19 2 2

2 25 22 3 3

3 22 24 -2 2

How it Works – Running SumHow it Works – Running Sum

• Keep a continuous running sum of the forecast errors

• Do not add absolute values

WeekActual

DemandForecasted

DemandForecast

ErrorAbsolute

ValueRunning

Sum

1 21 19 2 2 2

2 25 22 3 3 5

3 22 24 -2 2 3

How it Works - MADHow it Works - MAD

• Divide the summed absolute values by the number of periods to calculate MAD.

WeekActual

DemandForecasted

DemandForecast

ErrorAbsolute

ValueRunning

SumMAD

1 21 19 2 2 2 2.00

2 25 22 3 3 5 2.50

3 22 24 -2 2 3 2.33

The EquationThe Equation

• Tracking signal is mathematically defined as the sum of the forecast errors divided by the mean absolute deviation

Tracking signalTracking signal == (Dt – Ft)

MAD

How it Works – Tracking SignalHow it Works – Tracking Signal

• Divide the running sum of forecast errors by the corresponding MAD value

WeekActual

DemandForecasted

DemandForecast

ErrorAbsolute

ValueRunning

SumMAD Tracking

Signal

1 21 19 2 2 2 2.00 1.00

2 25 22 3 3 5 2.50 2.00

3 22 24 -2 -2 3 2.33 1.29

What Do These Values Mean?What Do These Values Mean?

• Ratio of cumulative error to average deviation

• 0.8 σ ~ 1.25 MAD

• Limits are usually between 2 to 5 standard deviations

Example 1Example 1

• Company XYZ has implemented a linear regression method to forecast sales. Actual sales for the months of January 2005 through January 2006 are given in Table 1 along with their corresponding forecasts.

  SALES (in thousands)  

MONTH PERIOD DEMAND FORECAST

January-05 1 $37 37.35

February-05 2 $40 38.97

March-05 3 $41 40.60

April-05 4 $37 42.23

May-05 5 $45 43.85

June-05 6 $50 45.48

July-05 7 $43 47.10

August-05 8 $47 48.73

September-05 9 $56 50.36

October-05 10 $52 51.98

November-05 11 $55 53.61

December-05 12 $54 55.23

January-06 13 $55 56.86

Table 1Table 1

Example 1Example 1

• Company XYZ would like to employ a tracking signal to measure the performance of its forecasting method.

Table 2Table 2

MONTH PERIOD DEMAND FORECAST ERROR ABS DVN RUNNING SUM MADTRACKING

SIGNAL

January-05 1 37 37.35 -0.35 0.35 -0.35 0.35 -1.00

February-05 2 40 38.97 1.03 1.03 0.68 0.69 0.99

March-05 3 41 40.60 0.40 0.40 1.08 0.59 1.82

April-05 4 37 42.23 -5.23 5.23 -4.15 1.75 -2.37

May-05 5 45 43.85 1.15 1.15 -3.00 1.63 -1.84

June-05 6 50 45.48 4.52 4.52 1.52 2.11 0.72

July-05 7 43 47.10 -4.10 4.10 -2.58 2.40 -1.08

August-05 8 47 48.73 -1.73 1.73 -4.31 2.31 -1.86

September-05 9 56 50.36 5.64 5.64 1.33 2.68 0.50

October-05 10 52 51.98 0.02 0.02 1.35 2.42 0.56

November-05 11 55 53.61 1.39 1.39 2.74 2.32 1.18

December-05 12 54 55.23 -1.23 1.23 1.51 2.23 0.68

January-06 13 55 56.86 -1.86 1.86 -0.35 2.20 -0.16

TRACKING SIGNAL

-1.00

0.99

1.82

-2.37

-1.84

0.72

-1.08

-1.86

0.50

0.56

1.18

0.68

-0.16

ExerciseExercise

• Your employer, Jones & Associates, has been using a linear regression method to forecast sales for 2006. After nine months have passed and actual sales data have been collected, your boss asks you to develop a tracking signal to measure the accuracy of the forecasts. The data for actual sales and forecasted sales is in Table 3.

Table 3Table 3

  SALES  

MONTH PERIOD DEMAND FORECAST

January-06 1 $3,769 3664.18

February-06 2 $3,912 3953.92

March-06 3 $4,212 4243.65

April-06 4 $4,861 4533.39

May-06 5 $4,672 4823.13

June-06 6 $4,937 5112.87

July-06 7 $5,346 5402.61

August-06 8 $5,783 5692.35

September-06 9 $6,021 5982.08

SummarySummary

• A tracking signal statistically determines if a forecasting method is out-of-control.– As long as tracking signal stays within 3 standard deviations,

probability of forecast error caused by random variation is high

• Used by companies to track changes in demand patterns• Calculated by dividing the most recent sum of forecast

errors by the most recent estimate of MAD• A tracking signal outside of established limits indicates

that a forecasting method should be modified.• Compatible with any forecasting method

Readings ListReadings List

• Chase, R. B. et al. (2004). Operations Management for Competitive Advantage 10th edition. McGraw-Hill Higher Education.

• Duncan, Robert M. (1992). Quality Forecasting Drives Quality Inventory at GE. Industrial Engineer, January edition.

• Hanke, J.E. & Wichern, D. W. (2004). Business Forecasting. Prentice Hall.

• Lawrence, F. B. (1999). Closing the logistics loop: A tutorial. Production & Inventory Management Journal, 40(1).

Useful WebsitesUseful Websites

• http://www.bestforecastingsoftware.com• http://www.IdeaWins.com• http://www.lehigh.edu/~rhs2/IBE098/forecating.ppt• http://is.ba.ttu.edu/faculty/ch11.ppt• http://www.microsoft.com/dynamics/intro/default.mspx

Appendix A – Solution to ExerciseAppendix A – Solution to ExerciseMONTH PERIOD DEMAND FORECAST ERROR ABS DVN RUNNING SUM MAD

TRACKING SIGNAL

January-05 1 37 37.35 -0.35 0.35 -0.35 0.35 -1.00

February-05 2 40 38.97 1.03 1.03 0.68 0.69 0.99

March-05 3 41 40.60 0.40 0.40 1.08 0.59 1.82

April-05 4 37 42.23 -5.23 5.23 -4.15 1.75 -2.37

May-05 5 45 43.85 1.15 1.15 -3.00 1.63 -1.84

June-05 6 50 45.48 4.52 4.52 1.52 2.11 0.72

July-05 7 43 47.10 -4.10 4.10 -2.58 2.40 -1.08

August-05 8 47 48.73 -1.73 1.73 -4.31 2.31 -1.86

September-05 9 56 50.36 5.64 5.64 1.33 2.68 0.50

October-05 10 52 51.98 0.02 0.02 1.35 2.42 0.56

November-05 11 55 53.61 1.39 1.39 2.74 2.32 1.18

December-05 12 54 55.23 -1.23 1.23 1.51 2.23 0.68

January-06 13 55 56.86 -1.86 1.86 -0.35 2.20 -0.16