Research Brief Forecasting Effectiveness · PDF fileto-business firms’ sales forecasting practices, quantifies ... This study was made possible in part through the underwriting support

Research Brief Forecasting Effectiveness March 2015

Summarizing research conducted in Q4 2014 with 92 participating business-to-business sales organizations

Research Underwriter

Author Robert J. Kelly Chairman The Sales Management Association

First published April 2015. Sales Management Association 1440 Dutch Valley Place NE Suite 990 Atlanta, Georgia 30324 USA +1 (404) 963-7992 http://salesmanagement.org Copyright © 2014-15 The Sales Management Association, Inc. All rights reserved; no part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without prior written permission of the publishers. Research Brief: Forecasting Effectiveness

This document has been prepared by The Sales Management Association for use by its members. The Sales Management Association has worked to ensure the accuracy of the information it provides to its members. This report relies upon data obtained from many sources, however, and The Sales Management Association is not engaged in rendering legal, accounting, or other professional services. Its reports should not be construed as professional advice on any particular set of facts or circumstances. Members requiring such services are advised to consult an appropriate professional. Neither the Sales Management Association nor its programs are responsible for any claims or losses that may arise from a) any errors or omissions in their reports, whether caused by The Sales Management Association or its sources, or b) reliance upon any recommendation made by the Sales Management Association. Descriptions or viewpoints contained herein regarding organizations profiled in this material do not necessarily reflect the policies or viewpoints of those organizations. About The Sales Management Association

The Sales Management Association is a global, cross-industry professional organization for sales operations, sales effectiveness, and sales leadership professions. We provide our members with tools, networking, research, training, and professional development. Our research initiatives address topics relevant to practitioners across a broad spectrum of sales effectiveness issues. Our research is available to members on our site at www.salesmanagement.org.

In addition to research we publish best practice tools, archived webcasts, and expert content. Visit our website at http//salesmanagement.org to learn more.

Copyright © 2014-15 the Sales Management Association. All rights reserved. 2

Research Brief: Forecasting Effectiveness

Contents

1

1.1 1.2 1.3

1.4

About the Research 5 Executive Summary 5 Research Approach 6 Research Timing and Scope 6 Research Underwriter 6

2

2.1 2.2

Forecasting Practices 6 Forecast Frequency 6 Performance Measures Included in Forecasting 8

3 3.1 3.2

3.3

3.4 3.5 3.6 3.7

Evaluating Forecasting Effectiveness 9 Defining Forecasting Effectiveness 9 Firm Forecasting Effectiveness is Low 10 Forecasting is More Difficult for Larger Firms 10 Rating Effectiveness For Individual Program Elements 11 Prioritizing the Impact of Forecasting Program Elements on Overall Forecasting Effectiveness 12 Forecast Accuracy by Performance Measure 14 Number of Required Performance Measures’ Impact on Forecasting Effectiveness 14

4

4.1 4.2 4.3

5 5.1

5.2

5.3 6 6.1

6.2 6.3

6.4

6.5 7

Comparing Forecasting in High- and Low-Performing Firms 16 Forecasting Effectiveness, Accuracy Correlates with Firm Sales Objective Achievement 16 Forecasting Program Characteristics of High and Low Performing Firms 18 Forecasting Frequency Correlates with Firm Performance 18 Evaluating Forecasting Capability, Success Factors 20 Elements Important to Forecasting Success 20 Rating Forecasting Effectiveness 20 Forecasting Capability Gaps 20 Respondent Descriptive Information 23 Firm Size 23 Sales Force Size, Structure, and Management Span of Control 24 Respondents’ Functional Job Role 25 Respondents’ Role in Forecasting 26 Firm Sales Objective Achievement 28 Appendix 29

Copyright © 2015 the Sales Management Association. All rights reserved. 3



1

About the Research

1.1 Executive Summary

Effective forecasting matters to sales organizations. Forecasts influence resource allocation, align selling activities with other functions in the firm, and help managers anticipate market changes. Inaccurate forecasts’ negative effects are therefore magnified for sales management and the firm. This research surveys business-to-business firms’ sales forecasting practices, quantifies forecasting accuracy, and identifies factors related to forecasting effectiveness.

Our research finds most firms unenthusiastic about their forecasting accuracy or efficacy. Just 31% of firms consider their forecasting efforts effective. Whether justified or not by their ultimate value, forecasts nevertheless command a healthy fraction of the sales organization’s attention; most firms ask salespeople to forecast at least once monthly, and many require weekly forecasts. We surmise that most firms do so despite acknowledging a gap between their forecasting activities’ value and the costs it demands (chiefly in time and attention) from salespeople and management.

That said, forecasting effectiveness varies widely across respondent firms. We identified a minority which forecast much more effectively than their peers. And, we found a high correlation between high-performing sales organizations and those with effective forecasting efforts.

For high-performing firms, forecasting program characteristics are notably different. Compared to low-performing firms’ ratings, high-performing firms’ forecasting efforts make much more productive use of technology, reflect a continuous improvement approach,


1 About the Research

incorporate better data accuracy, and hold both salespeople and sales managers more accountable.

1.2 The survey’s 92 participating firms field an aggregate sales force of 25,900 sales managers and 219,000 salespeople.

1.2 Research approach

This study aggregates participating firms’ responses to a web-based survey. The Sales Management Association developed the survey and recruited participants from our membership and broader audience of sales managers and sales operations professionals. In exchange for participating, we offer respondents advance copies of the detailed study report.

Before reporting results, we eliminate invalid or ineligible responses, and sometimes contact respondents to clarify their responses. Survey results are only reported in aggregate, and never in a way that would compromise the identity of any single respondent. All individual respondent data are treated with strict confidentiality.

1.3 Research Timing and Scope

This research represents summarized data from 92 participating firms, directly employing more than 219,000 sales professionals, and 25,900 sales managers. Data was collected from August 2014 to January 2015. Respondent demographics and descriptive information is summarized in report section 6 below.

1.4 Research Underwriter

This study was made possible in part through the underwriting support of Vantage Point Performance. Sales Management Association underwriters provide annual financial support to the Sales Management Association. Underwriters may suggest research topics, participate in ongoing research projects, and encourage participation or otherwise promote research initiatives.

Underwriters are not involved with research administration, data collection, analysis, interpretation, or report development, unless explicitly noted in the report. Also, unless noted, underwriters do not pay a research-specific fee or directly commission research initiatives.

The Sales Management Association is grateful for the support underwriters provide to our research efforts.



1 Adjusting for sales force size weights each firm’s response in proportion to its share of total salespeople in all responding firms.

2

Forecasting Practices

2.1 Forecast Frequency

Thirty-eight percent of firms require salespeople to forecast once per month; an additional 31% require forecasts monthly. Quarterly forecasting is required by 11% of firms. Only six percent of firms require salespeople to forecast on basis less frequent than quarterly.

2.1 Most firms require salespeople to forecast weekly or monthly.

Salespeople in larger firms are more likely to forecast weekly. Adjusted for sales force size1, 74% of the study’s salespeople are required to forecast weekly; another 21% are required to forecast monthly.

Respondents’ salespeople and sales managers are asked to forecast with approximately the same frequency (as shown in figures 7.1, 7.2).


2 Forecasting Practices

2.1.1 Salespeople in larger firms are more likely to forecast weekly.

2.2 Performance Measures Included in Forecasting

More salespeople are required to forecast sales revenue than any other performance measure.

2.2 About two-thirds of all firms require salespeople to forecast revenue, making it the most prevalent forecast performance measure.



2.2.1 Sales revenue is the most frequently required performance measure in forecasts.

Other frequently-required measures are pending opportunity status, overall pipeline value, percentage sales goal achievement, and product sales. Market share, profit performance, and customer segment performance are least likely to appear in salespeople’s forecasts.

We found that two-thirds of all firms require all salespeople to forecast sales revenue. The next most-frequently required measure is opportunity status (51%), total pipeline value (42%), and percentage sales goal achievement (39%). Less frequently required are market share (3%), profit (6%), and customer segment performance (17%).


3 Evaluating Forecasting Effectiveness

3

Evaluating Forecasting Effectiveness

3.1 Defining Forecasting Effectiveness

Our study asked respondents to rate forecast efficacy in two ways: through individual accuracy ratings for a collection of specific performance measures, and in a rating of overall effectiveness of firm forecasting efforts. The first ratings address forecasting’s predictive value – that is, its accuracy in predicting future performance. The second rating, of overall effectiveness, attempts to capture a broader and more nuanced assessment of forecasting’s value.

One aspect of this broader value is the context that forecasting provides for real-time opportunity management. For example, managers and salespeople often routinely meet to review pending sales opportunities, allocate resources, and prioritize activities. Many refer to this activity as “pipeline management,” though it is also frequently considered forecasting activity. Importantly, though these meetings might include inaccurate forecasts, they nevertheless provide significant value as a means of redirecting selling effort and optimizing performance.

This distinction, between forecasting’s predictive value, and its intrinsic value as a means for guiding sales activities, isn’t drawn in fine detail in this study.

Ultimately, management should gauge their own forecasting effectiveness in a context broad enough to accommodate both sorts of impact.



3.2 About 40% of firms consider their forecasting “Somewhat effective” (the median rating of 4 on a seven-point effectiveness scale). Twenty-eight percent believe forecasting in their firm is ineffective (ratings of 1, 2, or 3).

3.2 Firm Forecasting Effectiveness is Low

Just 31% of firms consider their forecasting efforts effective. Forty-one percent rate forecasting “somewhat effective” in their firms, and 28% consider forecasting “not effective.” Just 14% of firms rate their forecasting efforts in the top categories of effectiveness (6, or 7 on a seven-point scale).

3.3 Forecasting is More Difficult for Larger Firms

Forecasting effectiveness is more difficult for larger firms. Firms over US$10 billion in annual sales rated their forecasting effectiveness 15% less effective than firms under US$100 million in annual revenue.



3.3 Larger firms rate their forecasting efforts as less effective.

3.4 Rating Effectiveness For Individual Program Elements

Respondents rated their firms’ effectiveness in a number of forecasting program elements. We measured these ratings by asking respondents to what degree they agreed that descriptions of specific program characteristics accurately described their firms’ forecasting program. Though many of these might be considered fundamental components of even a rudimentary forecasting effort, their aggregate ratings were surprisingly low.

Manager accountability received the highest rating of forecasting program components, with an average rating of 4.9 on a seven-point agreement scale (where 7 reflects complete agreement and 1 is complete disagreement). It was followed by accurate data (4.3), clear expectations (4.3), accountable salespeople (4.3), and enabling technology (4.1). A continuous improvement approach to forecasting garnered a rating of 4, and all other components were below the media rating of 4. The lowest of these is salesperson training, at 3.1.



3.4 Manager accountability received the highest rating of forecasting program components, salesperson training the lowest.

3.5 Prioritizing the Impact of Forecasting Program Elements on Overall Forecasting Effectiveness

We compared firms’ performance ratings for each program element with their overall rating for forecasting effectiveness. We found that improvement in almost any program element correlates with an improvement in overall forecasting effectiveness.

Effectiveness rating elements were based on the seven-point agreement scale described in 3.4 for each program element. We isolated respondents into three categories for each element – those which rated their firms as “not effective” for that element (a rating of 1, 2, or 3), “somewhat effective” (a rating of 4), and “effective” (a rating of 5, 6, or 7). We then compared how each of the three segments rated their firms’ overall forecasting effectiveness.

With a single exception (“providing clear definitions”), we found that improving from “not effective” to “somewhat effective,” or from “somewhat effective” to “effective” in



any single element resulted in an improvement in overall forecasting efficacy.

One program element, “published forecasting guidelines,” showed a profound impact in overall forecasting efficacy (30% improvement) by improving from “not effective” to “somewhat effective,” but then reflected a negative impact (-11%) for further improvement (from “somewhat effective” to “effective”). This suggests firms reap the greatest benefit from simply meeting a modest standard for published guidelines, but that there are diminishing returns in further improvement.

3.4 Improvement in every individual program element resulted in an overall improvement in forecasting effectiveness.

Improving the clarity of forecasting definitions from “not effective” to “somewhat effective” actually diminishes overall forecasting efficacy (suggesting no definitions are favorable to “somewhat” clear ones). Firms that were “effective” at providing clear definitions, however, saw a



20% increase in forecasting effectiveness over those that were “not effective.”

The greatest improvement to overall forecasting effectiveness was found in firms that are “effective” in sales manager training (35% improvement in forecasting effectiveness), an improvement that outperforms even the advantages of accurate data (which resulted in 34% improvement in forecasting effectiveness). Other notable program elements that contribute to forecasting effectiveness are communicating clear expectations (33% improvement for those firms effective in doing so), training salespeople (32% improvement), and using continuous improvement approaches for managing forecasting (31% improvement).

Least impactful, though still reflecting a significant positive benefit, are holding sales people accountable (14% forecasting effectiveness improvement), providing published guidelines (19% improvement), and holding managers accountable (20% improvement).

3.6 Forecast Accuracy by Performance Measure

Respondents consider opportunity status the most accurate performance metric, in comparison to other forecasted metrics, over a three-month forecasting horizon. Over 12 months, sales revenue is most accurately forecasted.

In absolute terms, accuracy ratings low-to-moderate, however. Sales revenue received a 12-month accuracy rating of 4.5 on a seven-point scale (where 1 is “not at all accurate,” 4 is “somewhat accurate,” and 7 is “extremely accurate”). It was followed by opportunity status (4.5), percentage of sales goal achieved (4.4), and number of new accounts (4.0). No other performance metric was rated above the median value of 4. Considered least accurate were 12-month forecasts of market share, customer segment performance, account performance, and units or volume. 15

3.7 Number of Required Performance Measures’ Impact on Forecasting Effectiveness

We compared firms’ overall ratings of forecasting effectiveness with the total number of performance measures they required salespeople to forecast. Firm that require salespeople to include five or six performance



measures had the highest level of overall forecasting effectiveness (4.8 on a seven-point scale, where 7 is the most effective). Firms requiring seven or more performance measures were second-most effective in forecasting (a rating of 4.1), and firms requiring three or four performance measures were least-well performing (3.2).

3.6 Opportunity status is considered the most accurate three-month forecasting metric; sales revenue the most accurate 12-month metric.

Performance measures include any of the 13 shown in figure 2.2; Note that forecasting using a formal methodology counts as a performance measure, and sales revenue and sales quota achievement are two separate performance measures (our separation of these elements into separate measures may explain a higher than expected number of required measures for some readers).



3.7 Firms requiring five or six performance measures in forecasts (from the list in fig. 2.2) had the highest overall forecasting effectiveness rating.

4

Comparing Forecasting in High and Low Performing Firms

We segmented survey respondents into two groups according to achievement of firm sales revenue objectives. Our “high-performing” group met or exceeded the firm sales objective, “low-performing” firms did not achieve the firm objective.

4.1 Forecasting Effectiveness, Accuracy Correlates with Firm Sales Objective Achievement

Forecasting effectiveness ratings of firms achieving their annual sales objective are more than twice as high (139%) as their low-performing firm peers. These ratings reflect respondents’ opinion of their firms’ overall forecasting effort.

High performing firms also forecast individual performance metrics with greater accuracy. Their ratings of forecast accuracy were 77% better than low-performing firms’ for market share and customer segment performance, 74% better for units or volume, and 58% better for sales revenue. High-performing firms enjoy a


4 Comparing Forecasting in High- and Low-Performing Firms

forecast accuracy improvement of at least 31% in all performance metrics.

4.1 We found correlation between firms’ forecasting effectiveness and their achievement of the annual revenue objective.

4.1.1 high-performing firms’ three-month forecasting accuracy was substantially better than their low-performing peers.’



4.1.2 high-performing firms’ 12-month forecasting accuracy was also substantially better than their low-performing peers.’

4.2 Forecasting Program Characteristics of High and Low Performing Firms

High-performing firms’ forecasting programs are notably different in other ways. Their use of enabling technology is dramatically more effective (153% more effective than lower performing firms’). They’re also more likely to employ continuous improvement principles to forecast management (85% higher effectiveness ratings than low-performing firms for this characteristic); and they are more likely to have accurate data (86% advantage over low-performing firms), and accountable salespeople and sales managers (60% higher-rated for both).


4 Comparing Forecasting in High- and Low-Performing Firms

4.2 high-performing firms’ forecasting programs were markedly better-supported and managed.

4.3 Forecasting Frequency Correlates with Firm Performance

When responses are adjusted for sales force size, higher-achieving firms (those which met firm sales objective in the preceding year) appear to require more frequent forecasts from salespeople. These firms require forecasts weekly, on average, compared with their lower-performing peer firms which require salesperson forecasts slightly less often (about three times per month, on average). Similarly, higher-performing firms’ average sales managers forecasting frequency is weekly, while low-performing firms average a twice-monthly frequency requirement.



4.3 high-performing firms are more likely to require weekly forecasting from salespeople.

5

Evaluating Forecasting Capability and Success Factors

5.1 Elements Important to Forecasting Success

One-on-one meetings are considered the most important manager activity associated with forecasting effectiveness. It’s followed closely in importance by accurate data, enabling technology, manager accountability, and sales manager training. Considered least important, in relation to these, are meetings with multiple salespeople, meetings with multiple managers, and published forecasting guidelines. It should be noted that even these lowest-rated elements were rated above the media rating value of 4 on a seven-point scale, suggesting that all elements we asked about are more than “somewhat” important.


5.2 Rating Forecasting Effectiveness

Respondents rate themselves as least effective in training salespeople (3.4 on a seven-point rating scale, where 7 is most effective), publishing forecasting guidelines (3.5), training sales managers (3.6), and communicating clearly-defined forecasting guidelines (3.7). Firms are most effective at holding managers accountable (4.4), providing enabling technology (4.4), conducting one-on-one meetings with salespeople (4.4), and providing accurate data (4.3). In absolute terms, these ratings 13

5.2 high-performing firms’ forecasting programs were markedly better-supported and managed.

5.3 Forecasting Capability Gaps

A forced ranking of importance and performance ratings for all reporting attributes, plotted below on an x-y scatterplot, shows the following elements rated above the average value for importance, and below the average rating for current firm performance: sales manager training, salesperson training, continuous improvement approach, clear definitions, and salesperson



accountability. Though many forecasting program elements are under-performing, this analysis suggests that these elements are a higher-priority issue for our respondents as a whole.

5.3 Respondent firms’ forecasting capabilities plotted in an importance-performance matrix.

Capability gaps were all negative; that is, ratings of firm performance were all lower than importance ratings for each attribute. Sales manager training had the largest capability gap, with a variance of -2.6 (the difference between an importance rating of 6.2, and the firm performance rating of 3.6, each on a seven-point scale, as shown in Figure 5.2).

Other capability gaps, in descending order of magnitude, are salesperson training (-2.6), clear definitions (-2.4), continuous improvement (-2.3), published guidelines (-2.2), and clear expectations (-2.1).


6 Respondent Descriptive Information

6

Respondent Descriptive Information

6.1 Firm Size

Ninety-four participating firms vary in size. Forty-three percent of firms have annual revenue in excess of US$100 million; 25% have annual revenue of US$1 billion or more; and 12% are firms with US$10 billion or more in annual revenue.

6.1 Respondent firm size



6.2 Total Sales Managers By Firm

6.2 Sales Force Size, Structure, and Management Span of Control

Respondents firms have an average of 279 sales managers, and 2,354 salespeople; sales managers have 8.4 direct-report salespeople on average.

Corresponding median values are 45 salespeople per firm, 6 managers per firm, and a salesperson-to-sales manager ratio of 6.7:1. 3, 4

On average, respondent firms have 2,354 salespeople, and a salesperson to direct-manager ratio of 8.4.

6.2.2 Sales managers by firm 6.2.3 Ratio of salespeople to sales managers



6.3 Respondent Job Role

6.3 Respondents’ Functional Job Role

Respondents are predominately sales operations leaders in their firms (35%). Thirty percent of respondents are first-line sales managers (i.e., they directly manage salespeople). An additional 22% are senior sales leaders, managing sales managers. Eight percent are in non-sales-related management positions, such as marketing, finance, or operations; 5% are none of the above, and include various functional positions from individual-contributor salesperson to C-Level executive.

6.4 Respondents’ Role in Forecasting

By firm, 73% of respondents have a high level of responsibility, or are chiefly accountable for reporting forecast results in their organizations. Another 11% have “significant” responsibility for reporting, and 16% have little or no responsibility. (We assigned these designations based on self-reported ratings using a seven-point scale, where 1 represents “no involvement,” 4 represents “significant” involvement, and 7 represents chief accountability. In our chart, respondents with a rating of 1 to 3 are shown in the “Little or None” category, respondents with a rating of 4 are shown in the “Significant” category, and those with a rating of 5 to 7 are shown in the “High-Level to Chiefly Accountable” category.)



6.4 Respondents’ Role in Forecasting

By firm, 73% of respondents have a high level of responsibility, or are chiefly accountable for reporting forecast results in their organizations. Another 11% have “significant” responsibility for reporting, and 16% have little or no responsibility. (We assigned these designations based on self-reported ratings using a seven-point scale, where 1 represents “no involvement,” 4 represents “significant” involvement, and 7 represents chief accountability. In our chart, respondents with a rating of 1 to 3 are shown in the “Little or None” category, respondents with a rating of 4 are shown in the “Significant” category, and those with a rating of 5 to 7 are shown in the “High-Level to Chiefly Accountable” category.)



6.4.1 Respondents’ Role in Forecasting

6.4.2 Respondents’ Personal Involvement with Forecasting



6.4.3 Respondents’ Personal Involvement with Forecasting (Adjusted for Sales Force Size)

6.5 Firm Sales Objective Achievement

Sixty-eight percent of respondent organizations met or exceeded firm sales objectives in the preceding 12 months. Respondents were asked to rate their firm’s achievement of the firm sales objective based on a seven-point scale (“1” for far underachieved objective; “4” for met objective; “7” for far exceeded objective). We use this performance rating approach in order to normalize company performance across large and small firms, and high and moderate growth sectors. Fourteen percent of respondents rated sales objective achievement in the highest two categories (“6” or “7”).

6.5 Firm Sales Objective Achievement


7

Appendix

7.1 Required forecast frequency by firm, sales managers.

7.2 Required forecast frequency, sales managers, Adjusted for Sales Force Size.


Documents

Research Brief Forecasting Effectiveness · PDF fileto-business firms’ sales forecasting practices, quantifies ... This study was made possible in part through the underwriting support