Upload
red
View
65
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Do NFL Coaches Coach to Win?. An examination of the relation between in-game decision making in the NFL and a coach’s job security By Mitch Levy. NFL Rules Background. What are two point conversion attempts? What are fourth down attempts?. Theoretical vs. Actual Strategy. - PowerPoint PPT Presentation
Citation preview
Do NFL Coaches Coach to Win?
An examination of the relation between in-game decision making in the NFL and a
coach’s job securityBy Mitch Levy
NFL Rules Background
• What are two point conversion attempts?
• What are fourth down attempts?
Theoretical vs. Actual Strategy
• Modeling suggests coaches should attempt these plays more often
• ~2.5 times as often for fourth down attempts*
• ~5 times as often for two point conversions*
*From model by footballcommentary.com
Theories about the Discrepancy
• Modeling error
• Loss aversion/delaying outcome
• Lack of knowledge
• Dislike of criticism
Hypotheses
• Coaches are aware, that their decisions are suboptimally conventional (but not necessarily the degree)
• Non-expected utility theory• Dislike of criticism plays a meaningful role• Stronger incentives to win produce better
decisions
Idea of Study
• Evaluate coaches with different degrees of job security and see how their decisions change
• Hypothesis: coaches with their jobs on their lines will endure more discomfort to win more
Details of Study
• Coaches of teams that made the playoffs the previous year will be deemed safe – all other coaches are unsafe
• Evaluate conformity to suggestions of Markov model for “safe” vs. “unsafe” coaches
Some Possible Objections
• Playoff teams may have better conversion odds
• Playoff teams may have better coaches
• Playoff teams may have older, more conventional coaches
• Non-playoff teams may be worse, and pursue a high-variance strategy
Data Sources
• NFL Plays, 2002-2012 (Courtesy of Brian Burke, advancednflstats.com)
• Suggestions by model: nflcommentary.com, run by (NAME HERE)
• Model is a Markov Chain analysis
Two Point Attempts: Model Info
• Input variables: time remaining, score discrepancy
• Output variable: probability of success that justifies attempting the two point conversion, rather than kicking an extra point
All Two Point Attempts: Summary
• Total Two Point Attempts: 622
• Total Two Point Conversions: 319
• Overall Success Rate: 51.28%
• This rate used to determine which conversions should be attempted
2 Pt. Play, Safe vs. Unsafe Coaches
• Safe Coaches:– 159 Attempts, 720 Non-Attempts– Correct attempt rate: 18.1%
• Unsafe Coaches:– 331 Attempts, 1212 Non-Attempts– Correct attempt rate: 21.5%
“The Belichick Effect”
• Bill Belichick is an outlier, both in appearing in the playoffs 10/11 times in the sample (no other coach appears more than 4 times) and in his aggressiveness.
• The numbers, controlling for Belichick, are even more compelling
2 Pt. Play, Safe vs. Unsafe Coaches, Controlling for the Belichick effect
• Safe Coaches:– 141 Attempts, 688 Non-Attempts– Correct attempt rate: 17.0%
• Unsafe Coaches:– 328 Attempts, 1205 Non-Attempts– Correct attempt rate: 21.4%
Statistical Significance
• Without Belichick Effect?
• With Belichick Effect?
Fourth Down Attempts: Model Info
• Input variables: time remaining, score discrepancy, position on the field
• Output variable: probability of success that justifies attempting the fourth down play, rather than punting or kicking a field goal
Fourth Down Attempts• Conversion success on 4th and ...• 1 : 0.664885088188, 2 : 0.584415584416, 3 :
0.555831265509• 4 : 0.544668587896, 5 : 0.493710691824, 6 :
0.527659574468• 7 : 0.492462311558, 8 : 0.436090225564, 9 :
0.485148514851• 10 : 0.474137931034, 11 : 0.511904761905, 12 :
0.426666666667• 13 : 0.338461538462, 14 : 0.388888888889, 15 :
0.446153846154
Determining whether a coach “should” attempt a 4th down play
• If 4th and 1, use determined value of 0.665
• Otherwise, use 0.61 – 0.015*yds_to_go
4th Down, Safe vs. Unsafe Coaches
• Safe Coaches:– 910 Attempts, 1437 Non-Attempts– Correct attempt rate: 38.8%
• Unsafe Coaches:– 1768 Attempts, 2708 Non-Attempts– Correct attempt rate: 39.5%
4th Down, Safe vs. Unsafe Coaches (Controlling for the Belichick effect)
• Safe Coaches:– 859 Attempts, 1377 Non-Attempts– Correct attempt rate: 38.4%
• Unsafe Coaches:– 1755 Attempts, 2692 Non-Attempts– Correct attempt rate: 39.5%
Statistical Significance
• Without Belichick Effect?
• With Belichick Effect?
Implications
Controlling for Offensive Strength
• Calculate separate conversion probabilities for playoff, non-playoff teams
• Same methodology: separate 4th and 1 probability, linear regression on 4th and 2 through 4th and 15
Controlling for Offensive Strengthon 4th Down Attempts
• Safe Coaches:– 882 Attempts, 1429 Non-Attempts– Correct attempt rate: 38.1%
• Unsafe Coaches:– 1755 Attempts, 2692 Non-Attempts– Correct attempt rate: 39.5%
Model