It is hard not to laugh at how conservative NFL coaches are in their decisions. Even to the casual observer, it is clear that they have much more fear of a bad thing happening than they do hope for a good thing. Anyone who watches a game with me will hear the words "effing idiotic wimp" whenever a coach chooses to punt from the opponents 37 on 4th and 2.
It turns out that my gut reaction is also statistically correct. And, it is even more correct if you are discussing this decision at the college or high school level, where the results of a punt or field goal attempt are less sure than in the NFL.
There is a short article in Sport Illustrated this week called "Just Go For It" about a high school coach in Arkansas named Kevin Kelley who never punts or tries field goals, and also tries and on-sides kick on every kick off. His reason? He has done the numbers and determined that the potential gains from going for it are greater than the potential losses of failing. He's done the numbers and calculated the probabilities. Oh, and he wins a lot more games than he losses.
Meahwhile, over at the NY Times, stats geek Brain Murray has done his own study, and concluded that NFL coaches are far more conservative than probability models would suggest they should be. This is the same conclusion University of California economist David Romer came to a few years ago when he studied the same thing. You can read his whole paper here if you are so inclined.
So, why are these otherwise macho guys such wimps when it comes to risk? I would agree with what others have postulated: that coaches do what everyone else does so they don't get fired if a riskier decision goes wrong. It is also likely that most of them have never done the math, and that is they took a stats class in college they might have slept through it.
I read something about this a few years ago, and it turns out the NFL coach who goes for it most often on 4th down is Bill Belichick. Why? Maybe it's because his job is certainly very secure, or perhaps it has something to do with the fact that he has a degree in Economics.