Friday, April 27, 2007

The "defense first" strategy in college football OT -- part 2

In a post a couple of days ago, I noted a study that found a strong advantage to choosing "defense first" in college football OT. The first couple of paragraphs of that previous post describe the overtime rule ... you can go back and read them if you're not familiar with it.

That study found that the team that goes on offense last in the first OT (which I'll call "the second team" or "team 2") beat the team that went on offense first ("the first team" or "team 1") 54.9% of the time.

With some of the numbers listed in the original study, and some additional assumptions, we can try to figure out a *theoretical* probability with which the second team will win, and compare that to the observed 54.9%. For this theoretical calculation, I'm assuming the two teams are equal in skill.

The original study gave the distribution of first team scoring. I've combined 6- and 7-point touchdowns to keep things simple (which won't affect the results much):

.235 – team 1 scores 0 points
.299 – team 1 scores FG
.466 – team 1 scores TD

What is the distribution of second team scoring? It depends what the first team does, and we have to guess a bit.

Suppose the first team scores a touchdown. Then, the second team never goes for a field goal. So it's in what otherwise would be a field goal situation .299 of the time, but will have to go for a touchdown anyway. Suppose from fourth-and-something, they will score a touchdown 50% of the time, and score 0 points 50% of the time. In those cases, that would change their distribution to:

.385 – team 2 scores 0 after team 1 TD
.000 – team 2 FG after team 1 TD
.615 – team 2 TD after team 1 TD

Now, suppose the first team scored a field goal. We'll assume the second team plays exactly the same way as the first team:

.235 – team 2 scores 0 after team 1 FG
.299 – team 2 FG after team 1 FG
.466 – team 2 TD after team 1 FG

Finally, suppose the first team scored zero. It had a 76.5% chance to score, but failed. The second team must have a greater than 76.5% chance to score, because it's going to go for a field goal in some cases where the first team might have chosen to go for a touchdown (and fumbled or something). Let's call it 80%.

.200 – team 2 scores 0 after team 1 scores 0
.800 – team 2 FG or TD after team 1 scores 0

The chances of the first team winning in the first OT is the sum of these probabilities:

.180 – team 1 TD, team 2 zero (.466 * .385)
.070 – team 1 FG, team 2 zero (.299 * .235)
.000 – team 1 TD, team 2 FG (never)
------------------------------------------
.250 – Total chance team 1 wins in this OT

The chances of the second team winning in the first OT is this sum:

.188 – team 1 zero, team 2 TD/FG (.235 * .800)
.139 – team 1 FG, team 2 TD (.299 * .466)
------------------------------------------
.327 – Total chance team 2 wins in this OT

And the chance of a tie is 1 minus the above two totals, which works out to

.423 – Total chance of this OT ending in a tie

Now, the chance of the (original) second team winning the game, is this sum:

Chance of winning in the first OT + (Chance the first OT is a tie * chance of winning the second OT) + (Chance the first two OTs are ties * chance of winning the third OT) + ...

Also, if a given OT ends in a tie, the first team has to go second this period, and the second team has to go first. So the probabilities are switched in the even-numbered OTs. Therefore, the above sum works out to:

.327 + (.423 * .250) + (.423^2 * .327) + (.423^3 * .250) + ...

The sum of that infinite series (which is actually two intertwined geometric series) works out to .526.

Under the assumptions listed above, the chance of the "defense first" team beating the other, equally matched team in OT is .526.

But as we saw in the other study, the actual chance was .549. Why is our estimate different?

One possibility is that we're assuming evenly-matched teams. In real life, college games are often mismatches. However, mismatches should minimize the effect. The more mismatched the teams, the less likely winning the coin toss should affect the result (if a team is good enough, it'll keep scoring TDs and win even if it has to go first). So the real life number for equally matched teams should be higher than .549, not lower.

So what's going on? Why do these estimates not match the observed results? One of three possibilities:

1. My calculations and logic are wrong;
2. The assumptions are wrong;
3. Favored teams won a lot of coin tosses just by luck.

I'm assuming it's not luck. Any suggestions?

Labels:

At Monday, April 30, 2007 11:09:00 AM,  Jason Lisk said...

I think your assumptions are incomplete. I am assuming the study did not provide a breakdown on how teams score 0 points .235 of the time in the first overtime?

Teams are not in FG situations .299 of the time, they MAKE a field goal on .299 of the possessions, which is where I think your analysis broke down.

Remember that teams are in field goal range immediately at the start of the overtime (25 yard line). Sure, there are some interceptions or fumbles, but I would assume that the majority of the .235 chance of scoring 0 is due to a missed field goal on fourth down.

Let's assume it breaks down like this:

.075 team 1 scores 0 points due to turnover/turnover on downs
.160 team 1 scores 0 points due to missed field goal
.299 team 1 scores FG
.466 team 1 scores TD

I dont think this is unreasonable, and may be conservative, to assume college field goal kickers make a little less than 2/3 of kicks between 30-40 yards. College kickers are not nearly as consistent as pro kickers.

Here, team 2, facing a team 1 that scored a td, is now going for it the .459 that it would have attempted a FG, not just the .299 that it would have made the FG.

If we assume a 50% chance of scoring a td on those times the team would have otherwise attempted a FG, the chance that team 2 scores a td to force second overtime is actually almost 0.70, not .615.

At Monday, April 30, 2007 11:13:00 AM,  Phil Birnbaum said...

Jason: You're right ... that makes sense, thanks. I'll rerun the numbers.

At Monday, April 30, 2007 11:20:00 AM,  Phil Birnbaum said...

Okay, under Jason's updated assumption, which I think is much improved over the one I orignally made, the winning percentage rises to .540.

Still not as high as observed, but substantially better.

At Monday, April 30, 2007 1:38:00 PM,  Jason Lisk said...

Okay, so in your assumptions, you assume that teams employing a touchdown only strategy will score a td on 50% of the possessions they would have otherwise attempted a field goal.

I don't think that is an unreasonable estimate, because (1) looking at NFL rates, that would be equivalent to a team getting to about 4th and 3 or 4; and (2) a team employing a no field goal strategy might actually employ other strategies to shorten the fourth down attempt (e.g., be more willing to run draw on 3rd and 7, or be more aggressive on first down, when a team cannot play pass only or run only).

Though I am skeptical, it could be 40% on average (I dont see it being any lower than that). It could be 45%.

How much better would a team 1 strategy of "touchdown only when going first" be, under the assumed 50% success percentage? And how low would that success rate have to drop before it becomes a reasonable strategy to settle for a field goal when going first?

My guess: teams should be thinking like the other team has already scored a td when going first, and employ a td-only philosophy, except for rare exceptions. If teams employed such a strategy, they would reduce the benefit that going second creates.

At Monday, April 30, 2007 1:46:00 PM,  Phil Birnbaum said...

I can't really defend my estimates, because I kind of picked them out of the air ... if you're thinking 40% or 45% instead of 50%, I'm pretty much ready to trust you completely. That reduction would lower the second team's winning percentage by a few points.

As for the idea that the first team doesn't play for the touchdown enough (going for the FG on fourth down when that might not be the best strategy), I'm inclined to think you may be right, especially considering the evidence that NFL coaches also play too conservatively in those situations.

At Monday, April 30, 2007 7:21:00 PM,  Coach K said...

In a past lifetime I coached college football. Once I became a defensive coordinator, I argued in preseason staff meetings that if you are the team on offense in the bottom of the inning and you are down by 7, when you score you should go for 2.

i.e.
Florida and Miami go into overtime tied 10-10.
Florida scores to go up 17-10.
In the bottom of the inning, Miami scores to make it 17-16 with the extra point pending.
I argue Miami should go for 2.

Rationale:
You will be on offense in the top of the next inning (as your research indicates) giving you only a 45-46% chance of winning.
All you have to do is make the 2-point play about 43-44% of the time to make it a good play! (Assuming you'll miss the kicking extra point 1-2% of the time - this might actually be more because of "pressure")

I don't have the patience/time to do the research. If anyone does, I'd love to get an equilibrium point to share with the Head Coach who told me 1) I was crazy 2) He didn't have the "balls" to do it 3) If we didn't get it, I'll put a for sale sign in front of your house.

After #3 I decided not to pursue the issue any longer!

At Tuesday, May 01, 2007 3:01:00 PM,  Sam said...

I think you've underestimated the likelihood that team two will win when team one fails to score in 1OT.

Under your assumptions team two will win 80% of the time, which will almost always be done by kicking a FG. If you look at the breakdown in Jason Lisk's first comment he assumes team one misses their field goal 16% of the time, or an 84% success rate. If you give team two the same rate of success your original calculation changes from 0.526 to ~0.535. And granted, these numbers are all guesses but the likelihood of team two making their FG should be higher than that of team one, because in that situation teams will often kick on first down, to prevent the possiblity of a turnover. By kicking on first down you also guarantee that your kicker doesn't have to try from further than 25 yards. In the set of team one 1OT field goal attempts, there is bound to be a non-trivial amount of examples where team one actually lost yardage and has to kick from behind the 25 yard line, increasing the chance that they will miss, or that the kick will be blocked. So it wouldn't be unreasonable to bump the probability of team two winning to .86 or even .88 in the event that team one fails to score in 1OT.

At Wednesday, May 02, 2007 8:35:00 AM,  Phil Birnbaum said...

Sam, I'm not sure how the miss probability figures into the calculation. Won't the second team miss just as many as the first team?

In fact, shouldn't it miss even more? For team 1, some of its tries came from close in. For team 2, none will, because if team 1 didn't score, team 2 will try for the kick earlier (and farther) than team 1 did.

At Wednesday, May 02, 2007 12:57:00 PM,  Sam said...

Well, I looked at the set of OT games from the last two seasons where the first team on offense failed to score. In the 14 games where this happened, the team that went second won 13 times. 6 times by TD, 6 Times by FG, and 1 time by playing more overtime rounds. One of the two instances where team two failed to score on its first attempt, resulted in the only team two loss of the sample. Also my assumption that a team would kick on first down was shown to be incorrect. There was, however, one instance of a team kicking on 2nd down and two where they kicked on 3rd down. Even though 14 games is a very small sample it suggests that team two will win 90+% of the times when team one fails to score on their first attempt. In my previous comment I based my argument around the assumption that in this situation, team two will almost always try for a field goal to win the game. This turned out to be completely false, but the original premise was still correct.

At Wednesday, May 02, 2007 6:03:00 PM,  Jason Lisk said...

sam,

I wasn't saying that field goal kickers only miss on 16% of their kicks in overtime. I was assuming that 16% of the "team 1" possessions ended in a missed field goal, which is a vastly different thing. Using my assumption, kickers made .299/.459 = 65.1% of their kicks.

Now, I am willing to concede that assumption may be too high on percentage of misses. What we do know is that .235 ended with 0 points. This .235 consists of a combination of possessions ending in turnovers, possessions ending due to failed 4th down attempts, and possessions ending due to missed field goals.

For comparison, according to drive stats on Football Outsiders, the average team commits a turnover on 14% of possessions. However, (1) this includes all possessions, including those going beyond 25 yards, and (2) it includes "high turnover" possessions when a team is trailing by multiple scores and passing almost exclusively, which is not the case in a tie overtime game. Mitigating the other way, I do think college offenses turn the ball over more than pro offenses, other things being equal.

In order for kickers to be making 84% of their kicks in overtime, only .053 of the team 1 possessions would end in a missed fg, while .182 would end in a turnover or turnover on downs, which seems too high to me.

I guess that is a long way of saying, I dont think the 80% assumption is unreasonable. Some percentage of team 2's possession will end in turnovers, even if they adopt conservative strategies to reduce the risk. To reduce turnover rate, you also reduce the likelihood of a td, thus increasing field goal opps. You also, as Phil points out, would have longer than average fg's if you were that conservative to start the drive.

Your sample of 14 is not sufficient to disprove the 80%. But I would point out that 12 of 14 team 2's won in that overtime period (85.7%), not 13 of 14. The 80% figure is not the chances of team 1 losing the game when it scores 0 in the first possession, it is the chance that
team 1 loses the game during that overtime period after scoring 0.

As to Coach K's comments, I think it is a wash. The breakeven point is .46 (team 1 chance of winning if going to next ot as offense first) * .98 (chance of xp conversion) = .451. That is roughly equal to the 2 pt conversion rate. Either decision is defensible, depending on the specific game situations. Some factors that might justify going for 2 are:

You are the underdog
Your opponent's offense is better than their defense
You have a below average kicker
The opponent has an above average kicker

Now, the best strategy in your scenario would be to kick the xp, but then adopt a "must td" strategy. I ran the numbers, and it looks like team 1 could improve the chances to 49.1% by acting as if they must score a td when going first, if we assume 50% chance of converting possessions that would otherwise be fg to td. It is 48.3% if we assume only a 40% chance, which is still a substantial improvement over the current odds of playing it conservative and taking a fg on 4th.

At Monday, April 20, 2009 3:57:00 AM,  cvxv said...

At Tuesday, May 05, 2009 2:55:00 AM,  Kevin said...