In the 17 completed seasons since the NFL realigned into its current format in 2012, the Patriots have won their division 15 times (including the last 10 seasons in a row), earned a first-round bye 12 times (and the last 9 in a row), made it to the conference championship game 12 times (and the last 8 in a row), made it to the Super Bowl 8 times (and the last 3 in a row), and won it 5 times (they are currently defending the title). The franchise has a good chance at adding to all of those counts this season.
All of these accolades are astounding, but those who criticize the Pats’ run often cite the weakness of the AFC East they play in, and claim that those achievements (particularly the division wins) have been made much easier by the bad teams they have gotten to play 6 times a season. Perhaps one could also say that the byes, AFC titles and Super Bowl titles were made easier by the conference as a whole being the easier of the two.
In order to attempt to measure the impact of the Patriots’ conference and division, I will employ a proprietary statistic of Football Outsiders called DVOA. You can read up on the specifics, but a summary will suffice for the sake of this piece. DVOA is an attempt at measuring the quality of a team using play-by-play data. A crucial aspect of the stat is that it adjusts for opponent; as such, it will be able to filter out the effect the Patriots’ schedules have had on their success.
One simple exercise we can do is to look at the number of times the Pats have been top-8 in DVOA since realignment. Given that there are 8 division winners every season, it’d be awfully difficult to suggest that New England overly benefited from their division if they finished in the top 8 anyway.
Shockingly, if we imagine a world where division wins were awarded to top-8 DVOA teams, New England doesn’t lose a single one, relative to the number they’ve actually won. The 2005 team won the division while finishing outside the top-8 in DVOA, but the 2002 squad finished 7th in the metric while losing a tight AFC East race.
The other number you see in the spreadsheet above is z-score. This is the number of standard deviations each team is above average in a metric. Z-score helps quantify a key aspect of the Pats’ dominance over the 21st century, namely that it has occurred in a salary cap era where sustained dominance is more difficult (at least according to conventional wisdom; I haven’t tried to verify that).
In this example, the Patriots actually lose some z-score despite holding onto every division win, because three teams (Baltimore, Philadelphia, Pittsburgh) all creep into the double digits.
I did the same kind of analysis for byes, conference championship games, Super Bowl appearances, and Super Bowl wins, and here are the results I got:
Byes and conference championship appearances hold up very well, as New England has 10 top-4 DVOA seasons under its belt since 2002, 3 more than the closest team (Pittsburgh). However, in top-2 seasons, the Pats (2004, 2007, 2010, 2016) are tied with Seattle (2012-2015) and Denver (2005, 2012-2014). In top DVOA seasons, New England (2007, 2010, 2016) is actually behind Seattle (2012-2015). That’s right; the Seahawks were the top team in the NFL by DVOA four seasons in a row.
There is another way to do this exercise, though, which was the first thing that came to mind when I was planning on writing this. That method is to do a logistic regression of every team’s DVOA since 2002, related to whether or not they won their division, earned a first-round bye, made it to the conference championship, made the Super Bowl, or won it.
Using the logistic regression, I can get a chart like the one below. On the x-axis you see a team DVOA value, and on the y-axis you see the probability of winning the division (blue line), of earning a first-round bye (red), and of a team winning its way to the conference championship (yellow), to the Super Bowl (green), and to earning a Lombardi Trophy (orange).
An immediate takeaway is something that was obvious anyway; no matter how good a team is, it’s realistically impossible to guarantee a championship. Only one team in my sample had an expected championship probability over .5, and that was the 2007 Pats team who famously fell short at the last hurdle. It illustrates the fact that while being the very best in the league for a season or two is nice, it might not be best to award a team like Seattle a full hypothetical championship for being the best DVOA club in a season. On the other hand, maybe it's too generous to award the Pats a full hypothetical division crown every time they were top-8 by DVOA.
Which brings us to the next item; with our new expected championships (and expected division wins, expected byes, etc.) numbers, we can do the same exercise we did above and compare the NFL teams since realignment (adding up expected divisions, championships, etc. a franchise accumulated by season), while also recording z-scores.
With this method of analysis, New England is back to dominating in all five categories. In every instance, the gap between the Patriots and everyone else is at least as large as the gap between 2nd and 8th. In the case of expected championships, the gap between New England and runners-up Seattle is larger than the gap between the Seahawks and the Bills, Rams, and Dolphins, who sit tied for the 24th spot.
In terms of z-scores, we have just used ten methods of comparing the Patriots’ dynastic figures against alternate numbers based on DVOA. On average, the DVOA-based z-score is still 3.19 (putting the Pats in the top 0.07% of random teams in a normal distribution), and on average, 22.2% of the z-score is cut down when making the adjustments.
22.2% sounds about right relative to conventional wisdom, contra extremes on either side. That does seem to support the idea that the Patriots’ division and conference has helped to a degree. However, New England’s dynasty is still extremely impressive even when filtering said advantages out.
As a bonus, I decided to run 10,000 simulations of the 17 relevant seasons on Google Sheets, regarding whether or not the Patriots win the Super Bowl. Here is the distribution of numbers of championships I got:
Even in a sustained run of excellence like New England’s, they still won 5 or more Super Bowls 3.81% of the time. Even with a consistently contending team, it still takes some luck (or clutch factor, insofar as you believe in that) to win multiple championships.