Paraag Marathe arrived in San Francisco with a very clear directive from the 49ers, to reimagine the Jimmy Johnson draft chart. And after a couple months work, the senior associate from the Boston consulting firm Bain readied to show team president Bill Walsh and GM Terry Donahue his findings, in disbelief at the results being spit back at him.
It was uncanny.
“I tried to use historical trends and true value,” says Marathe, now the Niners’ chief strategy officer and EVP of football operations. “And it wasn’t like Coach Walsh was telling me, ‘Hey, a third rounder has more value than it says here, and a second rounder has lower value.’ It was meant to be totally independent. And once I was finished, I looked at all of Coach Walsh’s trades over the years. And it was a total match with all of Coach Walsh’s trades.”
All that work, and all that was proven was what some guy on his couch down the street in Santa Clara could’ve told them: Walsh was a savant.
That was 2001. Moneyball—the best-selling book based on the baseball team across the Bay from the Niners, which would shed light on the coming numbers craze—was still two years from release. The idea of using advanced statistics to drive decision-making in baseball was still in a nascent stage, at least publicly. And that, of course, implies the truth about the NFL then, which is that few in football had even given that concept a thought.
It’s a different time now. Analytics in the NFL have moved well beyond the point where a team hiring a consulting firm to run numbers constitutes outside-the-box thinking. Yet, there remains resistance, a battleground of thought, and a general cloudiness on how far you can take numbers, and how far numbers can take you.
And just the same as 16 years ago, you can place a genius in the center of it.
Perception holds that the Patriots are among the league’s most progressive teams, but there’s precious little evidence of their investment. They have people who are responsible for advanced statistics, but coaches and scouts are largely charged with integrating data gathered into their work. The “analytics guy” there is 64-year-old Ernie Adams, a former Wall Street trader and prep school buddy of Bill Belichick’s.
So why are people so convinced that the five-time champions are knee-deep in the numbers? As one informed long-time NFL exec explains it, “It’s because they’re completely consistent with what sophisticated analytics would tell you to do.”
“[Belichick] does it with intuition,” says one AFC executive. “You know because you’ve been coaching for so long, how you match these 11 guys against those 11 guys. It all makes sense to you. At some point, maybe we can all come to those conclusions without having Bill Belichick’s brain. We’re still a long way from that.”
The NFL is getting closer. The MMQB spent a month discussing analytics with more than 40 team officials from across the league—coaches, executives, scouts and analytics people—and there are some hard conclusions that can be made on where the league is.
1. Most teams don’t shy away from analytics. In fact, more than one official was offended by the notion that their team would be called “old school”. When it comes to player acquisition (which is what Moneyball was based on), the average NFL team is using the data. It’s just that it is being used to generate boundaries rather than drive decisions. Teams want to know when they’re making exceptions on one player, and they want to know what they might be missing on another they may have otherwise dismissed.
2. On the coaching side, analytics are generally used to make staffs more efficient. There may only be time for quality-control coaches to break down four or five of an opponent’s games in the week they have leading into a particular game. And that, in the past, would lead to guesswork on tendencies and strengths and weaknesses. The data allows the quality-control guys, and staffs, to crosscheck against larger sample sizes.
3. The limits in those two areas are the number of games (16) and the variance in players’ assignments and situations that affect plays. That makes it more difficult to collect the amount of data necessary to make hard decisions.
4. Conversely, the value of the data in those areas is proven in that nearly the entire NFL has subscribed to Stats LLC and/or Pro Football Focus, and some rely on smaller services, like Pro Scout Inc., which is run by 64-year-old former Utah coach Mike Giddings.
5. There aren’t sure marks of analytics-friendly operations on game days (as there is in basketball, with teams that go for the “two-for-one” possession at the end of the quarter or half). But on the personnel side, you can see it in asset management, with teams that trade down in the draft to pad their margin for error, and use cap space creatively.
6. There is one strong consensus league-wide: Analytics data related to injury prevention, which straddles sports science and comes through player tracking, is useful and will only get better. The NFL is just scratching the surface with this technology, and the floodgates will truly open only when the league makes available all the Zebra data it’s been collecting. Another step here that’s expected to come eventually is in profiling the minds of players.
7. The Walsh/Belichick robot is not on the market. Yet.
• SMARTER FOOTBALL WEEK: A series examining the cerebral side of the sport, including technology, analytics, how a brainy linebacker prepares and just what goes into a typical NFL play.
* * *
The cliché is still going strong. On one side you have the gym teacher with the whistle around his neck, on the other it’s the dweeb with the taped glasses, pocket protector, and stacks of hard-to-decipher numbers and data. And, if you believe the cliché, it continues that football’s tough-guy culture has made it slower to change than other sports.
The truth is more complicated. Part of the problem with finding analytics’ place in football is the term itself. Often, the perceived hesitancy to embrace quantitative analysis in the NFL is due to the fact that what is often referred to as “analytics” was already a huge part of the game. It just didn’t have that name.
The game itself is rooted in tactics and strategy and details, and so the study of those has always been inherent in the coaching of the sport and building of its teams.
“I think most of the stuff we’ve done for a long time,” says one NFC general manager. “[Analytics] is the buzzword. That’s how everyone clumps it together now. Like, until five years ago, I’d never heard combine data called ‘analytics.’ But now, someone smart can make it look pretty—analytics. To me, analytics is the Pro Football Focus numbers, finding a way to put a numeric grade on every play and quantify it.
“No one has enough people on staff to do all of that. A lot of it is just statistics, and then what the smart computer guys can make it tell you.”
Photo: Carl Iwasaki/Sports Illustrated
Bill Walsh was ahead of his time on analytics, even if he didn’t know it.
There’s a lot of gray to cut through. Quality control coaches have forever done what are now considered game-week analytics—identifying opponents’ tendencies, finding trends, and setting up their bosses to game plan. On the personnel side, scouting assistants have, likewise, undertaken analytical studies for decades to uncover advantages and inefficiencies in the draft and free agency.
“Up until 15 years ago, football was probably the furthest along of any of the sports as far as studying the game itself,” says an analytics manager for one AFC team. “It just wasn’t guys from fancy schools doing regression. But there was real systematic study of the game. And frankly, basketball and baseball only started doing the kind of film study that football has done forever just recently.”
So here’s where you start: The reason there isn’t an NFL team ignoring analytics is because analytics has been done in football since Paul Brown came along.
Some teams have very little in the way of analytics staffing and, based on their business practices, are considered advanced. Others have entire departments in place and those who study analytics wonder, based on the way those teams operate, if all their number-crunching has influenced even a single decision.
How is that possible? Well, as Jaguars SVP of football technology and analytics Tony Khan—the son of owner Shahid Khan—explains it: “The adoption rate is far behind other sports.” More than three-quarters of NFL teams employ either a director of analytics or have a full-blown analytics department. Others have their cap managers oversee it.
The divide in the NFL comes in how the data collected is put to work.
“There are a lot of skeptics,” Khan says. “And that’s honestly probably on the analysts and the statisticians. You have to be able to explain it to the football people in their terms. That’s why I try to study that, to be able to communicate with coaches and scouts on a meaningful level, instead of forcing them to use our terminology.”
It’s easy to figure out where and why the divide happens. Football lacks the repetitive one-on-one situations that make up a large portion of baseball and basketball, and it’s tougher to figure out what each player’s assignment is without hearing it from a coach. Those assignments are more divergent, too—in basketball, a center’s role on defense can be affected by the opposing point guard’s movements and decisions and the spacing of all the opposing players; in football, a right tackle’s objective doesn’t relate much to what a free safety is deployed to do.
That makes putting values and creating apples-to-apples judgments on players difficult, and the idea of filtering gameday decisions through a strict set of rules problematic. It also makes it challenging for services like PFF and Stats LLC to find the right way to assess players and plays on their own, and generate value for coaches and scouts.
Then, there’s the issue of sample size.
“In baseball, you have 162 games, and the same exact play starts every play, and that’s the beauty of it,” says one AFC general manager. “You start to appreciate the sheer volume of the numbers, and how they compare over time. In basketball, there’s a natural progression—up and down, up and down. The event of the team going up the court happens so many times, you can chart shots, rebounds.
“And so much of it in both sports is pass/fail. You break that down with the amount of times the events repeat themselves, you can build big data.”
In football, on the other hand, you get anecdotes like this one: There is one team that has a club exec who heads up data collection. He’ll call his head coach into his office and hand the data to him. But the report never makes it to the coach’s desk, only out to the hallway where, between the two offices, there is a trashcan.
• WELCOME TO 1967 WEEK: Crazy players, an unforgettable expansion season and an ill-fated third-place game. A look at what the game, the players and the culture of professional football were like a half-century ago.
* * *
Some teams have analytics directors. Others have departments. Some integrate data and make football people responsible for working it into their jobs. Others keep the sides separate, with an over-the-top manager (usually the GM) sorting it out. Some build their own systems. Others have the numbers people there only as window dressing.
One commonality? Most subscribe to services. Cincinnati-based Pro Football Focus now lists 27 teams as clients, and founder Neil Hornsby says, “I’d be surprised if we haven’t got 32 of 32 by next season, if not this season.” Similarly, Stats LLC has 26 teams as subscribers. The reason? It’s a way to become more efficient and guard against missing anything, even if you’re still not sure how to use all the numbers.
“I come from a consumer product and enterprise software background,” says John Pollard, a former exec at Stats LLC who now works as a consultant. “And what struck me in the early 2000s is when more information and research and analytics became available in over-the-counter pharmaceuticals and retail, it was just too much information. And we’re going through the same thing in sports now.
“It’s being driven by the information. Technology provides the efficiency. And analytics provide more effectiveness in the decision-making. The goal isn’t to replace people. It’s to make them more effective and efficient.”
Tony Khan heads up the analytics operation for his father’s team in Jacksonville.
Photo: Bill Frakes/Sports Illustrated
Tony Khan heads up the analytics operation for his father’s team in Jacksonville.
On the personnel side, there are clear examples of where the numbers are leading to change. In 2009, Dolphins linebacker Joey Porter had nine sacks, but those sacks accounted for an unusually high percentage of his hurries. The Cardinals signed him, at 33, to a three-year, $24.5 million deal anyway. He regressed (6 sacks total over the next two seasons), as the numbers indicated he would, and Porter was out of football before the third year of the contract.
Last year, Bills linebacker Lorenzo Alexander, at 33, had 12.5 sacks. But similar to Porter, those numbers accounted for a high percentage of his pressures, adding to concerns that a clear outlier season in his career would remain one. He hit free agency, and returned to Buffalo on a two-year deal worth less than $6 million.
That’s not to say the aging Alexander would’ve broken the bank in a similar spot 10 years ago. But it did give teams a clear red flag without having to look at 16 games of tape while assessing hundreds of players in building a free-agent board.
And on the coaching side, it backstops a quality control guy’s work. Most QCs only get through four or five of an opponent’s games in a week’s work of prepping a report. Having the extra data on tendencies, situational play, and use of formations and personnel doesn’t mean that assistant can watch more games. It does mean he can crosscheck his findings from the ones he did see.
“The problem with football, they still base a lot of analysis on the previous four games,” Hornsby says. “So let’s say you’re playing Cincinnati, and you want to look at their tendencies when they’re in base personnel. You might wind up with 40 snaps out of 280. And then you’ll make a judgment. Well, of those 40, how many were on third down? How many came on second down?
“So what do you do if there are say, two or three snaps on second down out of base personnel? You have no option but to guess. What you can do with analytics, you’re growing the base data in areas that really matter.”
In many ways, it’s analogous to what technology like XOS has done for film study—where you can call up plays by down and distance, area of the field, personnel grouping, formation, even hashmark. Coaches can now crosscheck their study of four or five games through the data from full seasons, the same way they check out plays by situation without going through a zillion old beta tapes.