Have you ever stopped to think about how NFL players are given their ratings in ‘Madden NFL’? And I don’t mean just ‘Peyton scored a lot of TD’s last year so he’s good for a bunch this year’, or ‘an algorithm will take care of it,’ I mean have you ever considered that somewhere in this country there’s a room full of ‘Madden Ratings Czars’ who hold the digital keys to stardom that every NFL player desires?
The brilliant team over at FiveThirtyEight considered the existence of these czars, then Neil Paine sought him out for an incredible feature on just how the special sauce that is ‘Madden Ratings’ gets cooked up.
Each and every one of you know the ‘Madden NFL’ franchise. It’s sold over 100 million copies (over $4 billion in revenue), and is easily the most recognizable name in sports gaming. First appearing in 1998, Madden releases a new version annually to keep up with the changing landscape of the year. And while a lot changes (players, graphics), much of it stays the same: 2,600 real-life NFL players each get 40 numerical grades to be applied to their digital avatars.
But as I mentioned before, with the existence of ‘ratings czars’ much of how the special sauce gets made is shrouded in secrecy…until now. Below is a series of hand-picked excerpts from FiveThirtyEight on how Madden ratings get made, but to read the full feature of ‘The Secret Process That Turns NFL Players Into Gods’ CLICK HERE.
As Moore wheeled around from his den of screens, he was confronted by not only Newton, but also an enormous boot on Newton’s foot, the result of recent ankle surgery.
“Yeah,” Moore said as Newton hobbled toward him, “let’s talk about your speed.”
Eventually, Newton was pleading with Moore to not make him slower.
Such is the power afforded Moore, a hyperactive Floridian who works as the official Ratings Czar1 for EA Sports’ Madden NFL video-game franchise.
The absolute apex of nerdom. You’re the man in charge of telling professional athletes what their value is in a video game, and there’s not a single thing they can do to influence you. Nerds > Jocks.
Moore’s job has morphed from a behind-the-scenes technician to a sort of celebrity — and villain — in gaming circles. And in an increasingly data-heavy sports landscape, it’s a role that sits squarely in two intersections of growing importance — between scouting and analytics, simulation and reality. Because in both Madden and the post-“Moneyball” sports world, humans become a collection of data points and then are turned back into a digital approximation of themselves.
Translating the athletic skills of flesh-and-blood humans into digital form has been a necessary part of sports gaming as long as real-life players have been incorporated into the software. According to Good, that practice dates at least5 as far back as the 1984 release of “Micro League Baseball” for the Commodore 64.
Am I the only person who’s constantly astonished at just how pervasive ‘Moneyball’ has become in sports? As a Tampa Bay Rays fan we’ve buttered out bread with ‘Moneyball’ tactics for the past decade, competing in a division with teams who spend a whole lot more than we’d ever be capable of doing. Yet to constantly see the ‘Moneyball’ approach to sports crop up in all facets of Sport (hockey, football, basketball, track and field), I’m constantly shocked at how willing the world of sports was to accept change.
But with the advent of licensed games, the stakes were raised. “You want Kirk Gibson to play differently from Tony Gwynn,” Good said.
The method of “Micro League Baseball” was to algorithmically translate a player’s real-world statistical benchmarks — his batting average or home-run total, for instance — into skill ratings that would presumably6 spit similar numbers back out on the other side of the simulation. It was effectively the computerized version of older board games like All-Star Baseball whose colorful spinners reduced the essence of a ballplayer to a series of probabilities, locked in an eternal marriage with chance.7
But player performance is notoriously more difficult to quantify in football than in baseball. On the gridiron, detailed individual statistics are kept for only a handful of positions, and those numbers frequently miss the whole story because of interactions between 11 players on each side of the ball.
I think it’s only fair to assume that player performance in football is more difficult to quantify because nobody’s taken the time yet to keep individual stats. They can keep using this as an excuse all they want, but game footage is there and all that needs to be done is have someone survey it to record those individual stats.
Early player ratings were simplistic. The legendary “Tecmo Super Bowl,” released in late 1991, kept 14 ratings per player. But only a few affected in-game performance for any given position. Meanwhile, the initial version of Sierra Online’s “Front Page Sports: Football Pro,” regarded among the most sophisticated football simulations of the mid-1990s, tracked just eight rating categories for each player, with the same traits taking on radically different meanings depending on the player’s position.
The assortment of skills that needed to be collected for each player grew as football games progressed through the 1990s. By the end of the decade, EA Sports’ NCAA Football series assigned 14 attributes to every player in the game,10 while Madden began handing out 17.
‘Cass, I thought you said we were going to learn all about how the ‘Secret Sauce of Madden Ratings‘ gets made?’….Sorry, we’re getting there….
Playing and testing the games, Moore got a firsthand look at the give-and-take between gameplay and player ratings. He recalls the time a long-forgotten defensive tackle for the University of Kentucky inadvertently became a superstar after a typo assigned him a speed rating11 of 85 — blazing for a lineman — and when the Oakland Raiders, long known for their real-life fixation on speed, became unstoppable because the Madden game engine translated the overall velocity of the Raiders’ roster into far better virtual performances than the team was capable of in reality.
Moore would eventually be promoted to a designer and associate producer along his path to Ratings Czar, but the relationship between Moore’s testing background and his current job is clear. “The [ratings] are probably the single biggest factor in gameplay,” Moore said.
And the ratings have far more moving parts now than when Moore began working at EA. Each player in the game is graded in 43 categories — many of which were added when Madden transitioned from the sixth generation of consoles to the seventh. There are also nearly 20 new player-tendency tags, known as “traits,” that control specific player behaviors.
The more I keep reading this feature, the more it just seems like a group of ‘ratings czars’ arbitrarily choose the ratings as they please.
When Moore sits down to build a player’s ratings for the newest version of Madden, he goes through one of two separate processes, depending on whether the player is a veteran or a rookie. Each type of player offers its own challenges. Veterans have existing ratings from previous games, but the degree to which each of their categories must be changed is uncertain. Rookies, meanwhile, must be created completely from scratch.
For the past few versions of the game, users can download Moore’s latest roster update every week of the season via Xbox Live or the PlayStation Network. That means a player’s ratings fluctuate based on how he plays each time he takes the field. To figure out whose ratings to tweak and by how much, Moore said he combines his observations and notes taken during games13 with subsequent film study, conventional statistics, and — increasingly — advanced metrics from outlets such as Football Outsiders (particularly for schedule strength) and Pro Football Focus.
I take back everything I said, this man is a scientist in charge of the most powerful sports video game franchise in the world. I was a fool to think that any ratings would be chosen on a whim. Though part of me believes that with the meteoric rise of Fantasy Football, at some point at least one player has been slighted in ‘Madden’ ratings due to how he spurned one of the czars in fantasy…It HAS to have happened.
Pulling all that data together, however, is when Moore’s instincts take over. Listening to him describe the process of rating a Madden player across all 43 categories, I began to realize that he has unwittingly adopted a sort of ad hoc Bayesian updating process.
He said there’s a bit of “What have you done for me lately?” in the ratings but that certain categories are fundamentally more or less prone to short-term adjustments — another Bayesian-sounding notion.
“Speed largely stays the same,” Moore said, “although when guys get hurt, I will make a change to [their] speed and agility” — a fact Cam Newton came to know firsthand. “Carry rating can be in flux if a guy fumbles a few times,” Moore continued. “Every position has certain ratings more impacted in-season than others.”
Can I go back to school for this? Which handful of degrees to I need to possess in order to be qualified as a Madden Ratings Czar? And now that we’ve come this far in the feature, what’s the formula for players?
Here’s the formula you’ve been waiting for:
Doesn’t make much sense to you? Well, it’s because there is A LOT to digest over on FiveThirtyEight, and I didn’t want to jack it all and put it here. So head on over there and read the feature in full if you’re still not clear how player ratings in Madden are made.