So far in Part 2 of this series we've talked about trying to account for the context in which a player puts up his numbers (Goals, Assists, +/-, corsi, etc.). To this end we've talked about measuring the difficulty of player ice time due to competition and where he starts his shifts, the ease of ice time due to having stronger teammates, and what special teams ice time does the player get. In addition, we've even talked a little bit about some measures, such as Relative Statistics discussed in Part 3, that can be used to account for differences in context and allow us to better evaluate and compare players.
This final part of the Context series introduces the concept of the Replacement Level, a concept (seemingly pioneered in sabermetrics) that is used by statisticians in every American Sport (and probably non-American Sports as well) to evaluate players and truly do so in a way that accounts entirely for the context in which those players play in.
A quick note before we go further: this concept is a good bit theoretical and a lot less useful than what we've talked about in parts 2.1-2.3 of this guide. If you understand parts 2.1-2.3 of this series, you don't really need to worry if this concept if it's confusing. But it is useful to know, and hopefully will be more useful in the future.
The concept of a replacement player is summed up simply like this: every team has an unlimited amount of players who, if necessary, they can call up to the NHL for the minimum salary and perform at a pretty poor level. These are your average AHL players, the guys who are clearly not good enough for the big show, but aren't prospects in any way - they're just career minor leaguers. Every team has plenty of these guys, or so the theory goes, and if they run out they can easily sign some more.
Regular NHLers should of course be better, if not far better, than the average replacement player. If an NHL player is performing worse than a replacement player would in that situation, the team should simply cut that player and replace him with a replacement player. This is pretty easy for the team, as they have unlimited replacement players in the AHL, so why should they stick by a player who is playing worse than that when for the minimum salary they can call up a better player?* Thus the teams should in theory replace those terrible players with the career AHL players they have available - which is how the term Replacement Player got its name. The level of play that a replacement player could put up is referred to as Replacement Level.
*Note that we're not talking prospects here.
Now the replacement level concept basically gives us a baseline to use to evaluate current NHL Players. What would happen if you took a top player, like say John Tavares, and replaced him with one of these replacement players? How much worse would the team be in that case? If we could find out that value, probably measured in goals or wins, we'd have a good measure of how valuable Tavares really was during a particular season.
Note, and this is key, that this measure of how valuable a player is would ideally take into effect context. So when we're evaluating John Tavares, we'd do so by comparing the team's performance with Tavares to the team's performance with a replacement player PLAYING THE SAME EXACT ROLE AS JOHN TAVARES. So the replacement player would be playing with Moulston and PAP, playing a good deal of PP time and no PK time. Thus, in theory, replacement player comparisons would allow us to perfectly take into account context, and to compare players who play in totally different contexts.
The Theory is FAR from perfect....certain players can play certain roles better than others, so it's pretty clear that each team doesn't have an infinite number of replacement players who could fit in in each role. Certainly, it seems clear that the Islanders had to reach down below replacement level for players at certain points last year, something that shouldn't happen according to the theory.
But essentially the point of replacement level theory is to create a baseline which allows us to compare players across roles, accounting for the ease of player to play certain roles and the ease of finding players who can play successfully certain roles - as you might expect it's easier to find a player to play a purely offensive role (such as Rangers' F Derek Stepan) than to find a player who can provide offense and some great defense as well (such as Isles' F Michael Grabner). Replacement level theory allows us to take this into account and better evaluate such players.
Okay so how do we use Replacement Level Theory to evaluate hockey players?
The problem with the theory, unlike in baseball, is that at the moment, there really aren't any great statistics using replacement level theory at the moment. The Best is Hockey Prospectus' statistic, Goals Versus Threshold (GVT), which attempts to measure the amount of extra "goals created" by a player as compared to that which would be created by a replacement player. You can find the GVT values of every player in NHL history of to a certain point in a nice spreadsheet here.
GVT has a whole bunch of issues however, and I'd warn about using it as your definitive measurement for player value:
First, it overrates goaltender value, by not account for goaltender performance that is caused mainly by random variance (luck). Thus Goalies lead the GVT Leaderboard and are on the bottom of it every year, and it's not even close.
Second, it doesn't take into account quality of teammates and quality of competition. GVT does account for zone-starts somehow (not really sure how), but this is a huge thing to miss. I suspect this results in lower defensive valuations for those who play tough competition and higher defensive values for those who play against weaker foes.
Third, I personally find its way of measuring defense to be not as optimal as one would think - it uses relative +/- as a huge factor, allowing for a bad goalie to muck up a player's defensive #s, even if it wasn't his own fault.
For a better explanation on GVT and some of its shortcomings, see my post here.
This is not to say that GVT is a bad measure - I find its a good first place to look at a player when you're curious about his worth to the team. However, when a player's value by GVT doesn't match up with your expectations, looking at other values (QUALTEAM, QUALCOMP, and other non-context valuations) is definitely worth your time, as the statistic is far from perfect.
In theory, replacement level comparisons would make evaluating hockey players pretty easy. Unfortunately, things aren't that easy, and while the prime statistic using the theory (GVT) is pretty solid, it's not something you can use to perfectly evaluate players due to its faults. Still, future measures of player value are likely to utilize replacement level theory, so it's a good thing for you to know.
This concludes our talk on how we can take into account and sometimes adjust for the context in which a player puts up his numbers. This is a really important topic, which is why I wrote 4 articles on it. And even then, I've likely left things out. Just please, when talking about the worth of a hockey player, remember this subject and don't forget to look at the context in which a player plays.
The Intro to Hockey Analytics/Advanced-Hockey-Statistics Primer so far:
Part 1: - What is the field of Hockey Analytics and Why Might You be Interested?
Part 2.1: - The Importance of Context Part 1 - Time on Ice
Part 2.2: - The Importance of Context Part 2 - Evaluating the Difficulty of Certain TOI through QUALCOMP and Zone-Starts
Part 2.3: - The Importance of Context Part 3 - Evaluating (and Compensating for) the Effect of Teammates via QUALTEAM and Relative Measures