Chad Johnson and the Maddening Voodoo of NHL Backup Goalie Numbers

With the New York Islanders having back-to-back games last weekend, many expected backup Chad Johnson would get an appearance, keeping with the Islanders' habit of splitting the starts in such situations. Instead, after Jaroslav Halak won with light work in Detroit, they used him again at home against the Lightning, when he again won.

Whether that workload had anything to do with the "maintenance" that kept Halak out of the two games since then, the bottom line is teams need two goalies each year, and the Isles are no different: Halak himself has never eclipsed 60 games in a regular season -- his breakout 2009-10 season for Montreal included 18 playoff games for a total of 63 regular season plus playoffs -- so the Isles came into this season expecting to lean on Johnson, too.

With Halak taking both games, Johnson had to wait until Tuesday against the Canadiens to get his 10th start of the season in the Isles' 34th game. So far he's been a disappointment (more on that later) with a .867 save percentage (.877 at even strength) -- numbers that actually went down a tick Tuesday, though the three goals he allowed on 21 shots were hardly cause for scapegoating.

If Johnson had been off to a better start, the Isles probably don't "reward" Kevin Poulin for his work in the AHL this season with a spot start last night in Buffalo, a 4-3 shootout loss in Poulin's second consecutive night of duty. (Poulin had shut out Providence the night before for AHL Bridgeport.)

History and Samples with Backup Goalies

But is 10 starts and 630 minutes enough to get a fair judge of a goalie? Historically, no. That's what makes gauging the progress and predicting the future of backup goalies so difficult.

And yet, is a career of up-and-down seasons in the AHL and one superior season in the NHL (albeit just 27 games with Boston last year) enough that we should've expected a down year to be a good possibility for Johnson? Historically, yes.

When we discuss goalies and try to measure their performance, we cite save percentage and even-strength save percentage not because they are spectacularly informative measures, but because they are merely the best standard measures we have. (There are other promising non-standard measures, but they're still in development and not part of the broader vocabulary.)

Save percentage doesn't tell everything over the long haul and tells very little in the game-to-game, but it's a proxy for the way things are going for a goalie. Unfortunately, as we've seen, "the way things are going" for a goalie can swing wildly and in the short-term -- like, half a season or more -- be influenced by many factors beyond his control.

Almost paradoxically, goalies are huge difference makers in a game, yet there's simply so much going on in a game that's beyond their control -- which makes us both acutely focused on and also easily deceived by their results in individual games.

At the Mercy of What You Can Control

That "beyond their control" part is the area that can make an average goalie look really good or really bad, depending on the weather, his defense, the opponent's own talent and luck, and the shots that go in off the inside of the post or instead mercifully graze the outside.

As Columbus goalie coach Ian Clark explained in a great piece about goalie stats at nhl.com:

"Goalies react to the hand dealt to them; they don't get to control play," Clark said. "Unlike a quarterback or a pitcher, who gets to control and dictate the play, goaltenders are recipients of the play, and as such there is a huge impact on how the team plays in front of them and statistical outcomes.

"Sergei Bobrovsky, Henrik Lundqvist and Tuukka Rask all face completely different things and have no control over it, whereas Peyton Manning controls what is going on over his playing surface when he lines up behind the ball. That affects statistics, and that is one of the reasons goaltending is a bit of an anomaly when it comes to using pure numbers."

This is quite true, although we've seen over a span of several years and thousands of shots that at least the top and bottom goalies start to separate from the vast pack in the bell curve.

James Mirtle in the Globe & Mail recently discussed why hockey's analytics are behind other sports because the sport is so much harder to distil down to one number or isolated events:

For one, there are far fewer scoring plays in hockey than other sports (think goals versus touchdowns etc.). Fewer scoring plays means more chance is involved, both at the end of a game and the end of season.

Ultimately, that means winning in hockey is more heavily influenced by luck (or randomness) than other sports.

The other big difference between hockey is there is a goaltender – the single biggest wild card in any game, as evidence by James Reimer and Carey Price stealing the show last week.

All that makes hockey harder to measure and quantify.

This Year's Surprises and Disappointments are Familiar

Later in that notes column, in an unrelated yet related way, Mirtle discussed the three most improved and three most disappointing goaltenders. Each group exemplifies why judging and projecting goalies is downright voodoo:

1. Devan Dubnyk. Mirtle wrote: "Was this the Oilers factor? Dubnyk appeared to have nearly played his way out of the NHL last season, when he cleared waivers and bounced around between three different organizations and even the AHL"

"Now Dubnyk is playing much better than the starter Mike Smith, the goalie the Coyotes have committed many years to despite some possible red flags in the numbers."

2. Jonathan Quick, Los Angeles. Mirtle notes that Quick, who has had some very hot and cold seasons -- yet if you've watched him when he's hot, you understand why he is revered -- is putting up numbers "better than even 2011-12 when he finished second in Vezina Trophy voting. Next to Nashville’s Pekka Rinne, who’s not on this list after missing most of last season due to injury, Quick has the best even strength numbers of any No. 1 leaguewide."

3. Craig Anderson. Anderson has had notoriously hot and cold seasons, to the point that he has carried Florida, Colorado and Ottawa in individual seasons, then seen those teams fall back the next year when his numbers fall back to merely mortal rather than elite.

The fact that this has happened multiple times with Anderson underscores the maddening voodoo required to evaluate goalies.

The Mysterious Backups

The wild swings from season to season in the table above illustrate the difficulty here. We know 200 minutes, even 900 or 1500 minutes are hardly indicative -- or rather, predictive -- of a goalie's true form. But here we are halfway into a season and that's what we have -- or less -- on a goalie's performance. Johnson's 1500 minutes last season told us something, but probably not as much as the several (mostly AHL) pro seasons that preceded it.

Of those listed in the table above, Khudobin is the one I'd worry least about: His numbers have been consistently promising in the NHL and AHL, so his first 280 minutes of 2014-15 are a mere blip on the way to a near-certain rebound.

I'm burying the lead here, but the three "most disappointing" goalies Mirtle references, as well as "most improved" Dubnyk, are each goalies that Isles fans have discussed as good acquisition targets during the pre-Halak years when the Isles goaltending was a mess:

1. Chad Johnson. "Johnson came out of nowhere to have a big season at 27 years old, with a .925 save percentage as Tuukka Rask’s backup last year," Mirtle writes. "As always with goalies, beware of small sample sizes."

2. Ben Scrivens. He was good as a backup in Toronto, traded to Los Angeles for not much, then traded to Edmonton for not much before going on a tear with the Oilers last year. We heard a lot from Isles fans who said the Isles were crazy not to grab him. Now this year, his numbers are awful.

But then again so are the Oilers, meanwhile ex-Oiler Dubnyk has resurrected in Arizona...

I'm reminded of this post from Contrarian Goaltender last season comparing the track records of Dubnyk and Scrivens, back when Scrivens was the hot name and Dubnyk the goat after being ousted from Edmonton. As he wrote in January:

Both Dubnyk and Scrivens are 27 years old and pending UFAs.  Scrivens has a .917 career save percentage to Dubnyk's .910, but Dubnyk has almost 4 times the sample size at 5079 [shots against] compared to 1365.

Once again, sample size.

3. Anton Khudobin. Khudobin is perhaps the biggest puzzle of this group. Like Johnson, he had impressive numbers as Rask's backup in Boston behind a good defense. But then his first year in Carolina made it appear that he was no Boston mirage. Yet this year he was winless in his first eight with an .892 save percentage.

Of course, that's a tiny sample as we know. And even if he's playing poorly and/or struggling, Khudobin's behind a bad team and history says he's likely to rebound. Sure enough, a few games later he has his first win and his save percentage is up to .903 in 11 games.

Filtering for Signs of Truth, Reasons for Hope

The bottom line is goalie numbers are volatile, their performance influenced by several outside factors that can obscure things for 20, 30, even 40 games.

The challenge with all of these backups, is that their very job description tend to play fewer minutes in a given season than what we need to be confidence that "this is his true expected performance level."

So is Johnson likely to continue to struggle? As with Jaroslav Halak's first few weeks, struggles in a handful of games stand out but are not predictive. Some of Johnson' observable mistakes and flaws in recent games -- the kind that make you have an .890 on a given night instead of a .920 -- are not the kind of thing a goalie could get away with and produce great numbers for 27 games even behind the Bruins defense.

However, we and the Islanders should've known even before this season that last year's 27 games at age 27 in 2013-14 were more likely to be outlier rather than a new norm.

Johnson's limited history -- limited also by our stats in the NHL and AHL -- says he's not this bad. He has five pro seasons of data suggesting we should expect much better than .867 from him. But that history also suggests when he does rebound this season it is likely to be only to the level of, well, a backup.

TL;DR

Save percentage (and even strength save percentage) are the best standard tools we have to measure goalies over the long haul. But hey are subject to many other variables that, well, vary -- particularly with backups who take multiple years (and leagues) to accumulate trustworthy numbers.

Chad Johnson surely isn't as good as last season's numbers, but also not as bad as this season's. We can reasonably expect better over his next 10 starts.