clock menu more-arrow no yes mobile

Filed under:

The (Many) Trouble(s) with Evaluating Goalies

How should we judge goalies in the moment? How much can we read into "streaks" and stretches of bad form?

By Dominik
*Genuinely* didn't have a chance on that one!
*Genuinely* didn't have a chance on that one!
Geoff Burke-USA TODAY Sports

Goaltending is a hot topic among New York Islanders fans, particularly given their 27th-ranked 5-on-5 save percentage (.894) and in particular starter Evgeni Nabokov's .897 save percentage overall (.907 at even strength).

These figures are below NHL average, which is typical of the 38-year-old Nabokov's career for the last several seasons. But they are also below Nabokov's "normal" average, which tells us they are likely to rebound upward unless his 38th birthday was more destructive than a Patrick Kane summer weekend.

Last season, too, Nabokov had a below-norm start before his form, or at least his numbers, rebounded and surged upward along with the team in front of him. I leave it to you to determine how much was chicken, how much was egg.

And therein lies the difficulty with evaluating goaltenders: Even more than team standings and individual skaters, the near-term goaltender numbers are never as informative as you'd like them to be. (For example, right now Cory Schneider and Nicklas Backstrom each have worse even strength save percentages than the much-maligned Nabokov. Small samples, yo.)

Gabe Desjardins and others have done a lot of statistical work over the years to refine how we measure goalies -- gone, thankfully, are the days of looking at the very team- (and shots against-) dependent goals against average (GAA). We know to focus on save percentage as a better tool, and even-strength save percentage as an even more refined if not fully informative tool. (It weeds out goals allowed on the penalty kill, which is no small potatoes but can also be influenced by variance and poor teammates.)

But even those only tell you so much, and mostly on an aggregate spanning lengths of time longer than is helpful. As Gabe wrote succinctly in 2010:

This is what makes it very difficult to find a good goaltender - you need four years of data to figure out how good he is, but you can usually only get one or two. Some people would say that lets GMs off the hook when they sign a guy to a big contract and he doesn't work out. I would say that level of risk means you should bet a smaller percentage of your portfolio on goaltending.

With Nabokov, we can be certain he is below average, and we can be pretty sure -- simply by age -- that he is in some decline whose curve is undefined, but we probably won't know the full picture until after the plane has crashed into the mountain. Or at best, after the plane has avoided the mountain and made the playoffs only to run out of fuel.

When data is scarce or suspect, you might turn to individual scouting and all the observer bias that invites. (For example, Rick DiPietro made admirable strides during his grueling rehabs, and the organization was pulling for him, but improvement over a damaged goalie can still end up being just a less-damaged goalie. "He looks great" was all relative, and not adequate.)

After watching NHL goalies for three decades and experiencing the mechanics of the position myself, my theory is that most NHL goalies who have reached this strata have the ability to stop NHL shooters and turn in outstanding performances, as evidenced by just about every NHL goalie having some shutouts. The challenge -- as with so many athletic traits -- is in whether they have the ability to repeat it consistently.

Reasons for inconsistency can be luck and fitness and injury and fatigue and opponent adjustments and any other variable you want to throw in. So rather than dwell purely on goals against (which are rare events in the grand scheme) and great saves (which can sometimes result from poor positioning preceding it), I try to watch a goalie's overall movements to get an idea of how he's playing, how he's moving if that blocked or missed shot did go through.

And that is where I like to use the NHL GameCenter app (which is the sponsor of this post).

People forget that a goalie's nightly workload is far more than the official shots faced; there are a bunch of shots blocked or wide, as well as all of the side-to-side and up-and-down movement that comes in anticipating potential shot attempts that never happen.

I'll use the GameCenter app to skim around and pick up on movements like that. Just like NHL Center Ice (if you record the games), you can go back over games and pore over all the agonizing details. But of course GameCenter gives you access to all the games from all the teams (some live blackouts notwithstanding) and, better yet, the playbar notes significant events like goals, saves, shots. So it's much easier to scroll to specific moments within a game replay.

Below is a moment from Tuesday's 6-2 loss to the Capitals, with the Caps on a power play. Alexander Ovechkin scored to make it 5-2, but this particular goal was absolutely not on Nabokov -- the Caps converted a great down-low cross-crease pass that Nabokov, to borrow a Butch Goring phrase, literally "never had a chance." You can see the result in the photo at the top of this post.

This was a case where the goal against did not tell us anything about how the goalie is playing, as the opposition created a slam-dunk opportunity. But another earlier moment on this same penalty kill did:

Nabby-sprawled_medium

After a shot from the point, Nabokov is face down in his crease.

That's Nabokov lying face down in his crease after facing a shot from the point that deflected behind the net. It's possible that Matt Carkner bumped him enough to make him fall over on the point shot, but if so it was minimal contact. Had the puck ended up anywhere but behind the net, Nabokov would have been in no position to stop the second attempt. Although this occasionally happens to everyone (and true, smart goalies don't waste energy recovering quickly if the threat is not immediate), ideally you'd like your goalie to not end up prone like this after a routine shot from the point.

Even when a goalie is not being torched with goals against, you can learn a lot from how he is moving in his crease on shots that never end up connecting, particularly how he recovers from initially going to his knees. If he's "scrambling" or "swimming" a lot after going down for the initial shot, he's probably not going to consistently be in the best position to handle rebounds, which are among the most dangerous opportunities hockey offers.

I remember this most obviously during Rick DiPietro's comebacks. Every goalie sprawls to some degree, so I cite it only carefully -- observer bias, individual styles, the fact that I'm not a professional scout, all of these point to caution when drawing conclusions. But DiPietro's discomfort when recovering from the butterfly was so obvious during his comeback, you could pick up on it even when he was shutting out the Devils (on 20 mostly unthreatening shots).

Here's another screenshot from Nabokov's rough night in D.C. This one was a goal against, and I call it out because though it was a legit scoring chance -- a turnover at the Isles blueline let John Carlson walk in to the faceoff circle -- it was one where Nabokov was beaten too easily: On a right-handed shot, coming from Nabokov's left side of the ice, beating him to the far, stick side. (You can watch the full video of the sequence here.)

Carlson-shot_medium

Nabby's angle appears ready for a shot ... from the empty space between Carlson and his winger.

Though this goal came off a turnover, where a goalie has less time to prepare his positioning, Nabokov's angle still looks unacceptably off. Carlson did not beat him with an exceptionally hard or deceptive shot. He just snapped it far side, and Nabokov didn't even get a piece of it. To unquote Butch Goring, Nabby definitely had a chance on that one.

I would say outside some good work in the first period, Nabokov's angles and movement was in no kind of form to bail out the Islanders' mistakes on Tuesday night.

GameCenter Features

Anyway, this is ostensibly a post about NHL GameCenter, but I thought I'd turn it into something topical for us while showing you how I use GameCenter: To illustrate the occasional Grabner breakaway analysis post or to go over game footage in painstaking detail -- particularly the footage (like Nabby sprawled above) that doesn't make it in the highlights package.

You can see from the screenshots above, the little "x"s and red dots point to significant moments in the game you can scroll too. Roll over them in the app, and they describe the event (player, shot/goal/save, time of game, etc.). You can also see options like choosing which team's feed, and choosing your download speed. At top, you can quickly navigate between games, to really gorge on hockey watching.

Obviously I use the app a lot for neurotic blogging purposes, but those same features are really useful -- more than Center Ice, honestly -- for the obsessive fan in me too.

The 38-Year-Old's Search for Consistency

As for Nabokov? He is what he is. In going over periods of other games this year, I've seen him move well and anticipate shots like an NHL goaltender should. You can isolate multiple 20-minute stretches from games this year and you'd never know he's a goaltender currently stopping fewer than nine of 10 shots. There were even multiple periods where he kept the Islanders "in the game" while they got their heads out of their...stick butts.

But the problem, of course, is consistency. Repeating it more often, for longer stretches.

Whatever one thinks of the Islanders defense, further diminished without Lubomir Visnovsky, it's not putting Nabokov under an undue workload. He has given up three goals or more in nine consecutive games, with at least two even strength goals against in all but one of them. He is playing worse than even his below-average norm.

Which means he's still likely to rebound. But it also means he's sure not getting it done right now.

More from Lighthouse Hockey: