Is Pavelec’s Recent Success Sustainable?

Ondrej Pavelec is a name that makes many Winnipeg Jets’ fans cringe; he’s the starting goaltender despite a large amount of evidence that he performs worse than many backups. Over his NHL career Pavelec has a .906 save-percentage (sv%), in the 2013-2014 season alone he has posted a .901 sv%, good for 70^th in the league, when the league average is .910. At even-strength he posts a similar .908 sv% which ranks him 74^th overall. By consistently playing Pavelec as much as they do the Winnipeg Jets are purposely hurting their chances of making the playoffs.

It was a positive note to see Pavelec start the first eleven games of the 2014-2015 with an above-average .921 sv% over 278 goals against. This positive start to his season has convinced some fans and media members that the 27 year old is a later bloomer and is turning his career around. Inspired by a twitter conversation from PensionPlanPuppets writer Steve Burtch and ArcticIceHockey’s Garret Hohl we analyze this situation to dig deeper into Pavelec’s success.

3000 shots

Analyzing goaltenders is very hard; they are very susceptible to shooting percentages and will see plenty of highs and lows in a short period of time; over time shooting and save percentages will regress towards the mean and smooth out. It is near impossible to use the ‘eye test’ to determine which goaltender is better as the difference between an NHL elite goaltender and an NHL average goaltender is less than 10 goals against on over one-thousand shots.

The guys at LeftWingLock performed a great study to simulate thousands of goaltenders to try and answer the question of how long it takes before you can be confident in a goaltender’s true talent. The take away from the article is that it takes over 3,000 shots against a single goaltender before you can be confident in their talent level. Pavelec is a .906 career goalie in over 8,758 shots against so it is very unlikely that he is going to improve from there.

Ondrej Pavelec’s 2014-2015 Season

The start to the 2014-2015 season has so far been good for Pavelec. In this time, as mentioned, he has posted a .921 in 11 games and 278 shots against. We know this is unlikely his true talent level but as we dig deeper into the numbers, posting this sort of save percentage over this small of a sample size is not unheard of. Looking back at his previous season there were 7 unique stretches of eleven games were Pavelec played with a sv% of .921 or higher. This sort of performance is not unique and he has performed at this level before. If he was not able to have stretches of games with this level of success, or higher, it’s unlikely that he would be in the NHL.

Conversely we can look at an elite goaltender such as Tukka Rask who has a career sv% of .927 in over 5,600 shots against. In 2013-2014 Rask had 31 unique stretches of 11-games where he posted a .921 sv% or higher and 17 unique stretches where he posted a .935 or higher in 11-games. But even elite goaltenders are not infallible as Rask had 2 unique stretches of 11-games where he posted worse than Pavelec’s career sv% of .906 or worse.

We know that Pavelec has likely demonstrated his true talent level of a below-average career sv% of .906 and despite that we know that in small samples of games bad goalies can look good and good goalies can look bad. We want to further analyze how likely the success of Pavelec has been in this stretch

Monte Carlo

Similar to the experiment performed by the writers at LeftWingLock I used a Monte Carlo simulation to simulate 250,000 theoretical Pavelecs. A Monte Carlo simulation is explained by Wikipedia as “broad class of computational algorithms that rely on repeated random sampling to obtain numerical results; typically one runs simulations many times over in order to obtain the distribution of an unknown probabilistic entity”. To do that we take repeatedly take a simulated Pavelec, throw 278 shots against him, with his career sv% of .906 and randomly determine if he was able to stop this. By repeating this many time we can average out the results and move towards the most likely outcome and from this we can see what the likelihood of posting .921 or better on 278 shots against is.

Running this simulation on over 250,000 iterations we find that a goalie with a career save percentage of .906 is able to stop 92.1% (or better) of 278 shots against 17.1% of the time.  Essentially it is not out of the norm to see a goaltender of Pavelec’s talent to post .921 or better one in every six tries.

Looking Forward

Pavelec is an interesting goaltender in that all of the analytics point to him being a terrible starter yet he is consistently played despite the proof that he is hurting the team. Part of the reason for his poor performance might be in his technique. @nmercad on twitter cites a large portion of his bad results are from his sloppy technique: Pavelec “refuses to use his stick to corral low shots. Gets sloppy when play is behind him and he has never challenged well. He still plays the same reckless style that has always done him in. Out of position and relying on athleticism to bail him out”.

We know that analyzing goaltenders is very hard. The statisticians at Five Thirty Eight agree with this in their own work on goaltenders and acknowledge that in a small sample size of even a season is not enough to truly know how well a goaltender performs. With Pavelec we have multiple seasons worth of data on him and despite his ups and and downs in performance, from his .914 in 2010-2011, to his .880 in 2008-2009, we can say with confidence that he is not a starting goaltender that a team should be depending on, and that despite the success he has seen so far in the start to this 2014-2015 season, it is not likely that he will continue to sustain this level of success.

To end on a positive note, despite all this evidence against him, and many other analysis in smaller samples, we should still be allowed to praise Pavelec for the performance he has displayed. Thanks in large part to him the Winnipeg Jets have gone 2-0-1 in their last three games despite only scoring a total of 2 goals. We just cannot forget that limited success in the past does not suggest success future success.