Traditional statistics have always played a significant part in the analysis of teams and players in the NHL. Statistical data influences decisions made by hockey executives regarding draft picks, trades, and contract negotiations. Coaches are dependent upon this information to determine game day line-ups, line matchups, and situational decisions during a game.
Over the last 10 years, the incorporation of advanced hockey analytics in hockey operations decision-making has taken the league by storm. However, the value of advanced analytics and its inclusion as a reliable assessment tool for NHL organizations continues to be a hot button topic.
The salary cap era has forced NHL operations departments to look at additional sources and statistics to make more informed decisions and the Collective Bargaining Agreement (CBA) has produced what the league set out to create: competitive balance. Minimum and maximum team salaries, revenue sharing, and the draft have given executives a league where it is challenging to stockpile and retain talent.
Off-Ice Talent
Given these CBA restrictions, how do NHL teams differentiate themselves from the competition? One way is to attract the most sought-after coach in the league. The Toronto Maple Leafs didn’t have to deal with the salary cap when they signed Mike Babcock to an eight-year contract worth $50 million. Babcock’s contract was not subject to the salary cap and allowed the cash-rich Leafs to attract bench talent that other teams in the league likely couldn’t afford.
A second way is to hire top support staff like nutritionists, strength and conditioning trainers, psychologists, and skills coaches to improve player performance. There is no limitation to what a team can spend on this vital component of an NHL franchise.
Advanced Analytics Experts
That brings us to the third way NHL teams can be more competitive. Some organizations have hired teams of advanced analytics experts who have the tools to collect vast amounts of performance data and, more importantly, to make sense of that data. What the team does with the information is the key to their investment in advanced analytics.
There is no consensus about how the information can be best integrated into the decision-making process. Should advanced analytics replace traditional statistics, or should it be used to support the conventional data that hockey operations staff have depended upon for decades? Should player talent decisions be made solely on advanced analytics, or should the results of the data analysis be used only to substantiate what management already thought about a player?
A Comparison
To help us answer these questions here’s a look at the advantages and disadvantages of traditional statistics and advanced analytics:
Traditional Statistics
Advantages:
- still universally accepted by hockey ‘purists’
- recognizes individual success at crucial times during a game (e.g., game-winning goal)
- identifies statistics not directly linked to goal scoring (e.g., faceoffs, hits, blocked shots)
Disadvantages:
- depends a great deal on qualitative analysis
- limited somewhat by the individual’s capacity to record data
- evaluation is often dependent on what the observer likes (biased opinion)
- based on low occurrence events (e.g., goal differential)
- results are dependent more on team success or who the player is playing with
- assumes that statistics results in wins
Advanced Analytics
Advantages:
- evidenced based analysis (test and prove)
- depends on quantitative data analysis
- relies less on ‘eyes of the beholder’ (e.g., scouts)
- supports the relationship between winning and factor ‘X’
- predictive of future goals and wins
- demands the validity of assumptions (accountability)
- based on high occurrence events (e.g., shot differential)
- can help to substantiate an opinion
- records specific shifts in puck control
- records complicated matchups
Disadvantages:
- depends exclusively on quantitative data analysis
- data recording difficult because of line changes
- doesn’t take into account player and team intangibles
- doesn’t take into account random effects on results (e.g., momentum and streaks)
- doesn’t guarantee success
- can’t measure team culture or player/team psychological factors
- the speed of the game and unpredictability make it difficult to track
Traditional statistics are not data-driven and depend a great deal on an individual who is limited in their ability to collect data and will often be biased in what metrics they believe are essential to performance and success. They may be prone to put qualitative analysis ahead of quantitative analysis because they see more value in the intangibles that a player may bring to the table.
On the other hand, advanced analytics brings accountability to the data by proving specific metrics results in more goals and wins. While this helps to bring a degree of predictability for decision makers, it fails to take into account the intangibles and randomness in the game that can affect both player and team performance.
League Acceptance
The survey results generated by Custance in 2015 offers the most recent data regarding the acceptance of advanced analytics in the NHL. The NHL rankings results are as follows:
- ‘All in’ – Chicago Blackhawks exclusively
- ‘Believers’ – 13 teams
- ‘One foot in’ – 12 teams
- ‘Skeptics’ – 3 teams
- ‘Non-believers’ – Colorado Avalanche exclusively
What is interesting to note is that since 2009 (10 seasons) the Blackhawks have won three Stanley Cups and the other four Cup winners since then, the Pittsburgh Penguins (3), Los Angeles Kings (2), Boston Bruins (1), and Washington Capitals (1) are all in the ‘Believers’ category.
The debate over the value of advanced analytics versus traditional statistics will continue for some time. Many in support of advanced analytics believe there is too much at stake financially when it comes down to player selection and the money involved in those types of decisions.
Parnass offers NHL hockey executives the following advice: “Rather than exclusively trusting the eye test, condemning players for misfortune in small samples, or labeling players as lazy or enigmatic based on reputation or hearsay, analytics has provided the opportunity to scrutinize decision-making and avoid those characterizations. Dismissing analytics means settling for an old method of evaluation and using an inefficient business practice.”