*Off the top, it is quite obvious that the only true test of accuracy of any draft ranking will be to reflect back 3 to 5 years from now.
Having gotten that out of the way, last week, I compared the various ranking systems from a couple of different perspectives and highlighted limitations to any type of analysis this close to after the draft {Click here for article}.
This week I present another methodology in gauging the accuracy of the major draft ranking services. Basically, a point was assigned for each draft position a prospect was taken in the actual NHL entry draft versus where the prospect was projected to go in the respective ranking.
For instance, if the rankings being examined forecasted Tavares, Hedman and Duchene going in just that order, the points would be “0”, because that’s where they were all actually drafted. However, if the rankings projected Duchene, Tavares, and Hedman, the points would total “ 4”, because Duchene was ranked 2 off from where he was actually was drafted, and Tavares and Hedman were both “1” off from their actual draft position.
In grading which scouting service and/or ranking system made the best predictions or was most accurate in terms of how the NHL 2009 entry draft went down, there are many limitations. It will likely be somewhere between three to five years before we can begin to understand which scouting/ranking system was most accurate in their prognostications, and compare this to how NHL team’s professional scouting teams performed.
However, until time elapses and players develop, the only true comparator we have is to compare in some fashion the various rankings to how NHL teams actually drafted. Again, there are limitations to such analyses and I will try and outline some of these when discussing of the comparisons I performed below.
The final totals calculated are listed in brackets below. In this analysis, the lower number indicates higher accuracy in terms of rankings closeness to where a respective prospect actually went in the draft.
1. The Hockey Spy{Click here for Mock Draft and Draft Preview} (496) – Just edging out McKenzie and his 10 NHL scouts.
2. Bob McKenzie – TSN.ca (499) – An extremely close second – admittedly, with numbers this close, it likely could go just as easily in TSN’s favour in a more detailed analysis.
3. Future Considerations (633)
4. The Hockey News (747)
5. McKeens (754)
6. Red Line Report (774)
7. International Scouting Services (848)
*Central Scouting Bureau/Services were not listed due to their segregated rankings. They separate North American skaters from European and also have skaters and goalies ranked separately.
Again, the significance of these analyses is unknown, but at present time it at the very least gives us some form of comparison. Given the limitations noted previously and the fact I performed the analyses, I welcome and would appreciate anyone who wishes to perform a separate analysis. Feel free to comment below or email me at hockeyspy@gmail.com .