After all, only about 35% of league play occurs with the score tied. And of that 35%, one-fifth consists of special teams play. Taken together, that means that time played at even strength with the score tied represents less than 30% of a typical NHL game.
Indeed, if we examine the relationship between a team's even strength shot ratio with the score tied and it's overall goal ratio for every season since the lockout, we find an average correlation of 0.556, meaning that even strength shot ratio only accounts for roughly 30% of the variance in outscoring with respect to a single NHL season.
However, because goals in the modern-day NHL are relatively rare events, a substantial proportion of the team-to-team variation in seasonal goal ratio can be attributed to luck. For example, random variation accounted for 47, 35, and 41 percent of team variation in goal ratio in 2007-08, 2009-10 and 2009-10, respectively.
In a hypothetical season with a sufficiently long schedule, that random variation would eventually disappear, leaving each team with a goal ratio commensurate with its abilities. What each team's goal ratio might look like in such a scenario can be approximated by taking its seasonal statistics - namely, shot ratio, shooting percentage and save percentage - and adjusting them to account for the extent to which each one is affected by random variation.* For both shooting and save percentage, the adjustment is significant as luck accounts for a majority of the variation in respect of both over the course of a single season, as indicated in the table below.
For shot ratio, however, the adjustment is less severe as the impact of randomness is comparatively smaller. Consequently, as the sample size increases, so too does the correlation between shot ratio and goal ratio.
If this exercise is performed for each post-lockout season, one is able to determine the relationship between true goal ratio and even strength shot ratio with the score tied. The results:**
Therefore, in an imaginary league in which luck is a complete non-factor, EV shot ratio with the score tied would account for roughly 65% of the variance in outscoring. In other words, even though the two variables may not be strongly correlated over the course of a single season, a team's EV shot ratio with the score tied serves as a reasonably good indicator of how it can be expected to perform over the long run. This is especially true for the three most recent seasons, in which EV shot ratio accounts for 75% of the variation in outscoring ability. It seems that as the level of parity between teams has increased, even strength shooting has become even more important.
Finally, the remaining 35% of outscoring variance indicates that there are other sustainable components of team success. Apportioning the remaining proportion of the variance between these components gives us an idea of their relative importance.
As special teams ability and EV tied shot ratio are correlated variables, residual special teams skill refers to the proportion of special teams skill that cannot be accounted for by EV tied shot ratio. Residual specials teams skill accounts for about 49% of the remaining variance.
Similarly, residual EV shot ratio refers to the proportion of even strength outshooting that cannot be predicted by EV shot ratio. This accounts for 7% of the remaining variance.
The rest of the remaining variance is explained by even strength shooting, even strength save percentage and residual variance. Residual variance is the amount of variance left over after subtracting the sum of the other four components from 1. It results from the fact that the four components are not uncorrelated, independent variables.
* even strength and special teams statistics were, of course, treated separately for this part of the analysis
**There is an alternative calculation that can be applied as a check on the correctness of these values. As the seasonal reliability of both goal ratio and EV tied shot ratio is imperfect, it is necessary to upwardly adjust the observed correlations between the two variables in order to ascertain their 'true' relationship - that is, the correlation that would result if each variable was perfectly reliable.
The adjustment involves dividing the observed correlation by the square root of the product of each variable's reliability co-efficient. In other words
r adjusted = r observed/ SQRT( reliability EV tied shot ratio* reliability goal ratio )
The application of the above formula involves determining the reliability co-efficients for each variable, which can be calculated as follows:
reliability = 1- [(1- split half reliability)/SQRT(2)]
The average adjusted correlation is 0.81, which is comparable to the average adjusted correlation obtained through the first method (0.804). It should be noted that this second method is likely to slightly overestimate the true correlation, given that the two variables are not truly independent.If these formulae are applied with respect to each post-lockout season, the following results:
EDIT: Accidentally used Fenwick ratio instead of Shot ratio when determining observed correlations for 2007-08, 2008-09 and 2009-10. Table and accompanying discussion has been edited accordingly.
EDIT 2: In re-thinking the method used in the alternative calculation, it occurred to me that the better way to adjust the observed correlations would be to calculate all three input values at the half-season level.
There's no sense in using the split-half reliabilities in order to estimate the full reason reliabilities for EV shot ratio and goal ratio when the split-half reliabilities can be used themselves, given that the split-half correlation between EV shot ratio and goal ratio is readily ascertained.
This approach produced the following results.*
* the half-season values were calculated through randomly selecting 40 games, randomly selecting another 40 games without replacement, and determining the correlation between the relevant variables across the data sets. This was repeated 1000 times, with the average values used..
.
Great post and another feather in the cap of the "outshooting theory".
ReplyDeleteJust a quick question, when you talk about EV Shot Ratio are you referring to just shots, Corsi or Fenwick?
Thanks.
ReplyDeleteI used EV shots rather than Fenwick or Corsi for the sake of consistency (I don't have data on EV missed or blocked shots from 2005-07).
Terrific stuff as always JLikens, you're on fire lately.
ReplyDeleteNot that I disagree entirely, but where does this:
reliability = 1- [(1- split half reliability)/SQRT(2)]
come from, J?
Thinking about it a bit more, just food for thought:
ReplyDeleteWith a lot of players, they take chances that sacrifice territorial advantage (outshooting). But it makes sense for them, because they can finish. They cheat for offense, cherrypick, create stretch pass opportunities ... call it what you like. They also will try to make plays at the blue lines that other guys just do.
Think Pavel Bure. Or in our era Heatley or Kovalchuk at the high end, Lupul or Ribeiro at the low side.
i.e. there is a negative covariance between outshooting and shooting% built in. And because it's generally weaker teams signing the one-dimensional star players, I don't like our chances of prising this information out of the population.
In the post above that should read:
ReplyDeleteThey also will try to make plays at the blue lines that other guys just don't.
Nice to see your comments again, Vic.
ReplyDeleteThe formula is actually empirically derived. Using one randomly selected quarter of the schedule as one data set, and another randomly selected quarter as the other, I determined the average correlation between the two data sets for each variable.
I then repeated the exercise through using half of the schedule as my data set.
I found that the formula was able to predict, with reasonable accuracy, the "half-schedule" reliability, through using the "quarter-schedule" reliability as the input value.
Then it occurred to me that there ought to be another way to estimate the seasonal reliability co-efficients - specifically, if you know the half-sample variance, the half-sample reliability, and the full-sample variance, then you ought to be able to determine the full-sample reliability on the basis that the variance due to luck should halve if the sample size was doubled.
I decided to see if this second method could correctly predict the half-sample reliability values from the quarter sample reliability values. Unfortunately, I found that it tended to overestimate the actual values by 0.03-0.05.
I never figured out what the problem was, so I decided to use the first method.
On the second method, were you using true quarter seasons (i.e. 20 consecutive games) or a random selection of 20 games without replacement?
ReplyDeleteRandom selection without replacement.
ReplyDeleteAnd in relation to your other post, you make an interesting point about the relationship between the percentages and outshooting.
Prior to the 2006-07 season, when there was less parity in the league, outshooting teams tended to do slightly better in terms of the percentages. Since then, however, the reverse has been true.
You suggest that the negative relationship between outshooting and shooting percentage is "built in", which seems sensible enough.
It could very well be that,in today's salary cap world, players that can finish are more or less uniformly distributed throughout the league, which allows the above relationship to manifest.
But prior to the lockout, it seemed as though the outshooting teams tended to have more players that could finish, which may have masked the relationship between outshooting and the percentages.
I'm not saying that's the answer - just offering it up as a possibility.
Great post JLikens.
ReplyDeleteI was curious about how you decided that there was more parity in the league over the last three seasons, and how that might impact the value of EV shot differential. I took a brief look at the goal differentials since the lockout and added them up as absolute values for each season, which gave me the following results:
2005-06: 1076
2006-07: 1168
2007-08: 650
2008-09: 870
2009-10: 834
That rather simple method suggests that your assertion is correct in that the first two years had a lot more variance from zero than the last three; however, the third season is quite different from the last two, and yet we don't see that result in EV shot differential contributing more to winning in that third year. I found that a bit odd given your hypothesis and was wondering if you could comment. Perhaps you're calculating level of parity differently and the third year is more similar to the last two? Or maybe it's just a shit happens thing...
In addition, I was curious if you did a similar exercise with "EV Close" data, and if so, how much different it was from "EV Tied".
Thanks Scott.
ReplyDeleteI use % of variance in goal ratio attributable to luck as a proxy for the level of parity in the league, but looking at the team-to-team spread in goal differential would accomplish the same thing (at least for recent years, where the scoring context hasn't changed that much).
The comment itself was basically just an offhand remark on my part. Thinking about it more carefully, the correlation between the two variables would depend less on the level of parity in the league and more on the structure of the data.
For example, there was relatively little parity in the league in 2003-04. But the correlation between EV Tied shot ratio and goal ratio was high because the teams that did well with the former tended to also do better with the percentages.
The level of parity in 2006-07 was about the same as in 2003-04. However, unlike in 2003-04, there was a negative correlation between the percentages and EV outshooting with the score tied. Consequently, the correlation between EV tied shot ratio and goal ratio was much lower.
And to answer your second question, I didn't repeat the analysis with EV tied, but I suspect the correlations would be slightly lower, given that score effects are still relevant when the score margin is one in the first two periods.
ReplyDeleteThough on the other hand, EV score close does provide a more adequate sample.
Thanks JLikens! I suppose I was curious about the "close" data because we're often trying to analyze small samples of games (near the beginning of a season, a playoff series), and it would be interesting to know if using the "close" data generally yields similar results over the long haul, since particularly in those smaller samples of games, it's nice to have the extra events.
ReplyDeleteTeam quality is something so vague. because they aren't constant so you can say that have quality.
ReplyDelete