Shotblocking and Save Percentage

In a recent post by the Contrarian Goaltender,  he collectively analyzes data from 1999-00 to 2007-08 in order to determine the relationship between shot attempts against and save percentage    One of his findings was that there was a negative correlation between blocked shots and save percentage over this period.

To what extent,  however,  is this effect mediated by shot quality,  as measured by Alan Ryder?   On the one hand,  if the majority of blocked shots are those coming from the point and other peripheral areas of the offensive zone,  then the shots that "get through" would tend to be those from areas closer to the net.   One would expect this to effect to be reflected by the shot quality data.   On the other hand,  if blocking shots has the effect of interfering with the sightlines of the goaltender, then this effect would not necessarily be reflected in the shot quality data.   Both of these effects, if real, could account for the fact that teams that block more shots tend to have lower save percentages, on average.  However, without analyzing the date, it's unclear to what extent each process is operative.

I think that it goes without saying that both processes are,  to some degree,  at work here.   It would be unreasonable to suspect that the relationship between save percentage and shot blocking can be entirely accounted for by one single factor;  indeed, most causal relationships that exist in complex phenomena, hockey included, are multifactorial.   What I'd like to determine,  however,  is how much of the correlation can be accounted for by the shot quality data.

What I did was analyze the relationship between team blocked shots,  team save percentage,  team shot quality,  and team shot quality neutral save percentage for each season between 2002-03 and 2007-08.  I determined the correlations for each season individually because the total number of blocked shots is not uniform over time. Specifically, the pre-lockout values are significantly lower than the post-lockout values, and the 2002-03 values are much lower than that for 2003-04 (and every other season, for that matter).  Thus,  analyzing the data as a whole would preclude interpreting the results with any degree of confidence.   It is worth noting that 2002-03 is no arbitrary cutoff -- there is simply no data on shot quality prior to this season.   As is the case with the shot quality data,  there is a clear arena bias with respect to the recording of blocked shots.   This is evidenced by the fact that the standard deviation in home blocked shots is much higher than for that for road blocked shots.   The data:


What I did,  then ,  is incorporate the figures for both total blocked shots and road blocked shots.   Presumably,  road blocked shots would provide a better indication of the 'true' number of shots blocked as it less subject to arena bias.   Here are the inter-variable correlations:
It should be noted that the shot quality data for 2006-07 and 2007-08 is for road shot quality only -- thus,  I was unable to include the correlations between road shot quality neutral save percentage and road blocked shots for any season prior to 2006-07.   The correlations between road save percentage and road blocked shots were also excluded for these seasons.

Several points:

1.   There are slight,  yet consistently negative,  correlations between total blocked shots and save percentage.  Although I doubt that any of the specific correlations are statistically significant,  the fact that all of them are in the same direction is suggestive of an underlying relationship.    Also,  this accords with The Contrarian Goaltender's finding of a correlation of -0.30 between blocked shots and save percentage over a larger data sample.

2.  The correlations between total blocked shots and shot quality neutral save percentage show a similar trend in that for every season the relationship is mild yet inverse.  Thus,  shot blocking seems to have an effect on goaltender save percentage that is residual and cannot be accounted for by the shot quality data.  Of course, it's also clear that shot quality is also partly driving the relationship.    For one, there seems to be somewhat of a positive correlation between blocked shots and shot quality against.   Additionally, the fact that the relationship between blocked shots and SQN % is not as strong as the relationship between blocked shots and save percentage proper also shows how shot quality partially mediates the correlation.  If it did not, these two groups of correlations would be near identical.

3. The correlations for the 'road' data are generally weaker than the overall correlations,  which is counterintuitive as both road shot quality and road blocked shots are less distorted by bias.   This can probably be explained by the attenuated sample size for road games in each individual season (41 vs 82 games),  which diminishes the resolution of the data.    Indeed,  this also occurred in previous posts where the shot quality figures themselves were analyzed.

Shot Quality

I think that in the case of the vast majority of hockey fans, there is a considerable amount of resistance to non-mainstream hockey statistics. Shot quality is no exception to this.

I'm not going to go into specifics in terms of what shot quality actually is. There are multiple articles at Alan Ryder's site that describe in detail what shot quality is and how it is measured. Furthermore, if you're reading this blog, there is a good chance that you're already acquainted with the idea of shot quality.

Rather, the purpose of this post is to explore the construct validity of shot quality, as well as its reliability. Because the figures for shot quality against (hereafter SQA) are more readily available than those for shot quality for, this post will deal with the former.

Firstly, the issue of validity. The biggest issue with shot quality, I think, is concern over the degree to which the shot quality figures actually mirror reality. Do teams with higher SQA (i.e. worse) really give up more dangerous shots against, or is it merely an artifact produced by arena bias or some other element of the measurement process? I think that the best way to answer this is to look at the relationship between team SQA and team save percentage. If teams with lower save percentages tend to have higher SQA, then that's fairly solid evidence that shot quality is measuring something genuine. The data:



For every single season, there is a negative correlation between SQA and save percentage at the team level, meaning that, on average, the teams that give up more dangerous shots against tend to have lower save percentages. Granted, not all of the correlations are statistically significant, but I think the fact that all of the correlation are in the same direction is suggestive of a reasonably strong relationship. Interestingly, the correlation is somewhat lower post-lockout than pre-lockout. I suspect that part of this might have to do with the fact that the figures for 2006-07 and 2007-08 are only for road games only, which was done in order to reduce the effect of arena bias. The save percentage figures I used for 2006-07 and 2007-08 are for away games only, so it can't be caused by my attempting to correlate road SQA against with overall save percentage. Rather, the cause probably relates to the fact that the sample size has been halved, thus reducing the resolution of the data.

But about it's reliability? Is there a correlation between SQA at Time A and SQA at Time B? The easiest way to determine this would be to look at it's split-half reliability at the team level, which would involve, for example, determining the correlation between a team's SQA against in odd numbered games and the corresponding figure for even numbered games. However, because I only have the figures for the entire season, I'm not able to do this. What I can do, though, is determine its inter-year correlation at the team level. Is SQA repeatable in this sense?




The data is fairly unequivocal. For each season pairing, there is a considerable correlation, and the degree of relationship is highly consistent across seasons. Indeed, the strength of the correlation is remarkably similar to the inter-year correlation for shots against over the same period. Admittedly, the pre-lockout values could be partly driven by arena bias, but that seems unlikely considering that the values after 2006-07 are for road shot quality only and the correlation is equally strong. Therefore, it can be said with some degree of confidence that a team's SQA for any given season is highly correlated with the values for both the preceding and following seasons, and that this relationship does not appear to be mediated by arena bias.

While shot quality is not likely to be embraced by mainstream hockey fans anytime soon, I feel that it has tremendous utility in that it quantifies one of the two components - the other being shots against - of team defense. Based on these findings, SQA can be said to be:

1. A real measurement of the average relative dangerousness of the shots allowed by any given team, as manifested in the lower team save percentages of teams that allow relatively more dangerous shots on average.

2. A reliable, enduring element of team defense, as manifested in the substantial inter-season correlation at the team level.

Save Percentage and Shots Against

Recently, at Hfboards and elsewhere, I've come across the assertion that, among NHL goaltenders, there is a positive relationship between save percentage and shots against. Some examples.

Naturally, I was skeptical of the claim , not least because no compelling evidence was ever offered in support of it. Moreover, there doesn't appear to be any reason why this should be so. Conceivably, there could be some degree of trade off between shot quality and shots against at the team level, in that teams that are better at preventing shots do so at the expense of allowing more quality ones (at least proportionately) and vice-versa. However, there doesn't appear to be much of a correlation between the two.






The chart shows the correlation between shots against and shot quality against (at the team level) in the right column with the corresponding season in the left column. In only two of the seasons (2002-03 and 2003-04) was there something of a relationship. What's interesting, however, is that the correlations are positive, meaning that, in those particular seasons, the teams that were better at preventing shots also tended to allow lower quality shots. There is absolutely no evidence of a trade off between shots against and shot quality.

Of course, it's possible that there could be a relationship between save percentage and shots against independently of shot quality. It's often been suggested that goaltenders play better when they're frequently tested and poorer when underworked. If this were true, a correlation between save percentage and shots against should emerge at the team level. But what does the data say?






As can be seen, there isn't much of a pattern between the two variables. In some years, the correlation is positive; in other years, negative. What's important, I think, is that all the values are fairly close to zero. The only year where the correlation is even remotely significant at the 5% level is 1999-00, but the direction is contrary to what would be expected . The important point is that teams that give up more shots against do not tend to have higher save percentages.

There is absolutely no evidence that high shot totals have an inflationary effect on goaltender save percentage.

Why, then, is it often argued that such a relationship exist? More than anything else, the phenomenon seems to be driven by wishful thinking on the part of the claimants, a disproportionate number of whom belong to certain fan bases. I'll say no more.