Loose Ends - Part III A: The Power Play

[EDIT: It appears that I made an error when calculating the skill standard deviations for EV and PP shooting percentage at the team level. The tables and numbers referenced in the post have been edited to reflect the correct values.]

______________________________________________

I've written about the powerplay a few times in the past, with one post focusing specifically on the powerplay itself, and the other relating to special teams performance in general.

The purpose of this post is not only to address some questions that were left unanswered by the two previous ones, but also to look at two as yet unaddressed (at least, unaddressed to the best of my knowledge) issues relating to the powerplay.

Because the treatment of each issue is relatively extensive, I've decided to address them in separate posts.

______________________________

The first issue relates to whether powerplay shooting percentage is more or less 'random' than even strength shooting percentage. Admittedly, the use of the term 'random' leads to some confusion here. For both metrics, skill - or more properly, non-luck - would account for 100% of the team to team variation over the long run. What we're really after is whether the team spread in powerplay shooting talent is wider or narrower than the team spread in even strength shooting talent.

In a post from earlier this year, I included a table that showed the percentage of variation attributable to luck for various shooting metrics over the course of the regular season, based on data from the post-lockout era. I've reproduced that table below.

As indicated, whereas roughly 90% of the team variation in powerplay shooting percentage can be attributed to luck by the end of the regular season, the corresponding figure for even strength shooting percentage is only 67%.

Unfortunately, this fails to resolve our issue, for the reasons specified earlier. Teams take much fewer shots on the powerplay over the course of the regular season as compared to even strength. The disparity in sample size must be controlled for.

Coincidentally, this very issue arose in the comments section of a post made at behindthenet earlier this week. While I was in the process of working on this post at the time, I figured I'd address the matter then and there. Here's what I had to say:

Using seasons since the lockout, the variation in EV SH% at the team level is 33% skill and 66% luck, whereas the variation in PP SH% at the team level is 9% skill and 91% luck.

But the average team takes far fewer shots on the powerplay (~500) than at even strength (~1800). It goes without saying that the % of variation due to luck varies as a function of sample size (i.e. number of shots).

In order to compare apples to apples, it’s necessary figure out how many extra goals a team that is one standard deviation above the league average with respect to EV shooting talent can expect to score over a team that is exactly league average in that respect, per X number of shots.

If the same calculation is repeated in relation to powerplay shooting percentage, the results can be compared.

We’ll use 1000 as the value for x, which is the number of shots.

The results:

EV SH% – 2.64
PP SH% – 1.43

[EDIT: The correct values are 4.83 for EV SH% and 4.77 PP SH%]

So a team one standard deviation above the mean with respect to EV shooting talent can expect to score 2.64 more goals than a team with average EV shooting talent, per 1000 shots.

(We’ll ignore the fact that EV shooting talent and EV outshooting appear to be negatively correlated at the team level).

And a team one standard deviation above the mean with respect to PP shooting talent can expect to score 1.43 more goals than a team with average PP shooting talent, per 1000 shots.

So the implication is that team talent differences in EV SH% are wider than team talent differences in PP SH%.

So there you have it. Team talent differences in shooting talent on the powerplay appear to be smaller than team talent differences in even strength shooting percentage.

[EDIT: The correct values suggest that team skill differences in powerplay shooting percentage are roughly equal in size to team skill differences in even strength shooting percentage.]

One drawback with my method was that I looked at overall powerplay shooting percentage, rather than 5-on-4 shooting percentage. It's possible that the inclusion of other man-advantage situations (5-on-3s, namely) has affected our result.

In order to make sure that that wasn't the case, I made sure to ran the numbers for 5-on-4 shooting percentage as well, using the data available on behindthenet. Here are those results:

% RANDOM = percentage of variation attributable to randomness
% Skill = percentage of variation not attributable to randomness
1 Sigma/1000 = the number of goals a team one standard deviation above the mean in 5-on-4 or EV shooting talent (as the case may be) would be expected to score, relative to an average team, over the course of 1000 shots]

While the differences between the two values are smaller when the 5-on-4 numbers are used, the conclusion remains - teams appear to be more varied with respect to even strength shooting talent as compared to powerplay shooting talent.

[EDIT: If anything, the correct values indicate the opposite - that teams appear to be more varied with respect to 5-on-4 shooting talent as compared to even strength shooting talent.]

How confident can we be that teams are, in fact, more deviated from one another in terms of even strength shooting talent than powerplay shooting talent? Not very. There is some uncertainty in our estimate for the luck component of powerplay shooting percentage at the team level over the course of a season. The figure of 91% is based on an observed standard deviation of 0.158 and a predicted standard deviation of 0.015. If the observed standard deviation was 0.0165 - i.e. slightly higher - then our estimate for the luck component would change to 84%. If the luck component was 84%, our 1 sigma/1000 value then becomes 2.55, which is comparable to the 2.64 1 sigma/1000 value obtained for even strength shooting percentage.

In other words, it's quite possible that teams are similarly distanced from one another with respect to both measures. Support for this proposition will be offered in the next post on this subject.