A while back, I wrote about the effect of randomness on even strength* shooting percentage. For those who didn't have an opportunity to read that post, I'll briefly summarize the findings:
1. It would appear that a large portion of the inter-team variation in EV shooting percentage can be accounted for by random variation.
2. The spread among NHL teams is very slightly broader than what would be expected by chance alone. Therefore, it would appear that teams do in fact exert some influence on their EV shooting percentage.
* I actually only looked at 5-on-5 shooting percentage. I assume that the results are generalizable to even strength play as a whole, although that may not be the case.
In making my playoff predictions this year, I looked very closely at each team's shot rate (both SF and SA) in each of the main game situations (that is, 5-on-5, 5-on-4, 4-on-5). I also looked at how often each team had played in each game situation over the course the season. My method of determining the 'better' team basically revolved entirely around these two factors.
My method of evaluation didn't really accord much weight to the percentages, regardless of game state. I did this under the -- somewhat faulty -- assumption that most of the inter-team variation in the percentages is due to randomness. If true, there wouldn't be much point in taking the percentages into account when attempting to predict future results.
As it happens, that really isn't true at all. It appears to largely be true in terms of EV shooting percentage. However, this finding isn't necessarily generalizable to other game states. For one, it doesn't seem to be case for EV save percentage. More on that later.
It also doesn't seem to apply to 5-on-4 shooting percentage. While I began my analysis under the expectation that the majority of the inter-team variation in 5-on-4 shooting percentage could be explained through randomness, that doesn't appear to be true.
My methodology was basically identical to that used in my analysis of 5-on-5 shooting percentage, with one obvious difference -- instead of looking at 5-on-5 play, my focus this time was on 5-on-4 play. Here's a quick explanation of my method.
Firstly, I looked at how many shots each team took at 5-on-4 during the 2008-09 regular season. The values can be viewed at behindthenet. I then figured out the average 5-on-4 shooting percentage in the league (~0.128). I then simulated 100 'seasons'. In each 'season', the number of shots taken by each team was the number of 5-on-4 shots taken by that team during the 2008-09 season. However, the percentage of scoring a goal on each shot for every team was 0.128 -- the league average 5-on-4 shooting percentage. That is, each team was assigned the exact same shooting percentage. This is significant as, in any particular 'season', any deviation from the mean is strictly due to randomness, thus allowing one to determine how the spread in 5-on-4 should appear through the impact of randomness alone.
The first graph is fairly straightforward. The blue distribution is the 'predicted' distribution. It represents the spread in 5-on-4 shooting percentage over the course of the 100 simulated seasons. Thus, it's an approximation of what the spread among teams in 5-on-4 shooting percentage would look like if each team had the exact same underlying 5-on4 shooting percentage.
The red distribution is the 'actual' distribution. It represents the spread in 5-on-4 shooting percentage among NHL teams for the 2008-09 regular season. That mini-peak on the far right of the graph represents Philadelphia, who led the league with a gaudy 5-on-4 shooting percentage of 0.181.
This graph is a 'smoothed' version of the above graph. The blue distribution required no smoothing and is therefore identical to the one above.
However, the red distribution did require smoothing. Thus, the red distribution in this graph is simply a normal distribution with a mean of ~0.128 and standard deviation of 0.02. Why 0.02? That was the standard deviation in 5-on-4 shooting percentage among NHL teams during the 2008-09 season.
Here's the raw data. It's more or less self-explanatory.
The first two columns are the start and end points for the percentage 'ranges' (it's hard to make sense of non-discrete data). The 'actual' column -- the third one from the left -- shows the numerical distribution in 5-on-4 shooting percentage for the 2008-09 season. So, for example, one team in the NHL had a shooting percentage between 0.18 and 0.185 this season. The 4th column merely shows the relative frequency of the third column values.
In terms of the predicted values, the 5th column shows the frequency of each percentage range for the 100 simulated seasons, while the 6th column expresses those values as a relative frequency. So, for example, out of the 3000 simulated team-seasons (100 seasons * 30 teams = 3000 team-seasons), two teams had a shooting percentage falling between 0.165 and 0.170.
The final column merely shows the numerical distribution in shooting percentage in a hypothetical 30 team league where each team has the same underlying shooting percentage. This allows for a comparison to be made with the actual distribution, shown in the 3rd column.
Finally, the supplemental data. For each of the 100 simulated seasons, I calculated the standard deviation in shooting percentage among the teams. 'PREDICTED ST DEV MEAN' is the average standard deviation over the 100 seasons. 'PREDICTED ST DEV MIN' is the minimum standard deviation for the 100 seasons. 'PREDICTED ST DEV MAX' is the maximum standard deviation for the 100 seasons. 'ACTUAL ST DEV' is the standard deviation in 5-on-4 shooting percentage among NHL teams for the 2008-09 regular season.
The fact that the actual standard deviation is larger than the maximum standard deviation for any of the simulated seasons is fairly conclusive proof that randomness alone cannot account for the inter-team variation in 5-on-4 shooting percentage. For what it's worth, when I did my analysis on the effect of randomness on EV shooting percentage, some of the simulated seasons had a larger standard deviation than the actual standard deviation.
Lastly, the final row shows the inter-year correlation -- that is, for 0708 and 0809 -- in 5-on-4 shooting percentage at the team level. The value is non-trivially positive, which lends support to my above finding that teams can reliably influence their 5-on-4 shooting percentage.* Not surprisingly, the Flyers were tied for the league lead in 5-on-4 shooting percentage last season.
*On the other hand, Tyler at mc79hockey, in a post examining this type of thing at the start of the 2008-09 season, found an inter- year correlation that was somewhat lower, on the order of ~0.30.
Also relevant and of interest:
Vic Ferrari, in the comments section of this post -- in which he examines the ability of individuals player to effect PK SV% --, reports that individual Oilers had a substantial effect on powerplay shooting percentage while on the ice. This tends to support the idea that powerplay shooting percentage is highly non-random in its distribution.