Saturday, March 19, 2011

Loose Ends - Part II: Score Effects and Minor Penalties



Back in November of last year, I looked at whether there were any score effects in relation to minor penalties. The conclusion? Playing from behind has a significant positive effect on powerplay differential. That is, teams tend to be much better at drawing penalties when trailing, as compared to when leading or when the game is tied.

While my initial article only looked at data from the 2007-08 and 2008-09 seasons, I've since ran the numbers for 2009-10 as well. Here are the aggregate numbers for all three years:

[PD=penalties drawn; PT; penalties taken; P % = penalties drawn/(penalties drawn + penalties taken)]

I should include a reminder that only penalties that were not accompanied by the calling of another penalty at the same point in time were included in the above totals.

In the original post, I asserted that trailing team's penalty advantage was not owing to its superior play, but was instead caused by favorable officiating. In support of this, I noted that actual team-to-team distributions in trailing and leading penalty percentage were roughly what one would expect them to be if the putative bias affected all teams equally.

Although I remain confident that my assertion was correct, I suspect that others may have found my explanation to be less than convincing. And in turning my attention to the subject for a second time, I think that there's a better way in which I can illustrate my point.

In determining whether the trailing team's penalty advantage is the product of bias or earned on merit, it becomes necessary to ask what result we would expect to observe, based on what we know about what causes some teams to be better at drawing penalties than others.

One of those causes is even strength outshooting. If we look at the relationship between EV tied Corsi and tied penalty differential over the last three seasons, each unit increment in the latter equates to 0.027 in the former.

It's well established that the average team does much better in terms of Corsi when playing from behind. Over the three years in question, trailing teams had a collective Corsi percentage of 0.552 (107706 For, 87079 Against). Given the positive relationship that exists between outshooting and penalty differential when the score is tied, the trailing team's advantage in Corsi may be able to account for it's advantage in penalty percentage.

However, upon performing the required calculations, it becomes clear that this factor can only explain part of the difference.


In other words, only about one third of the gap can be attributed to outshooting.

Not only that, but it's clear that the shot statistics flatter the trailing team, given that playing from behind encourages a team to take more risks and play more desperately. For example, during the period in question, trailing teams only scored 51.9% of all non-empty net even strength goals (4623 For, 4292 Against), despite, as mentioned above, generating 55.2% of all Corsi events. It's more than arguable that goal differential, and not Corsi differential, provides the best measure of how well the trailing team actually performs.

As with outshooting, there is a positive relationship between even strength goal differential and penalty differential when the score is tied. Based on data from the three seasons in question, each net goal is worth 0.26 in net penalties drawn. We're able to use this figure to determine what kind of penalty advantage we'd expect the trailing team to have, based on its goal differential.


As the table indicates, we would expect the trailing team to do only slightly better than the leading team in terms of penalty differential on the basis of its advantage in even strength goal differential. Thus, however which way you approach it, referee bias must account for a substantial part - and probably almost all - of the penalty gap.
 

Thursday, March 17, 2011

Loose Ends - Part I: Predicting Future Success



My plan is to put out a series of posts - hopefully all within the next while - that relate to subjects that I've posted on previously. The object of these posts is to address certain outstanding issues that weren't resolved when I tackled these subjects the first time around.

The first post in the series is an extension of a post that I published last month that looked at how various shot metrics - all of them calculated at even strength with the score tied - predicted future success at the team level.

One related issue that wasn't explored is how well those same shot metrics predict future success when compared to more conventional measures of team strength, such as winning percentage and goal ratio.* This question is actually more fundamental than the one investigated in the original post. After all, if shot metrics like Fenwick and Corsi failed to predict future success better than the conventional measures, then that would render them considerably less useful.

The method employed** was similar to the one used in the first post. Because of the relative complexity of the process, including a step-by-step description may be helpful.

Firstly, I randomly selected a certain number of games from each team's schedule, with each team having an equal number of home and road games selected.

Secondly, I calculated how each team performed over those games with respect to certain variables. The variables that were calculated were even strength Corsi with the score tied, overall goal ratio (with empty net and shootout goals excluded), and winning percentage. Winning percentage was defined as WINS/(WINS+LOSSES). Games that ended in a shootout were considered ties, and were therefore not included in the calculation.

I then randomly selected a second, independent group of games. That is, if a game was included in the first grouping, it was not eligible for selection in the second grouping. As with the first grouping, an equal number of home and road games were selected for each team.

I then determined how each team did in terms of winning percentage over this second group of games, and looked at how each of the three variables calculated in relation to the first group correlated with winning percentage in the second group.

The relationship between the size of the two groups can be expressed as y=(80-x), where x represents the number of games included in the first group, and y the number of games in the second group. So, for example, if 20 games were selected for the first group, the second group would consist of 60 games. Ultimately, I elected to use x values of 20, 30, 40, 50, 60 and 70.

The raw data used was from the 2007-08, 2008-09 and 2009-10 regular seasons. The table included below shows the results for each individual season, as well as the average results. The values represent the average correlation over 1000 calculations.

A couple points:

- Corsi Tied is the best predictor of how a team will perform over the remainder of its schedule, regardless of the point in the schedule at which the calculation occurs.

- Corsi Tied is only marginally more predictive of future success than goal ratio or winning percentage when looking at samples of 60 games or more. In other words, as the sample size becomes increasingly large, there are diminishing returns with respect to the predictive advantage of Corsi. By the end of the season, all three variables seem to predict future success equally well

- The above fact has implications in terms of determining playoff probabilities at the team level, with the results suggesting that a composite metric would work best

- The aggregate values for Goal Ratio and Winning Percentage are remarkably similar. The implication is that once shootout results are controlled for, winning percentage is as good of a measure of a team as goal ratio is

Next up: Score Effects and Minor Penalties.

*Some readers may have observed that the split-half reliability of goal ratio (0.417) was lower than the predictive validity co-efficients for both Corsi Tied (0.444) and Fenwick Tied (0.429). The implication is this is that the two latter variables are better able to predict goal ratio from one half of the schedule to the other than goal ratio is itself.

** I should note that this method was actually developed and first used by Vic Ferrari. See here.

Addendum

Scott Reynolds had a question in the comments section on how the results would differ if we looked at future EV performance rather than overall performance. Using the same method as the one described above, I looked at which of EV Corsi Tied and EV goal ratio (empty netters removed) was better able to predict future performance at even strength (which I operationalized as future EV goal ratio). Here are the results:


The results aren't too different - Corsi Tied is a much better predictor early in the schedule, but the two measures have about the same predictive power by the end of the year.
.
.
.

Sunday, February 27, 2011

EV Data for Games 1-939



[If you're having difficulty viewing the document, click here to view the spreadsheet directly at googledocs.]

Notes

The document contains three worksheets. The first sheet shows even strength data for all situations. The second shows even strength data for when the score was close (i.e. whenever the score margin was 1 or 0 in the first two periods, or tied in the third period or overtime). The last sheet contains data for when the score was tied.

Empty net goals have been removed from the data.

Missing Games List

Game 124 - WSH@CAR
Game 429 - ATL@NYI

Abbreviations

GF: goals for
GA: goals against
SF: shots for, where shots = goals + saved shots
SA: shots against
SHOT%: shots for/(shots for + shots against)
SH%: shooting percentage
SV%: save percentage
PDO: shooting percentage + save percentage
FF: fenwick for, where fenwick = shots + missed shots
FA: fenwick against
F%: fenwick for/ (fenwick for + fenwick against)
CF: corsi for, where corsi = shots + missed shots + blocked shots
CA: corsi against
C%: corsi for/ (corsi for + corsi against)

Wednesday, February 16, 2011

Shots, Fenwick and Corsi

From time-to-time, I'll find myself surfing aimlessly throughout the hockey blogging world in search of articles, discussions, and other interesting stuff. In doing so, I'll occassionally find that others have referenced or linked to my blog. Typically, the link or reference will relate to the even strength data that I've been publishing periodically throughout the season.

There seems to be a decided preference for EV tied and EV close data over the raw numbers. That makes sense - the raw data is subject to score effects, which makes the information less valuable with respect to distinguishing good teams from bad ones.

Interestingly, however, there doesn't appear to be a general agreement as to which of shot, fenwick and corsi percentage serves as the best metric to use once score effects have been controlled for. While fenwick seems to be the most popular, there are some who like corsi, and there are even a few prefer shot percentage over both. This raises the question: which of the three measures ought to be looked to for the purpose of team evaluation?

As Gabe Desjardins once correctly observed, there is a stronger relationship between fenwick and winning percentage than there is between corsi and winning or between shot differential and winning. In fact, according to Gabe's numbers, the correlation between corsi and winning percentage was about the same as the correlation between shot differential and winning percentage, even though including blocked shots substantially increased the sample size. The upshot is that the inclusion of blocked shots in the analysis doesn't add much information.

Gabe's discovery may account for the slight preference towards fenwick discussed above.

However, the weaker relationship between corsi and winning can be partially accounted for by score effects. In particular, the trailing team does better in terms of corsi than it does with respect to either shot percentage or fenwick.


As such, while overall corsi has a lower correlation with winning than overall fenwick, the same may not hold with respect to score tied corsi and score tied fenwick.

In an attempt to resolve this issue, I performed a series of calculations, the results of which have been posted below.


This table shows the split-half reliabilities for score tied corsi, score tied fenwick and score tied shot percentage. The split-half reliabilities for each variable were calculated by randomly selecting 40 games, randomly selecting an independent group of 40 games (that is, a game chosen in one group was necessarily excluded from the other), and using the two data sets to determine the correlations for each variable. This was repeated 1000 times, with the above table showing the average values.

Not surprisingly, corsi is more reliable than either fenwick or shot ratio at the half-season level, which is a product of the fact that there are simply more corsi events then fenwick or shot events in our sample. Thus, corsi should prima facie be considered the superior metric of the three due to its superior reliability.

Ignore the goal ratio column for now - it's only been included for the purpose of performing a subsequent calculation.


This table shows the predictive validity of the same three variables with respect to overall goal ratio. Here, predictive validity was determined by randomly selecting 40 games, calculating each team's score tied corsi, fenwick and shot percentage within that sample, and looking at how each variable correlated with overall goal ratio in an independently selected 40 game sample. As with the first table, the numbers here are the averaged values over 1000 trials.

The predictive validity of each variable is commensurate with its reliability co-efficient, with corsi having the most predictive validity. In other words, a team's score tied corsi over a 40 game sample is a better indicator of how it will perform over the remainder of its schedule than is score tied fenwick or score tied shot percentage.

Of course, the fact that corsi has the most predictive validity in practice doesn't necessarily mean that it serves as the best measure of team skill in theory. As discussed in a previous post, the observed correlation between two variables is contingent upon the reliability with which each variable can be measured. Fortunately, there exists a formula that can be used to calculate what the correlation between two variables would be if each could be measured with perfect reliability. That formula involves dividing the observed correlation by the product of each variable's reliability co-efficient.

r xy adjusted = r xy observed/ SQRT( reliability x * reliability y)

As we already have the split-half reliability co-efficients for all of the variables, we only need to determine the split-half correlations between score tied corsi, score tied fenwick and score tied shot percentage, on the one hand, and goal ratio, on the other.

After inputting all of the relevant variables into the above formula, the following values are obtained:


Therefore, while corsi has more predictive validity with respect to goal ratio at the within-season level, fenwick and shot percentage appear to correlate more strongly with goal ratio over a sufficiently large sample of games. In other words, in theory, both fenwick and shot percentage seem to serve as better measures of team quality than corsi does.

One caveat: the differences between the values here are small, and we only have three seasons of data. It may very well be that all three variables correlate equally well with goal ratio over the long run. This subject may require further study in the future when more data is available.




Friday, February 11, 2011

EV Data for Games 1-820



[If you're having difficulty viewing the document, click here to view the spreadsheet directly at googledocs.]

Notes

The document contains three worksheets. The first sheet shows even strength data for all situations. The second shows even strength data for when the score was close (i.e. whenever the score margin was 1 or 0 in the first two periods, or tied in the third period or overtime). The last sheet contains data for when the score was tied.

Empty net goals have been removed from the data.

I didn't make an adjustment for schedule difficulty this go around because - and this is embarassing - when I recently performed a system restore, I forgot to transfer that particular file to my external hard drive. Having said that:

1. There seems to be more interest in the raw numbers.

2. The schedule adjustment would be negligible for most teams at this point in the season.

Missing Games List

Game 124 - WSH@CAR
Game 429 - ATL@NYI

Abbreviations

GF: goals for
GA: goals against
SF: shots for, where shots = goals + saved shots
SA: shots against
SHOT%: shots for/(shots for + shots against)
SH%: shooting percentage
SV%: save percentage
PDO: shooting percentage + save percentage
FF: fenwick for, where fenwick = shots + missed shots
FA: fenwick against
F%: fenwick for/ (fenwick for + fenwick against)
CF: corsi for, where corsi = shots + missed shots + blocked shots
CA: corsi against
C%: corsi for/ (corsi for + corsi against)

Wednesday, January 26, 2011

Even Strength Outshooting and Team Quality

Readers familiar with the team even strength data that I've published over the course of the season might wonder why I seem to place a large amount of emphasis on even strength shot ratio with the score tied.

After all, only about 35% of league play occurs with the score tied. And of that 35%, one-fifth consists of special teams play. Taken together, that means that time played at even strength with the score tied represents less than 30% of a typical NHL game.

Indeed, if we examine the relationship between a team's even strength shot ratio with the score tied and it's overall goal ratio for every season since the lockout, we find an average correlation of 0.556, meaning that even strength shot ratio only accounts for roughly 30% of the variance in outscoring with respect to a single NHL season.



However, because goals in the modern-day NHL are relatively rare events, a substantial proportion of the team-to-team variation in seasonal goal ratio can be attributed to luck. For example, random variation accounted for 47, 35, and 41 percent of team variation in goal ratio in 2007-08, 2009-10 and 2009-10, respectively.

In a hypothetical season with a sufficiently long schedule, that random variation would eventually disappear, leaving each team with a goal ratio commensurate with its abilities. What each team's goal ratio might look like in such a scenario can be approximated by taking its seasonal statistics - namely, shot ratio, shooting percentage and save percentage - and adjusting them to account for the extent to which each one is affected by random variation.* For both shooting and save percentage, the adjustment is significant as luck accounts for a majority of the variation in respect of both over the course of a single season, as indicated in the table below.


For shot ratio, however, the adjustment is less severe as the impact of randomness is comparatively smaller. Consequently, as the sample size increases, so too does the correlation between shot ratio and goal ratio.

If this exercise is performed for each post-lockout season, one is able to determine the relationship between true goal ratio and even strength shot ratio with the score tied. The results:**


Therefore, in an imaginary league in which luck is a complete non-factor, EV shot ratio with the score tied would account for roughly 65% of the variance in outscoring. In other words, even though the two variables may not be strongly correlated over the course of a single season, a team's EV shot ratio with the score tied serves as a reasonably good indicator of how it can be expected to perform over the long run. This is especially true for the three most recent seasons, in which EV shot ratio accounts for 75% of the variation in outscoring ability. It seems that as the level of parity between teams has increased, even strength shooting has become even more important.

Finally, the remaining 35% of outscoring variance indicates that there are other sustainable components of team success. Apportioning the remaining proportion of the variance between these components gives us an idea of their relative importance.


As special teams ability and EV tied shot ratio are correlated variables, residual special teams skill refers to the proportion of special teams skill that cannot be accounted for by EV tied shot ratio. Residual specials teams skill accounts for about 49% of the remaining variance.

Similarly, residual EV shot ratio refers to the proportion of even strength outshooting that cannot be predicted by EV shot ratio. This accounts for 7% of the remaining variance.

The rest of the remaining variance is explained by even strength shooting, even strength save percentage and residual variance. Residual variance is the amount of variance left over after subtracting the sum of the other four components from 1. It results from the fact that the four components are not uncorrelated, independent variables.



* even strength and special teams statistics were, of course, treated separately for this part of the analysis

**There is an alternative calculation that can be applied as a check on the correctness of these values. As the seasonal reliability of both goal ratio and EV tied shot ratio is imperfect, it is necessary to upwardly adjust the observed correlations between the two variables in order to ascertain their 'true' relationship - that is, the correlation that would result if each variable was perfectly reliable.

The adjustment involves dividing the observed correlation by the square root of the product of each variable's reliability co-efficient. In other words

r adjusted = r observed/ SQRT( reliability EV tied shot ratio* reliability goal ratio )

The application of the above formula involves determining the reliability co-efficients for each variable, which can be calculated as follows:

reliability = 1- [(1- split half reliability)/SQRT(2)]

If these formulae are applied with respect to each post-lockout season, the following results:


The average adjusted correlation is 0.81, which is comparable to the average adjusted correlation obtained through the first method (0.804). It should be noted that this second method is likely to slightly overestimate the true correlation, given that the two variables are not truly independent.

EDIT: Accidentally used Fenwick ratio instead of Shot ratio when determining observed correlations for 2007-08, 2008-09 and 2009-10. Table and accompanying discussion has been edited accordingly.

EDIT 2: In re-thinking the method used in the alternative calculation, it occurred to me that the better way to adjust the observed correlations would be to calculate all three input values at the half-season level.

There's no sense in using the split-half reliabilities in order to estimate the full reason reliabilities for EV shot ratio and goal ratio when the split-half reliabilities can be used themselves, given that the split-half correlation between EV shot ratio and goal ratio is readily ascertained.

This approach produced the following results.*

* the half-season values were calculated through randomly selecting 40 games, randomly selecting another 40 games without replacement, and determining the correlation between the relevant variables across the data sets. This was repeated 1000 times, with the average values used.



.
.

Wednesday, January 19, 2011

EV Data for Games 1 - 692

The spreadsheet that should appear below contains detailed EV data at the team level for games 1 to 692. As NHL.com has inexplicably failed to publish play-by-play data for the following games, they were not included:

Game 124 - WSH@CAR
Game 429 - ATL@NYI
Game 491 - PHX@PIT*

*The play-by-play feed for this game was initially available, but is no longer accessible. Consequently, I have EV and EV close data for this game, but no EV tied data.



[If you're having difficulty viewing the document, click here to view the spreadsheet directly at googledocs.]


A couple of points:

The document contains three worksheets. The first sheet shows even strength data for all situations. The second shows even strength data for when the score was close (i.e. whenever the score margin was 1 or 0 in the first two periods, or tied in the third period or overtime). The last sheet contains data for when the score was tied.

Empty net goals have been removed from the data.

The abbreviations are defined as follows:

GF: goals for
GA: goals against
SF: shots for, where shots = goals + saved shots
SA: shots against
SHOT%: shots for/(shots for + shots against)
SH%: shooting percentage
SV%: save percentage
PDO: shooting percentage + save percentage
FF: fenwick for, where fenwick = shots + missed shots
FA: fenwick against
F%: fenwick for/ (fenwick for + fenwick against)
CF: corsi for, where corsi = shots + missed shots + blocked shots
CA: corsi against
C%: corsi for/ (corsi for + corsi against)
ADJ: refers to the fact that an adjustment for schedule difficulty has been made. See here for the details of the adjustment process.

Finally, as mentioned earlier, I've included EV tied data in this go around. The reason for that is that it appears that score effects are still very much relevant when the score margin is 1 in the first two periods. For example, consider the table below which shows how teams trailing by one goal in the first two periods have performed with respect to shot percentage and PDO over the last three seasons.

In other words, teams trailing by one goal in the first two periods tend to play more aggressively, which increases their shot differential yet hurts their PDO. Additionally, there appear to be strategic differences between teams with respect to style of play when the score margin is one in the first two periods. That's a topic that I plan to explore in more detail in the upcoming weeks.

The result of all of this is that Fenwick or Corsi percentage with the score tied should be a better measure of a team's true ability to control the play at even strength, given that EV score close shot statistics tend to favor teams that play more from behind and/or play more aggressively, relative to the average team, when leading or trailing by one in the first two periods.
.
.
.