Are goalkeepers prisoners of defense? A statistical look at MLS

Tim Howard had the greatest game ever…in a loss.

Effusive praise in other American sports is reserved solely for winners. Had Curt Schilling not won Game 6 he would not live in Boston lore. Michael Jordan’s buzzer beater in ’89 would have been forgotten if it had merely tied the game.

Yet, Tim Howard has become the most recognized US soccer player for his 15 save performance against Belgium. Howard kept the Belgians at bay for more than 90 minutes, in a game the US did not deserve to win. Perhaps collective wisdom is instructive in this case. Howard was merely at the mercy of his defense and midfield. He did all he could.

In 2012, San Jose won the regular season with the second worst goalkeeping in the league. That same year, Kansas City had the best goalkeeping and finished second. San Jose scored 15 more goals than anyone to compensate for a save percentage of 62.9%. Kansas City only scored 42 goals, the fewest among winning teams, while sporting a 75.4% save percentage. From this it seems odd to suggest that goalkeepers matter very little or are all that matters in soccer. However, it does open the question how much does a goalkeeper matter in soccer?

Soccer is a game of scoring goals and not giving them up. Score more goals than your opponent and you win the game, do not give up a goal and you are guaranteed one point.  Goalkeepers can have an offensive role; players that are adept at passing can help build possession which can lead to goals. However, goalies are paid to keep the ball out of the back of the net.

The clearest measure of that ability is save percentage. Make a save and your save percent goes up.  For the last two MLS seasons, the league average and median save percentage is 68%. Save percentage is the pitcher v. hitter measure of soccer, it’s the goalie against the attacking player.

In 2012-2013, there was a negative correlation between goals conceded and save percentage. In simpler terms, as save percentage increases, goals conceded decreases. When you run the numbers, you find an r-squared value of around 40%. Essentially that means that 40% of the change in the data is directly related to the relationship between save percentage and goals scored.

However, save percentage is not the most accurate predictor of goals conceded. Examination of other widely reported statistics like saves, shots, and shutouts yields intriguing results. The strongest predictor of goals conceded is shots. In many ways this is a simple finding. The more shots against you the more goals you will concede, but the numbers are more nuanced than that. When you run the correlation between save percentage and shots you find a small negative correlation. This suggests that not only do the quantity of shots increase, but also do the quality of shots. The correlation coefficient between shots and goals conceded is just under .8, and results in an r-squared value of 64%. Which again suggests that 64% of the change in goals conceded is directly related to shots.

This may suggest that goalkeepers can be prisoners of talent deficiencies beyond their control. Facing more shots results from strikers failing at holding onto the ball, midfielders losing battles to the other team, and the back line struggling with a number of issues. With the exception of rebounds, the number of shots the goalkeeper faces is wholly dependent on the strength of those playing in front of him.

Now this is not to excuse poor play on the part of goalkeepers, or to say that goalkeepers are not important. By quantifying the impact goalkeepers have on goals allowed, you can discern real player values, and offer insight into the value of positions. MLS has a fixed salary cap. In European soccer, if you overpay for a goalie you have just wasted money. If you overpay in MLS, you have sunk more than just that position. It’s much like the MLB draft signing bonus cap, budgeting  is the strategy.

On the issue of player salaries, the research shows no correlation between goalkeeper salary and goals conceded. To run this test I included all goalies with more than six appearances and added their salaries together to create a single team value. There was a very small, statistically insignificant, negative correlation (r=-.08). While this may suggest that the optimum strategy would be to pay goalkeepers $70,000 or less a year, I think such analysis is overly simplistic. The findings offers convincing evidence that a designated player goalkeeper is not a wise investment, and that paying a goalie $250,000 is also probably not prudent. However, these numbers may not be totally reliable.

In the end, I sought to quantify the impact that goalkeepers have on goals conceded, and use that to rate players. Much like in other sports, I will use the concept of a replacement player. Unlike other sports, replacement value in soccer is tricky. Considering that players move freely between higher and lower quality leagues the baseline of what an easily acquired player performs at becomes murky. For goalkeepers I settled on the 25th percentile for save percentage, 64%. Thus, any percentage above 64% is above replacement value.

In measuring goals allowed below replacement (GABR), I took save percentage above replacement times shots faced and then multiplied by 40%. 40% being roughly the relative weight of the r-squared values between save percentage and shots faced. With this I believe you get an accurate measurement of how many goals a particular keeper could have saved and did not, or did save that your average available goalkeeper would not. With GABR, a positive value indicates the goalkeeper allowed that many goals fewer than replacement value, negative values indicates how many goals more than a replacement player was allowed.

In 2012, the GABR leader was Andy Gruenebaum of the Crew at +7.104, the worst was Donovan Ricketts of the Timbers at -1.81. In 2013, Donovan Ricketts was the leader at +4.55, while Carlo Cudicini was the lowest performer at -1.81.

Players needed to make 14 appearances to be listed.

Gruenebaum (CLB) +7.104
Johnson (CHI) +6.592
Hamid (DC) +6.272
Nielsen (SKC) +4.688
Gspuring (SEA) +3.88
Rimando (RSL) +3.504
Pickens (COL) +3.12
Reis (NE) +2.72
Perkins (MTL) +2.512
Hall (HOU) +2.176
Saunders (LAG) +1.792
MacMath (PHI) +1.616
Hartman (DAL) +1.312
Cannon (VAN) +1.296
Kennedy (CHV) +.848
Kocic (TOR) -.224
Busch (SJ) -.688
Ricketts (POR) -1.81

Ricketts (POR) +4.544
Rimando (RSL) +4.128
Busch (SJ) +4.028
Robles (NYRB) +3.568
Johnson (CHI) +3.408
Gruenebaum (CLB) +2.8
MacMath (PHI) +2.768
Irwin (COL) +2.608
Fernandez (DAL) +2.176
Hall (HOU) +1.904
Nielsen (SKC) +1.6
Hamid (DC) +1.536
Gspurning (SEA) +1.216
Shuttleworth (NE) +1.024
Bendik (TOR) +.416
Kennedy (CHV) -.896
Cudicini (LAG) -1.81

Returning back to the issue of correlation between player salary and goals allowed, in 2012, second and third place went to young USMNT prospects Bill Hamid and Sean Johnson. The inclusion of players early in their respective careers who clearly have above average talent, could make them outliers.  Outliers that would skew the correlation, particularly considering the sample size. However, even when you change Johnson and Hamid to league average salary, the correlation only gets marginally weaker and is still far from statistical significance.

In 2013, Ricketts, the starting Jamaican goalkeeper, and Rimando, the only US based US World Cup goalkeeper led GABR. The presence of international quality goalkeepers at the top of the list may suggest that as MLS grows, the difference in the quality of midfielders and defenses may be on the wane. As quality of chances becomes more uniform, the quality of goalkeeping becomes of paramount importance.

This is a useful measure for determining what individual goalkeepers are worth, but still leaves us far from resolving our first issue. Are goalkeepers merely prisoners of fate?

To measure this, I created a combined statistic that primarily uses shots faced and save percentage. It also includes an additional statistic that measures penalties saved and shutouts, but that statistic is only about five percent of the whole coefficient. When examining this created coefficient, the correlation between it and goals conceded was r=.916. This was the strongest correlation found in my efforts. It suggests that around 84% of the data is linked between the coefficient and goals conceded. In exploring correlations between the various factors within the coefficient, when the shots faced and save percentage were given equal weight, the correlation was nearly the strongest found.

Using this metric, in 2012, the best team at preventing goals was Sporting Kansas City, and the worst was Chivas USA. In order they were:

Sporting Kansas City, Real Salt Lake, Seattle, Houston, LA Galaxy, Chicago, San Jose, Vancouver,  Philadelphia, New England, Colorado, Montreal, DC United, Portland, Columbus, New York Red Bulls, Dallas,  Toronto, Chivas USA

Tim Howard had a wonderful game. His performance is unlikely to be forgotten.The answer is clearly that goalkeepers are not prisoners of fate. Based on the available evidence it suggests, in most cases, goalkeepers are directly responsible for around 40% to 60% of the total goals conceded.

Prior to the World Cup, commentators suggested Howard would be headed to MLS after Brazil. While that star power may be alluring, the numbers are clear: It’s not the right investment for an MLS club.

For all the pageantry, emotion, punditry, and shenanigans soccer is a game of simple numbers.

Goals scored and goals allowed.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s