Yet we in the analytics community still try - sporadically, at least - to evaluate goalkeepers using what data we have. Thing is, we have yet to find any element of a goalkeepers performance that is in any way repeatable.
Save% isn't repeatable. Save% adjusting for location isn't repeatable. Goals Saved Above Average isn't repeatable. We got nothing. Maybe that nothing is due to the poor quality of data we have? Or maybe that nothing is merely due to the low number of events (shots) a 'keeper faces every year.
Either way, I'm at a loss in terms of evaluating goalkeepers. This will be my last look at anything to do with them. Damn them, I say!
Table
Our data set contains info on all goalkeepers to play in the Premier League from seasons 2010/11 to 2014/15. The table below contains 3 different 'keeper stats, the number of games played and the age of the player during that season.
From this table we can look into age curves and repeatable performance.
(feel free to use this data)
Now, the sharps among you will have noticed that there's a ton of goalkeepers in that data set who weren't starters and played a very small number of games.
For the moment, I am going to discard any goalkeeper who wasn't a starter (15 games is my fairly harsh cut-off) and visualize the data found in the table above.
The Good Guys
The 'keepers in this chart are the starters who posted above average save% seasons.
Looking toward the top right of the chart we can see some familiar names: Cech, Hart, De Gea, Mignolet, and Begovic appear on at least two occasions.
The other curious item of interest is the age of some of the 'keepers who posted outstanding save% seasons: There's a lot of young guys in that list. Yes, we see appearances by Cech, Jaaskelainen, and Howard, who were all in their 30's, but the majority of the guys in that top right quadrant are young.
De Gea at ages 20 & 21. Hart at ages 23 & 24, Alex McCarthy at age 22, Szczesny at age 22, Begovic at 23, Mignolet at 24, Adrian at 26, Foster at 27. It's a curious thing, more on this later.
The Bad Guys
There's some really bad single seasons of save% posted here (here's to you, Robinson). Tim Krul, Carson, and Szczesny are three young guys who posted poor seasons of save%, but the majority of the guys in the bottom left of this chart are late 20's or early 30's goalkeepers.
I've been hinting at a slight link between age and outstanding seasons of save%, so what of it?
Well, gradually I am coming around to the idea that goalkeepers peak years are not in their 30's, as has long been held true, and that the actual peak for goalkeepers may well be closer to the peak of outfield players which is ages 23 -28. Two excellent pieces of hockey writing were the catalyst for this thinking:
Colby Cosh Why Goalies Can't Save Themselves
Garik16 How Well Do Goalies Age
I recommend reading both articles but the short of them is this:
Now, hockey isn't football. They are different games, with very different dynamics. Not to mention hockey has incredibly large amounts of data by which to evaluate goalies.
Interestingly, the narrative around hockey goalies and peak age was almost identical to the narrative we currently see in football: goalies constantly improve and continue to do so into their late 20's/early 30's; the decline in physical attributes if more than off-set by improved positioning, awareness and decision making - a learning of the game, basically.
This "constant improvement" and "goalkeepers peak later" narrative may be just as nonsensical for football as it turned out to be in hockey. Guys who are good late in their careers were likely outstanding in the peak of their careers. Guys who were average in the peak of their careers, in most cases, probably don't get better, no matter how much you learn about the game.
This illustration from Eric Tulsky helps us to understand that:
Having said all that, there is no real way for me to prove, with data, that a goalkeepers peak years are at a much earlier age than is currently thought. Save% tells us nothing. Save% and GSAA plotted on an age curve tells us nothing.
Is there any other way that we can at least hint that a goalkeepers peak years are at an earlier age than currently thought?
Age & Usage
All Goalkeepers
The graph below shows us the percentage of players (count%) and the percentage of games (games%) for each age group.
We see a rough peak in both count and usage of 23 -29 years of age. What happens if we remove backups (less than 15 games) from this chart?
We see a rough peak in both count and usage of 23 -29 years of age. What happens if we remove backups (less than 15 games) from this chart?
Full Page Link
Starters Only
If we look at just starters the rough peak that we saw in the graph above becomes a definite peak in the graph below.Usage and count both tell us that the peak is 23 - 29 years of age. Young goalkeepers don't play much and goalkeepers from 30 onward are an increasingly rare breed. Naturally, the very best, or the most durable, goalkeepers survive and flourish well into their 30's.
Raw Count
Full Page Link
So, do either of these usage age curve charts tell us anything about the peak years of goalkeepers? It tells us a little, at the very least.
In a perfect world we could rely on save% or any other metric to tell us what a goalkeepers peak is and thus we could use that information to break down the current narrative about goalkeepers peak years being the late 20's and early 30's. But we can't use Save%.
Usage and count by each age group may well be the best method we have, and the best method we may ever get, to determine a 'keepers peak years.
If the current narrative is "Goalkeepers continue to improve and then peak in their late 20/early 30's" then why doesn't it show up in the way managers use their goalkeepers? Why don't we see that many starting goalkeepers in their early 30's?
Because those goalkeepers are declining, and their managers can spot it.
Managers - oft-wrong in the eyes of fans - aren't actually dummies. Top level managers didn't just stumble into the dugout and start managing by chance, they are there because they are likely experienced and talented enough to have earned the chance to be there. Knowing this, I'm inclined to trust most managers unless given overwhelming evidence not to do so. And I think, in general, managers are using their goalkeepers in the correct manner.
It is important to remember that a few exceptional talents (Cech, Van der Sar, Schwarzer, James, Buffon etc.) who played and excelled at an older age doesn't make playing well into one's 30's the norm. Those guys don't get to push back the peak age for goalkeepers all by themselves. The exceptions I mentioned are the goalkeeping equivalent of Terry and Lampard, of Scholes and Pirlo. Brilliant in their 20's and still above average well into their 30's.
But they are still the exceptions, not the norm. The norm is made up of talented 20 somethings with an overall skill set that is already declining once into their 30's. Usage, percentage of games played and count agrees with this.
I just wish we had a trustworthy objective performance metric to back up what we see in the age curve charts that are shown above.
Repeatability
A quick and dirty repeatability test for guys who played 5 games or more in consecutive seasons in the Premier League.
There's not a lot here, folks.
Save%
GSAA
GSAA p90
Final Thoughts
We don't have a trustworthy metric to objectively evaluate goalkeepers. 5 years of data is insufficient and the number of shots a goalkeeper faces in a single season is insufficient.
We have nothing right now.
Of all these words and charts and pretty pictures, the only thing of value is the age curve of goalkeepers and how this doesn't jive with the current narrative about goalkeepers, gradual improvement, and peak years of the late 20's/early 30's.
I was a goalkeeper throughout my 20's, and I absolutely loved it. I relished the challenge, the mental pressure, the amount of thought that went into thinking how to play the position. But it's not an easy position to play, and it's an even harder position to evaluate by number or by eye.
'Goalkeepers are voodoo' - Everyone.
