19 August 2016

Zach Britton is the 2nd Best Pitcher in the American League

Begrudgingly, nearly everyone in baseball data science will say that they do not completely comprehend the value of relief pitching.  Often, you will hear how dominant relievers are incapable of starting.  That is pretty much true.  Often, you will hear that closers are not employed at the most consequential moments and often are given a clean slate when entering a game.  That is also true.  Often, you will hear how utterly confused analysts are when a reliever is paid big money, given a lot of years, or is acquired in exchange for multiple, notable prospects.  Often, you will hear how several teams put together dominant bullpens that are collections of spare, rubbish heap arms, which is also true.

If you simply read what most data analysts say, relief pitching continues to be overrated and dominant relievers are failed starters.  Of course, this statement seems a little silly, right?  How many failed starters are there each year at the MLB level?  At least 30.  Very few of them become dominant relievers let alone dependable middle relief arms.  There certainly is something more to it.  Relief arms tend to need one or two above average pitches and at least mediocre control.  The reality is that this is not a common thing among those failed starters.

Furthermore, maybe our ability to appreciate relievers with metrics like WAR simply measures their value inadequately.  In this post, I try to look at things from a couple different angles.  First, a somewhat traditional way looking at saves and blown saves.  To do this, I took the players with at least 60 save opportunities from 2013 to 2015 and selected the top 20 players.  I assumed replacement level closing was the average success rate of the bottom 5 of those top 20 closers.  This might actually be an overestimation simply because to rack up 60 save opportunities, you have to be an arm that a club has been devoted to.

What we find is that the bottom rung long term closer blows 6.5 games per 40 opportunities (84% success) every year.  A top five closer blows 3.1 games per 40 opportunities (92% success) every year.  That is a difference of 3.4 games.  A blow game does not mean a loss.  I would guess that a blown save in a closing situation may be a loss 70% of the time, which would mean that the bottom run closer would lose 2.4 more games per year.  At a cost per win of 7 MM, that would suggest on average that the elite closing arm is worth about 18 MM, which suggests that maybe closers are somewhat undervalued in the market.

Zach Britton is currently 37 for 37.  Based on the above, we would expect a bottom run closer to blow 5.92 games over that stretch.  If we depreciate those blown saves in a conversion over to losses, then we have 4.1 losses.  This would suggest that Britton has actually been worth over 4 wins so far this season.  This would nestle him right behind Corey Kluber's 4.3 fWAR for second in the AL among all pitchers.

Maybe the simplicity of looking only at save opportunities leaves your brain unmoved and your heart cold.  Well, we can dive into RE/24.  RE/24 looks at run expectancy before and after each event in a game and attributes those to a pitcher without consideration of anything other than run expectancy.  A starting pitcher can benefit simply by racking up successful innings and a reliever benefits by coming in to high leverage situations.  RE/24 often is most unfair to closers who tend to enter the game with a clean slate and somewhat indulgent to middle relievers who successfully enter games with men on base.

Anyway, I batched all starters together and all relievers together for each team, ran those variables along with RE/24 for team batting, and regressed all of that against team wins.  What I found is that relief RE/24 was 78% the value of starting pitcher RE/24, which is not accounted for with innings pitched.  I then took the RE/24 AL pitcher leaderboard and scaled down relief pitcher RE/24.  Next, I accounted for park factors in home and away stadiums as well as the defensive ability for each team.  What resulted was a RE/24(x) metric that I created.  Here is that leaderboard:


Britton shows up as sixth on this board.  He is not exactly challenging the leaders much, but he still shows he is in the conversation for Cy Young.

Of course, the argument might wind up being that while Britton excels at closing, so would several of the other pitchers on this board.  One way to look at that would be to see what exactly the impact of higher velocity might have on a pitcher's success.  An increase of 1 mph in general decrease a player's FIP by about 0.40.  Not all starters when pressed into relief roles enjoy an increase in velocity, but lets be kind and simply assume all starters would see a jump of 3 mph that suggests a FIP improvement of 1.20.

Our leaderboard using the players above would yield:

1 Corey Kluber 1.81
2 Zach Britton 2.00
3 Aaron Sanchez 2.09
4 Danny Duffy 2.13
5 Jose Quintana 2.22
6 Michael Fulmer 2.26
7 Chris Sale 2.29
8 Cole Hamels 2.46
9 Justin Verlander 2.46
10 J.A. Happ 2.69
11 Marco Estrada 2.96
12 Chris Tillman 3.07
All of this tends to suggest that maybe all those who are upset with Zach Britton being in the Cy Young conversation might be relying a bit too heavily on unsteady data science analysis with respect to relief pitching.  Many analysts may be harping loudly about concepts and ideas that have been firmly in place within the sabermetric community for over a decade and may be forgetting that this is an area of baseball that has yet to firmly establish what exactly the value is of a closer in a definitive way.

Perhaps in the coming years, data science will find ways to measure relief pitching quality that has a more substantial methodology than we currently have.  Maybe that methodology will show that closing is not the realm of inadequate pitchers, but more perhaps something completely different.  The skill set needed to be an elite pitcher differs from those who start and maybe that means these are truly two different positions within the pitcher class instead of a first and second tier.

Or maybe not.


Roger said...

How about if you measure FIP across every starting pitcher's first inning? Or first two innings? Even though I realize many SP try to go easy in the first inning to save energy for later, any SP that can't get out of the first inning will be a lousy reliever. However, you can still be a passable SP with shaky first innings and better 2-6 innings. I also realize that eliminates the idea of a leveraged situation, too, but it might be an interesting factor.

Another measure might be how well pitchers do the first time through a lineup. Those that drop off significantly the second time through might make better relievers. Those that drop off the third time through might make better swingmen.

I think a HOF type pitcher will be that in any given situation. They are just better than other pitchers. Types such as Smoltz and Eck. A failed pitcher will also likely be a failed pitcher in any given situation. Both are ends of a bell curve. Somewhere in the middle you have pitchers who pitch short stints well and others who need more innings to achieve better results. That's where the real skill in evaluating metrics comes from.

Alan said...

The problem with the save statistic is that coming into a game in the 9th inning at the beginning of the inning with a 3 run lead and holding the lead is not really a big deal. There was a big study looking over 73 years of mlb stats up to around 2003, and all pitchers were able to be successful about 96% of the time in this situation of a 3 run lead after 8 innings. So, a save really should be broken down to 1 run leads entering the 9th inning (successful about 85% of the time in the 73 year analysis) and 2 run leads entering the 9th inning (successful about 94% of the time in the analysis). Here is the article that includes the results of the long study: http://www.espn.com/espn/page2/story?page=caple/080805

Jon Shepherd said...

why do we think the different closers had different class save frequencies? it makes as much sense to me as breaking down HR distances.

Anonymous said...

Why is it that your analysis resulted in only one reliever on the list? It's obvious the data you chose to employ is skewed in some way. I'm no statistician, but it doesn't require one to recognize and understand your methodology is severely flawed.

Jon Shepherd said...

It has only one reliever because relievers have far less opportunity to impact a game. you have to be an exceptional reliever to get a mention with the best guys who toss 200+ innings.