03 October 2013

Is Jim Johnson an Elite Closer?

Jim Johnson Saves | photo: Keith Allison

On a later date, Stuart Wallace will address the bullpen and that means his thoughts on what exactly to do with Jim Johnson.  As you may know, Johnson cost the team 6.5 MM last year as both he and the team avoided an arbitration hearing.  Johnson has one more year of free agency left and, based on his final 2013 numbers (2.94 ERA, 50 saves), he stands to see another increase in money.  Some might suggest non-tendering him or believe that the Orioles could take him to arbitration and win the greatest amount they could reduce his salary to (4.55 MM), but I think it won't happen that way.  Instead, I think it is likely that we see him earn about 33% more than he did last year.  Typically, your second to last arbitration date results in a salary worth about 60% of the market rate and the last one being 80%.  With that in mind, we should probably expect a 2014 salary of 8.67 MM.  For comparison, Mariano Rivera took a 5 MM cut in 2013 and played for 10 MM while Joe Nathan is on tap for 9 MM next year.  An 8.67 MM salary assumes Johnson is an elite closer and pays him just a little south of that mark.

Is he worth it?

To answer that question, it probably helps to figure out to what degree relievers in general perform.  We can do this by looking at some measure of WAR or FIP or ERA or whatever.  I decided to go a little more big picture here and look at successful conversion of save and hold opportunities.  So much ink, talk radio, and internet space had been devoted this past season to Johnson blowing saves that it seems somewhat proper to look at a collection of these binary events.  In other words, perhaps there is something special about game situations where a save is on the line to the extent that maybe it is not captured with other measures.  Additionally, perhaps with a large enough sample size we are able to see through the noise and measure something that is related to actual performance ability.

Below you will find a table of relievers from 2004 to 2013.  Included are players who managed to have the most save chances as defined by having the ability to blow a save.  This definition of course will include tight games in late innings as well.  Also included is a ranking of the 32 pitchers over the past ten years who have accumulated 100 or more saves.  The list includes 124 players and a good bit of analysis follows the table, so keep on reading.

Player Saves Holds Blown S+H Chances % RNK
Randy Choate 6 94 5 100 105 95.2
Jason Grilli 38 74 8 112 120 93.3
Sergio Romo 55 82 11 137 148 92.6
David Robertson 8 116 10 124 134 92.5
Javier Lopez 12 111 10 123 133 92.5
Pedro Feliciano 4 103 9 107 116 92.2
Greg Holland 67 28 9 95 104 91.3
Mariano Rivera 369 0 36 369 405 91.1 1
Joe Nathan 340 8 34 348 382 91.1 2
Sean Marshall 16 95 11 111 122 91.0
Craig Kimbrel 139 2 15 141 156 90.4 3
Grant Balfour 72 94 18 166 184 90.2
C.J. Wilson 52 46 11 98 109 89.9
Santiago Casilla 39 75 13 114 127 89.8
Joakim Soria 160 15 20 175 195 89.7 4
Eric O'Flaherty 0 103 12 103 115 89.6
Casey Janssen 65 37 12 102 114 89.5
Rafael Soriano 173 71 29 244 273 89.4 5
Sean Burnett 10 90 12 100 112 89.3
Kenley Jansen 62 37 12 99 111 89.2
Joel Peralta 11 137 18 148 166 89.2
Mike Adams 4 144 18 148 166 89.2
Arthur Rhodes 16 128 18 144 162 88.9
Brad Ziegler 32 79 14 111 125 88.8
Drew Storen 55 47 13 102 115 88.7
Trevor Hoffman 249 2 32 251 283 88.7 6
Jose Valverde 276 14 37 290 327 88.7 7
Takashi Saito 84 40 16 124 140 88.6
Aroldis Chapman 77 23 13 100 113 88.5
Brian Wilson 171 17 25 188 213 88.3 8
Joe Smith 3 117 16 120 136 88.2
Scott Downs 26 167 26 193 219 88.1
Brian Shouse 5 84 12 89 101 88.1
Jonathan Papelbon 286 7 40 293 333 88.0 9
Michael Gonzalez 56 89 20 145 165 87.9
Bob Wickman 111 5 16 116 132 87.9 10
Carlos Marmol 117 83 28 200 228 87.7 11
F. Rodriguez 302 81 54 383 437 87.6 12
Eric Gagne 80 11 13 91 104 87.5
Derrick Turnbow 65 39 15 104 119 87.4
Alan Embree 18 86 15 104 119 87.4
Mike MacDougal 44 46 13 90 103 87.4
Jason Motte 54 49 15 103 118 87.3
Matt Lindstrom 45 85 19 130 149 87.2
Jose Veras 26 83 16 109 125 87.2
Darren Oliver 5 97 15 102 117 87.2
Juan Rincon 3 85 13 88 101 87.1
Dan Wheeler 41 120 24 161 185 87.0
Scott Eyre 1 99 15 100 115 87.0
Joaquin Benoit 36 137 26 173 199 86.9
B.J. Ryan 114 25 21 139 160 86.9 13
George Sherrill 56 76 20 132 152 86.8
Billy Wagner 197 7 31 204 235 86.8 14
Damaso Marte 15 110 19 125 144 86.8
Brad Lidge 224 39 40 263 303 86.8 15
Jim Johnson 122 62 28 184 212 86.8 16
Ryan Dempster 87 5 14 92 106 86.8
Brian Fuentes 200 48 38 248 286 86.7 17
Chris Perez 132 23 24 155 179 86.6 18
Matt Thornton 23 170 30 193 223 86.5
Edward Mujica 41 68 17 109 126 86.5
Bobby Jenks 173 5 28 178 206 86.4 19
Ryan Madson 52 113 26 165 191 86.4
Akinori Otsuka 39 74 18 113 131 86.3
Todd Jones 135 28 26 163 189 86.2 20
Bob Howry 17 108 20 125 145 86.2
John Axford 106 25 21 131 152 86.2 21
Brandon Lyon 70 115 30 185 215 86.0
Luke Gregerson 16 132 24 148 172 86.0
Salomon Torres 55 68 20 123 143 86.0
Huston Street 234 18 41 252 293 86.0 22
Jeremy Affeldt 24 111 22 135 157 86.0
Justin Speier 7 85 15 92 107 86.0
Heath Bell 168 83 41 251 292 86.0 23
Jason Frasor 36 97 22 133 155 85.8
J.C. Romero 6 102 18 108 126 85.7
Trever Miller 6 84 15 90 105 85.7
Fernando Rodney 169 69 40 238 278 85.6 24
Andrew Bailey 89 12 17 101 118 85.6
John Grabow 6 95 17 101 118 85.6
Frank Francisco 73 67 24 140 164 85.4
Tom Gordon 48 97 25 145 170 85.3
David Aardsma 69 23 16 92 108 85.2
Guillermo Mota 9 94 18 103 121 85.1
Chad Cordero 127 10 24 137 161 85.1 25
Scot Shields 20 150 30 170 200 85.0
Joel Hanrahan 100 30 23 130 153 85.0 26
Kyle Farnsworth 50 108 28 158 186 84.9
J.J. Putz 189 59 44 248 292 84.9 27
Hideki Okajima 6 84 16 90 106 84.9
Todd Coffey 11 78 16 89 105 84.8
Tyler Clippard 33 110 26 143 169 84.6
Mike Timlin 25 73 18 98 116 84.5
Francisco Cordero 304 23 61 327 388 84.3 28
Matt Guerrier 6 117 23 123 146 84.2
Jesse Crain 4 124 24 128 152 84.2
Jose Mesa 72 27 19 99 118 83.9
Rafael Betancourt 73 138 41 211 252 83.7
Jon Rauch 62 97 31 159 190 83.7
Danys Baez 83 48 26 131 157 83.4
Jason Isringhausen 170 26 40 196 236 83.1 29
Shawn Camp 12 81 19 93 112 83.0
Kevin Gregg 177 18 40 195 235 83.0 30
Ryan Franklin 84 47 27 131 158 82.9
Juan Oviedo 92 29 25 121 146 82.9
Chad Qualls 51 159 44 210 254 82.7
Scott Linebrink 8 148 33 156 189 82.5
Eddie Guardado 71 33 22 104 126 82.5
LaTroy Hawkins 57 129 40 186 226 82.3
Jonathan Broxton 111 84 42 195 237 82.3 31
Joe Borowski 96 20 25 116 141 82.3
Luis Ayala 14 96 24 110 134 82.1
Matt Capps 138 36 38 174 212 82.1 32
David Weathers 61 69 29 130 159 81.8
David Hernandez 19 65 19 84 103 81.6
Armando Benitez 92 14 24 106 130 81.5
Jamie Walker 8 75 19 83 102 81.4
Brandon League 74 51 29 125 154 81.2
Bobby Parnell 36 54 21 90 111 81.1
Kerry Wood 63 39 24 102 126 81.0
Octavio Dotel 81 63 34 144 178 80.9
Matt Belisle 5 94 24 99 123 80.5
Manuel Corpas 34 60 23 94 117 80.3
Aaron Heilman 16 96 31 112 143 78.3
The list above is a familiar list.  All of them, at one time or another, were consider among the elite relievers.  They were players teams wanted at the end of games.  For those reasons, it is important to recognize that the above population is a subset of relievers.  Jim Johnson's 86.8% save rate is in the middle, but it is in the middle of all of these top flight relievers.  In other words, an average value here does not necessarily mean that a pitcher is average.  He is average amongst very good relievers, which is a valuable thing.

Below is a table showing the general percentiles and average for the entire 124 players above.

All Relievers
Percentile %
50th 86.4
67th 87.4
Average 86.3
StDev 3.1
Johnson's 86.8% is a shade above average, but still well within the standard deviation for this population.  He does not appear to be exceptional so far based on these numbers.  You may contend that Johnson is a special case where he has risen to a new level of ability over the past two years.  In that case, you may prefer to look at his 89.4% success rate from 2012 to the present.  In that case, he is on the cusp of being an exceptional reliever and certainly is a first division closer (I consider anything above the 67th percentile to be a first division closer).

Looking more closely at the closer who accumulated at least 100 saves:

Closers >100 Saves
Percentile %
50th 86.8
67th 87.7
Average 86.7
StDev 2.3
For his career, Johnson is at the 50th percentile.  If you look only at the last two years, then Johnson is tied for 5th best amongst the closers (instead of 16th out of 32) and should be considered elite.

Last year, you may remember my concerns about Johnson and how it was sensible to deal him, letting O'Day and Hunter fight out the closer role.  That concern was based on him having a rather low strikeout rate and swinging strike rate.  They were rather historical exceptions for a pitcher with more than 35 saves.  This year, Johnson managed to move toward a near imposing strikeout rate just shy of 20% along with a sizable increase in swinging strike rate.  Perhaps a matter of coincidence, he also saw a near doubling of his home run rate.  The home run rate and a slight decrease in velocity are the only two red flag I see this year.  I'd say that we should expect him to continue to perform as an elite reliever with the potential for being an elite closer (something he has shown at times).

That said, 8 MM or more is a lot of payroll to take up with someone who throws about 60 innings a year and who could be replaced with only a minor impact on games (perhaps an additional blown save or two at most).  Of course, these questions will be addressed by Stuart Wallace later this month in his piece on righties in the bullpen as well as by me again in my final conclusion piece for making the 2014 Orioles into a world champion.

4 comments:

Matt P said...

Do you know whether there's a strong relationship between save conversion rates and ERA/FIP/xFIP?

A lot of set-up men and closers are still under team control or are starting arbitration. Is it fair to compare salaries? Kimbrel is making 500k this year but would probably be making 15m if he was eligible for free agency.



Joe Reisel said...

Would the conclusion be different if the obvious left-handed one-out guys like Randy Choate were not included?

Jon Shepherd said...

Matt - For ERA...r^2 is 0.21, so there is some. That said there could be other issues in play like half of this time period being in an offensive era. I do not immediately have numbers for FIP and xFIP.

If you want to put that together, you could guest a post (with perhaps some editing before going live). My assumption is though that you would have a very close relationship between those values and success rates simply because pitching well will likely overwhelm any other factor that may or may not be in play (e.g. the supposed closer mentality).

Joe - If guys like Choate were not included then I would guess that the averages for relievers in general would shift downward and make Johnson look better in general. However, with his place in the group not being affected much when you put him in the sub-demo of long standing closers, I doubt it would affect things significantly.

Matt P said...

I guess it's easy to tell I've written maybe five documents over five pages over the past ten years?

That sounds familar actually. Maybe in a few weeks I can take a look.