Relievers Getting Paid the Same and Resolving the Orioles Closer Postion

Hunter is Baltimore Backup Plan | Photo by Keith Allison

Yes, we have written a great deal about the Orioles' closer position over the past few months.  It has ranged from a lament about how much in terms of resources was devoted to Jim Johnson, to potential replacements, to the Grant Balfour quasi-fiasco, and, now, where to go now.  Before we move on, let me write one thing, I doubt the Balfour situation changed anyone's mind about the Orioles.  I think all it did was provide evidence for people to solidify their thoughts about the Orioles.  This, I believe, is primarily true because most people in baseball have negative thoughts about how the team has been, is being, and will be run.  With that in mind, the club can still compete for high profile players and retain high profile home grown talent.  They have done it and they will do it.  That said, not all teams and talent will be willing to listen too long given the history of the franchise.  However, I digress.  That is a topic to explore at another time.

In general, closers are overrated.  Over time, that view has ebbed and flowed.  As the save statistic was embraced and bullpen specialization flourished in the 1980s, we began seeing importance grow for pitchers who ended game and threw maybe 60-70 innings a year.  The high water mark took place in 1989 where for a few weeks the Royals' signing of Cy Young winning closer Mark Davis to a 4 year, 13 MM contract made him the highest paid player in baseball.  Since Davis, no closer has been the highest paid player in baseball.  A lesser water mark took place in early 2000 when Mariano Rivera set a record for a salary awarded through arbitration when he lost and received 7.25 MM.

Currently, there appears to be an ebb after Ruben Amaro Jr. awarded the last major free agent deal to a closer when he signed Jonathan Papelbon to a 4 year, 50 MM deal with a vesting option for another 13 MM.  This off season there has been a common refrain that the golden age of closers has ended as teams become more aware that the difference between an elite 60 innings pitched and a good 60 innings pitched might not matter all that much.  That, perhaps, not all relievers have the mentality to be closers, but many of them do.  The longest contract a closer has received has been 2 years (i.e., Joaquin Benoit, Edward Mujica, Joe Nathan), the great commitment was 20 MM (i.e., Joe Nathan), and the largest annual salary was 10 MM (i.e., Joe Nathan, Brian Wilson).  Those numbers may not seem much different than Papelbon's.  This seems especially true when you consider the injury histories, age, and performance of the pitchers mentioned.  In other words, this idea that the golden age is done might be more sound than fury.

Maybe the tide is turning, but I don't see the proof of it yet.  Here is a quick table showing top relief pitcher signings over the past several years and contract salary adjusted for this offseason (assuming 20% increase in costs this offseason):

		MM	Adj MM
2014	Joe Nathan	10	10
	Brian Wilson
2013	Rafael Soriano	14	16.8
2012	Jonathan Papelbon	12.5	16.6
2011	Rafael Soriano	12	16
2010	Jose Valverde	7	10.5
2009	Francisco Rodriguez	12.3	16.5
2008	Mariano Rivera	16	21.3
2007	Danys Baez	6.3	9.1

By my eye, it appears that this may simply be a down year and teams are valuing elite closers as guys worth about 2-3 WAR.  Perhaps the only real exception would be Rivera who had two things going for him: (1) Hall of Fame closer ability and (2) deep Yankee pockets.  Nathan and Wilson (throw in Jim Johnson if you wish) are simply closers with some questions attached to them in their placement as a first division closer.  Nathan and Wilson have injury concerns while Johnson closes in an incredibly non-traditional way.

Now, with that slight aside over with, whom should the Orioles target?  Below is a quick rundown of five candidates who would be on my board (in order of my preference):

Jesse Crain
In July, the White Sox placed Jesse Crain on the disabled list for a shoulder strain/tendonitis.  The Rays traded for him while he was on the disabled list and never pitched for them even though he was activated at the end of September.  The Rays lost LHP Sean Bierman and SS Ben Kline for their overtaking of Crain's rehab.  His time missed for a shoulder problem is worse than Balfour's long junked shoulder.  However, Crain will carry a lower price tag because of it and arguably has been the better pitcher during this time.  He is a fly ball pitcher, like Balfour, who dominates right handed batters and makes lefties look slightly below average.  It certainly is a split that can work as a closer.  He can throw in the mid 90s with a plus breaking ball.  He would be a target as a low base with incentives type of deal.  Crain should also be cheaper as he has never experienced closing opportunities as a MLB pitcher.

Grant Balfour
Unless you have been out in the remote tundra, you probably know who Balfour is, the troubles the Orioles have had with him, and how the closer market moved on while he danced with the Orioles.  Why do I prefer him?  I imagine his arm is no worse than it was the past several years (which means that his arm has been on the brink of falling apart for a long while as it is with many older pitchers with injury history)/  I have him second because he has been a solid fly ball pitcher.  He is certianly fine as a closer.  With many things being equal and if Crain looks like he cannot pitch, I'd try to make things right with the Orioles and mend this relationship.  Not signing him likely won't affect things much, but it would be good to show some good faith that can be leaned on with other players in the future.

Suk-Min Yoon
Yoon is a free agent after a rather successful Korean Baseball Organization career.  As a starter, he profiled as a low 90s pitcher with a good changeup and an average breaking ball.  It is a profile that looks good as a backend arm in a rotation, which is supposedly what Yoon is looking for.  Part of the reason he has yet to sign is that the starting pitcher market has been frozen as teams have been waiting to see what happens with Masahiro Tanaka.  With his posting ending in late January, free agent pitchers probably will need to simply move forward.  The other issue with Yoon is that performance in KBO is often held as suspect and he has also experienced shoulder trouble.  From my perspective, I think a multiyear deal with an initial handling of closer duties would put Yoon in a position for success.  He might see an uptick in his velocity as well, which will make his offering more playable in MLB.  Worse comes to worst, he can be a middle reliever and spot starter.  I'd offer something around 3 years and 10 MM and see what comes of it. I don't think he is the third best closer available, but he should be able to admirably serve in this role while freeing up the team to spend elsewhere.

Fernando Rodney
What irks people most about Rodney is his rendition of Katniss from the Hunger Games after every successful save.  Aesthetically, it can be pretty annoying.  However, most fans like it when their team wins games and Rodney does a great job putting teams in the position of winning games.  He is one of the hardest throwers in baseball and misses a lot of bats.  You can rather easily argue that he is the best closer on the market.  With the Rays, he found performance levels similar to those he experienced early in his career and he is the only elite talent out there without the specter of arm troubles.  He is looking for Joe Nathan money and commitment.  Teams have no agreed so far with that assessment.

Miguel Gonzalez
MiGo is the second player on the list whose health is not questioned and is the only player on this list already owned by the Orioles.  So, why MiGo?  He simply is not that good of a starter and the reason for that is also the reason why I think he would do fine as a reliever.  Last year, the first time he runs through a lineup, he holds them to a .607 OPS.  Second time through, it rises to .671.  Third time, OPS of .903.  That deterioration is not something that works for well for a starter.  He tends to blow up in the 5th or 6th innings.  His ideal position would likely be as a long reliever, but he could work as a closer capable of going longer than three outs.  He also performs rather well against batters on both sides of the plate.  That is what the difference is between him and another starter who deteriorates rapidly after the first time through the order: Wei-Yin Chen.  Chen is rather susceptible to right handed batters.  Of course, a problem with this approach is that it will require the acquisition of a starting pitcher, which is more expensive than signing a closer.

Why not Darren O'Day and Tommy Hunter?
I hesitate making O'Day the closer.  I think he is the best reliever the team has and his services are needed to get the team out of sticky situations in the 7th and 8th innings in order for the eventual closer to be handed a clean slate in the 9th.  The worship of the save stat often prevents a team from employing their best reliever in some of the more important situations.  That explanation likely will irritate most people, but we have to acknowledge how pitchers are actually used.

Hunter?  Hunter displayed a rather amazing split between lefties (.857 OPS) and righties (.344).  He simply would not be best utilized as a closer.  Having trouble with lefties will not be exploited as well as Chen would be with righties simply because there are more righties than lefties in baseball as well as benches rarely having any decent options to pinch hit.  As an arm in the pen, Hunter can be deployed in right handed hitting heavy innings to make short work of the side.  This makes him a solid option as the third righthander in the pen, but not much more than that or he will be exposed.  This does not mean he would be a horrible closer.  He would be acceptable in that role.  The issue is that I think his value would likely decrease in that role.

More on Jim Johnson and Elite Closers Status

A couple months back, I wrote about making an argument that Jim Johnson is effectively an elite closer.  This gave rise to a few thoughts from Matt Perez.

-------

Thoughts on Jim Johnson and Elite Closers
by Matt Perez

A couple months ago, Jon wrote a post discussing whether JJ Johnson is an elite closer by looking at JJs conversion rate of save and hold opportunities. While blown holds and saves do get a lot of notice, I was wondering whether they were related to run prevention. Do pitchers with a significantly lower ERA have a higher conversion rate than those pitchers with a significantly higher ERA? 

In order to test this, I looked at individual seasons for all relievers with both a qualifying amount of innings and also ten or more saves plus holds as well as all relievers with eighty or more saves plus holds total from 2003-2013. For each of these relievers, I determined their Save plus Hold conversion rate (in order to save time, I’ll use the abbreviation Sv_Hld_Pct for the rest of this post) as well as their ERA and FIP . Then I determined their correlation coefficient . 

A correlation coefficient is a statistic that determines how strongly specific metrics resemble each other.  In this case, the metrics are ERA and Sv_Hld_Pct. The possible values range from -1 to 1. The closer the value is to -1 or to 1, the higher the likelihood that the two metrics resemble each other. The definitions for when a correlation is strong, weak or non-existent depends on what one is studying and can be subjective rather than objective. However,  a general rule of thumb is that an r with an absolute value below .4 or above -.4 is weak, between .4-.7 or -.4 to -.7 is moderate and higher than .7 or lower than -.7 is strong. 

Listed below is a table with the results.

Individual Seasons

Metric                    r

With ERA:            - .3657

With FIP               -.2715

Career Numbers

Metric                  r

With ERA            -.5038

With FIP             -.4431

These results indicate that there is only a weak correlation between Sv_Hld_Pct and ERA/FIP for individual seasons and a moderate correlation between Sv_Hld_Pct and ERA/FIP for a full career. It would seem that it wouldn’t make sense to judge a closer based on his Sv_Hld_Pct.

It is possible to split up pitchers into two categories; pitchers that have more saves than holds and pitchers with more holds than saves. Determining the correlation between ERA/FIP and the above categories save or conversion rate will determine whether one category has a significant correlation. Listed below is a table with the results.

Individual Season

Metric                  r for More Saves       r for More Holds

With ERA            -.6059                           -.2644                           

With FIP             -.4798                           -.1780

Career Numbers

Metric                r for More Saves     r for More Holds

With ERA          - .6837                        -.3945

With FIP            -.6306                         -.3229

There are drastically different results for these two categories. ERA and FIP have a moderate to high correlation with save conversion percentage but only a low to moderate correlation with hold conversion percentage. 

There are a few reasons why this could be the case. Closers are typically brought into the game at the start of the ninth. In contrast, a setup man is more likely to be called into a game in the middle of an inning. Suppose a reliever comes into the game in the middle of an inning when there is an offensive player already on base. If that reliever allows that runner to score then he wouldn’t be charged the run but he may blow the lead and thus get a blown save. This is because runs are charged to the pitcher that allows an opponent to reach base and not to the pitcher that allows an opponent to score.

However blown saves are charged to the pitcher that allows a runner to score but not to reach base. If a reliever allows many inherited runners to score then he could in theory have a low Sv_Hld_Rate as well as a low ERA.

However, I think the main reason is how closers and setup men are used. The average closer gets either a save, a hold or a blown save 57% of the time. The average setup man gets a save, hold or a blown save 33% of the time. It would make sense that the average closers’ Sv_Hld_Rate would be more likely to resemble his ERA then it would for the average setup man because the average closer simply comes into the game a lot more frequently in one of those situations. 

In order to properly judge Jim Johnson’s save conversion rate we need to compare him to other closers. I looked at all qualified relievers from 2003-2013 who had at least two complete seasons where they had ten saves and had at least three times as many saves than holds.  Any pitcher that fulfilled these conditions was at minimum the closer of a team for most of the year.  There are a total of 57 closers who fulfilled these conditions. Then, I compared how these relievers did in their first year to how they did in their second year. 

It turns out that the median reliever who fulfilled all of the above conditions converted 89.1% of their save attempts in their first year but only 86.3% in their second year. Likewise, ERA goes from 2.54 in their first year to 3.19 in their second year while FIP goes from 3.01 to 3.45. If we look at relievers who have a 90% conversion rate or higher in their first year, we see a drop in their conversion rate from 91.3% in their first year to 88.8% in their second year.  Their ERA drops from 2.50 in their first year to 3.22 in their second year and their FIP drops from 3.08 in their first year to 3.51 in their second year. These differences are significantly high and suggest we could have expected Jim Johnson to be significantly worse in his second season than in his first season. 

Now that we know this, suppose we look at all qualified relievers from 2003-2013 who had all the same qualifications as above except had three complete seasons instead of two. There are a total of 30 closers who have fulfilled these conditions.

It turns out that the median closer who closes for three seasons has a conversion rate of 90%, an ERA of 2.45 and an FIP of 2.79 in his first year, a conversion rate of 89.4%, an ERA of 2.69 and an FIP of 2.78 in his second year and a conversion rate of 88.1%, an ERA of 2.77 and an FIP of 3.23 in his third year. While these differences aren’t significant, they do indicate that the average closer continues to degrade in performance the longer he closes.

The problem is that it doesn’t matter whether a closer was elite his first year. The question is whether he can repeat his performance in future years. Unfortunately, this indicates that even closers like JJ who were successful in their first year become increasingly likely to fail in future years. Just because a closer has one successful year doesn’t mean he’ll be successful in later years.

---

Closing in Camden: Who Gets the Ninth Inning in 2014?

As the proverbial paint continues to dry on the deal that sent AL saves leader Jim Johnson to the Oakland Athletics for second baseman Jemile Weeks, there were a number of knee jerk reactions to the deal, with most of them revolving how 'empty' the back-to-back 50+ save seasons Johnson had really were. True value of the save statistic notwithstanding, the swap was a curious one for Baltimore, with many questioning whether the return on Johnson was a bit lacking in talent and that Oakland General Manager Billy Beane perhaps got the best of his Baltimore counterpart. While Weeks has not panned out as many had projected when he was drafted in the first round of the 2008 MLB draft, a bigger, more pressing question looms for the Orioles.

Who closes?

The pitcher who comes in for the ninth inning in the post-Johnson era does have some sizable shoes to fill; yet, the need to keep the chain of 50 save seasons alive isn't necessarily the ultimate goal for the new closer. Consistent outings and an ability to get both lefties and righties out while keeping the ball in the park will be crucial mission requirements, with an upgrade from Johnson's paltry 7.17 K/9 rate in 2013 being desirable perk from the new closer.

As we have outlined previously here at Camden Depot, the Orioles bullpen heading into 2014 has some decent talent already in place from both the left and right side -- here's what the 2013 bullpen looked like, sorted by RE24 and a minimum of 10 innings pitched in relief:

Name	IP	K/9	BB/9	HR/9	BABIP	FIP	xFIP	WAR	RE24	SD	MD
Tommy Hunter	86.1	7.09	1.46	1.15	0.249	3.68	3.63	0.7	17.4	22	10
Darren O'Day	62	8.56	2.18	1.02	0.248	3.58	3.59	0.7	9.67	26	10
Brian Matusz	51	8.82	2.82	0.53	0.292	2.91	3.59	1	7.93	15	8
Jim Johnson	70.1	7.17	2.30	0.64	0.327	3.45	3.38	0.9	7.20	40	12
Josh Stinson	11.1	7.15	1.59	0.00	0.179	2.25	3.22	0.2	5.06	3	1
Kevin Gausman	23	11.35	2.74	0.39	0.296	2.00	2.33	0.6	4.99	5	4
Troy Patton	56	6.75	2.57	1.29	0.295	4.42	4.10	0	1.69	8	9
Steve Johnson	11.2	13.11	6.94	1.54	0.227	4.68	3.97	0	1.37	1	0
Francisco Rodriguez	22	11.45	2.05	2.05	0.351	4.28	2.31	0	-0.72	4	1
T.J. McFarland	72	6.75	3.38	0.88	0.304	3.94	3.75	0.3	-0.96	8	8

Already we can see where some of the nay-saying regarding Johnson's place in the pecking order of top flight closers comes from; three pitchers in the O's bullpen had a higher RE24, with each of those three notching fewer meltdown outings (MD) than Johnson's 12 last year. With it, two of the three had better strikeout rates than Johnson, a stat that many feel prevented him from being a truly elite closer.

So what of these three current Orioles? Does one or a combination of Tommy Hunter, Darren O'Day, or Brian Matusz have the stuff to shutdown the ninth?

In a word, no, at least not in a long term role; the next table explains why:

Name	OPS, LHB	OPS, RHB	L-R OPS Diff
Matusz	0.502	0.747	-0.245
Hunter	0.857	0.344	0.513
O'Day	0.922	0.443	0.479
Johnson	0.740	0.653	0.087

As we can see each of the three has fairly significant lefty-righty splits -- here we use OPS to display the splits -- which is something that your prototypical closer will not express in such a dramatic fashion. Here, I have included Johnson's splits as a reference; as you can see while Johnson's strikeout rates were less than stellar for a closer, his ability to adeptly mix his sinker/curve combo along with the occasional change up against both sides of the plate did much to make up for the lack of swings and misses and kept him on the mound regardless of hitter handedness. The aforementioned trio of Hunter, Matusz, and O'Day all are above average against same handed hitters, but struggle to get opposite handed hitters out consistently. In the case of O'Day, this situation arises from his submarine pitching style, while for Matusz and Hunter, the lack of a consistent third pitch that moves away from the opposite handed hitter creates much of the disparity in their splits. As such, each would be best fit in setup roles or carefully selected closing roles where the opposing lineup does not have the ability to pinch hit for opposite handed batters or lacks switch hitters; this environment is also ripe for a closer by committee set up for manager Buck Showalter in 2014. While he does have a history of shutting down the ninth, Francisco Rodriguez is firmly planted in the twilight of his career, with his repertoire best suited for middle inning relief roles.

Looking beyond the current roster, the ability to pick up a proven closer on the free agent market has diminished drastically in the last few days, with the likes of Joe Nathan, LaTroy Hawkins, Brian Wilson being signed; Grant Balfour and Fernando Rodney appear to be the only legitimate, somewhat reliable options remaining on the free agent market that doesn't carry with them questions regarding their ability to fill the role or their medical history. As such, the Orioles could be bigger players on the trade market for the remainder of the offseason, given the slim pickings of the free agent market.

For now, the role of closer appears to be of the committee persuasion, with a trio of capable arms able to to handle shared duties, but hampered by the reality of their splits keeping them from consistent outings in the ninth inning. That being said, a plethora of talented arms who have adeptly filled the closer role but have found themselves on the outside looking in -- or at least pitching the eighth -- on their current squads, but could be attractive trade targets for the O's to keep an eye on. Pitchers such as Drew Storen, Mark Melancon, and Ryan Cook qualify as pitchers with closer pasts or potential that could be had for the right price or prospect. Whatever transpires for the Orioles as they prepare for 2014, the role of the closer will remain one watermarked by its fluidity and volatility in spite of it being a high profile position.

Jim Johnson: Being Sent to the Ravine?

Jim Johnson - aka "The Janitor"

Over the past decade and a half there has been a growing discord in the value of a closer.  First this was seen in the fringe statistical element, then flashing in the majors (most memorably with the Red Sox failed closer by committee attempt), and now reaching a slow boil among the general mainstream fan base.  The attack on closers comes in three flavors:

1. Playing Time Argument: How can a player who at most spend 70 innings in the game be worth 10 MM a year?
2. Sabermetrics Approach: Mariano Rivera, the best closer ever, was worth about 2 rWAR a year without much variation, so how can the quasi-elite arms be worth just as much?
3. Practical Concept: The difference between the best and worst closers is maybe blowing 3 more games or so, which can be mitigated by putting in the hot hand.

In an abstract sense, I agree with all of those sentiments.  I also think that for a team like the Orioles, who are probably saddled with a payroll limit of maybe 100 MM or so, it makes little sense to devout 10% of your resources to a player who plays so little and whose impact is not so great considering the alternatives.  With that in mind however, for the right team, it makes complete sense to pay a seemingly exorbitant amount of cash for a relief pitcher.

Sabermetrically-inclined teams spend about 5% of their budgets on closers whether it is Kyle Farnsworth or Fernando Rodney for the Rays or Grant Balfour for the Athletics or Joel Hanrahan for the Red Sox.  Wherever those teams are located in the scheme of things with payroll, they tend to wind up around 5% of their contract money on that position.  That is part of what lead to my view that signing Joe Smith as a potential replacement for Jim Johnson was a good idea (Smith wound up signing with the Angels for 3/15.75MM).  Teams with less statistical care or perhaps more money choose to be more free with the money they alot their closer (e.g., Phillies).  Still, even the rich teams tend to pay their closer about 5%.  The Yankees did that with Rivera.

With that in mind, if Johnson is going to see about 10 MM, then the team will be looking to clubs with payrolls over 150 MM (assuming the Orioles are willing to kick in a million or two).  This leaves us with Boston, Detroit, LAAA, Philadelphia, LAD, and San Francisco.  What we need to look for are teams in need of closers and two fringe B level prospects to grab.

Boston
The Red Sox are unlikely feeling much need for a closer.  Koji Uehara filled the position in nicely and the team has several interesting arms to back fill positions.

Detroit
The Tigers certainly have in-house options, but they have never shied away from securing the noble proven veterans.  They could potentially be an option.  I would come in hard asking for SS Eugenio Suarez and RP Bruce Rondon and hope either for that or downgrade Rondon to RP Drew VerHagen.  The Tigers may also wish to cash their chips on Rick Porcello, but I think he probably is more highly valued than Johnson.  I doubt the Orioles would package Johnson with a prospect to acquire one year of Porcello.

Angels
Ernesto Frieri is their closer and I see no reason why they would think otherwise.  Money has also been a bit tight for them with Josh Hamilton and Albert Pujols, so a 10 MM closer might be a bit frivolous.  Additionally, their minors are pretty rough and they may be hesitant to deplete it further.

Philadelphia
Jonathan Papelbon ends the conversation.

Dodgers
They have a perceived need at the closer slot and the money to resolve that issue.  There are two ways the Orioles could go about things here.  One, there is word that the Dodgers are seeking to be involved in the bidding for Tanaka or another free agent pitcher.  That might mean that they are looking to move Josh Beckett or Chad Billingsley.  Both pitchers come with injury issues, but the Dodgers may be willing to stomach Johnson and then throw in some money to reduce payroll enough to seek a more desirable player.  Alternatively, the Orioles could look to build up their prospect base more by asking for LHP Chris Reed and LHP Onelki Garcia or perhaps settle for depth in C Tim Federowicz.

San Francisco
Sergio Romo performed well enough last year, so I doubt this is a viable destination.

Conclusion
I think in all practicality, the two teams likely to seek Johnson without the Orioles throwing too much money into the deal would be the Detroit Tigers or the Los Angeles Dodgers.  Both teams have decent fits for the Orioles to find fringe meaningful prospects as well as opportunities to fill in the veteran hole left by Jason Hammel.  Johnson certainly is a very solid closer and a pitcher any team would like to have on the roster.  Most teams would love him as a closer.  However, nearly all teams would shy away from that price.

This all said, there could always be a fringe team that would be interested in only investing one year at 10 MM instead of paying someone like Joe Nathan 10 MM a year on a multi-year commitment.  A team like the Reds could see value in what Johnson has to offer if they really think Aroldis Chapman is a starter.  The same, perhaps, could be said of the Rangers as well.

Is Jim Johnson an Elite Closer?

Jim Johnson Saves | photo: Keith Allison

On a later date, Stuart Wallace will address the bullpen and that means his thoughts on what exactly to do with Jim Johnson.  As you may know, Johnson cost the team 6.5 MM last year as both he and the team avoided an arbitration hearing.  Johnson has one more year of free agency left and, based on his final 2013 numbers (2.94 ERA, 50 saves), he stands to see another increase in money.  Some might suggest non-tendering him or believe that the Orioles could take him to arbitration and win the greatest amount they could reduce his salary to (4.55 MM), but I think it won't happen that way.  Instead, I think it is likely that we see him earn about 33% more than he did last year.  Typically, your second to last arbitration date results in a salary worth about 60% of the market rate and the last one being 80%.  With that in mind, we should probably expect a 2014 salary of 8.67 MM.  For comparison, Mariano Rivera took a 5 MM cut in 2013 and played for 10 MM while Joe Nathan is on tap for 9 MM next year.  An 8.67 MM salary assumes Johnson is an elite closer and pays him just a little south of that mark.

Is he worth it?

To answer that question, it probably helps to figure out to what degree relievers in general perform.  We can do this by looking at some measure of WAR or FIP or ERA or whatever.  I decided to go a little more big picture here and look at successful conversion of save and hold opportunities.  So much ink, talk radio, and internet space had been devoted this past season to Johnson blowing saves that it seems somewhat proper to look at a collection of these binary events.  In other words, perhaps there is something special about game situations where a save is on the line to the extent that maybe it is not captured with other measures.  Additionally, perhaps with a large enough sample size we are able to see through the noise and measure something that is related to actual performance ability.

Below you will find a table of relievers from 2004 to 2013.  Included are players who managed to have the most save chances as defined by having the ability to blow a save.  This definition of course will include tight games in late innings as well.  Also included is a ranking of the 32 pitchers over the past ten years who have accumulated 100 or more saves.  The list includes 124 players and a good bit of analysis follows the table, so keep on reading.

Player	Saves	Holds	Blown	S+H	Chances	%	RNK
Randy Choate	6	94	5	100	105	95.2
Jason Grilli	38	74	8	112	120	93.3
Sergio Romo	55	82	11	137	148	92.6
David Robertson	8	116	10	124	134	92.5
Javier Lopez	12	111	10	123	133	92.5
Pedro Feliciano	4	103	9	107	116	92.2
Greg Holland	67	28	9	95	104	91.3
Mariano Rivera	369	0	36	369	405	91.1	1
Joe Nathan	340	8	34	348	382	91.1	2
Sean Marshall	16	95	11	111	122	91.0
Craig Kimbrel	139	2	15	141	156	90.4	3
Grant Balfour	72	94	18	166	184	90.2
C.J. Wilson	52	46	11	98	109	89.9
Santiago Casilla	39	75	13	114	127	89.8
Joakim Soria	160	15	20	175	195	89.7	4
Eric O'Flaherty	0	103	12	103	115	89.6
Casey Janssen	65	37	12	102	114	89.5
Rafael Soriano	173	71	29	244	273	89.4	5
Sean Burnett	10	90	12	100	112	89.3
Kenley Jansen	62	37	12	99	111	89.2
Joel Peralta	11	137	18	148	166	89.2
Mike Adams	4	144	18	148	166	89.2
Arthur Rhodes	16	128	18	144	162	88.9
Brad Ziegler	32	79	14	111	125	88.8
Drew Storen	55	47	13	102	115	88.7
Trevor Hoffman	249	2	32	251	283	88.7	6
Jose Valverde	276	14	37	290	327	88.7	7
Takashi Saito	84	40	16	124	140	88.6
Aroldis Chapman	77	23	13	100	113	88.5
Brian Wilson	171	17	25	188	213	88.3	8
Joe Smith	3	117	16	120	136	88.2
Scott Downs	26	167	26	193	219	88.1
Brian Shouse	5	84	12	89	101	88.1
Jonathan Papelbon	286	7	40	293	333	88.0	9
Michael Gonzalez	56	89	20	145	165	87.9
Bob Wickman	111	5	16	116	132	87.9	10
Carlos Marmol	117	83	28	200	228	87.7	11
F. Rodriguez	302	81	54	383	437	87.6	12
Eric Gagne	80	11	13	91	104	87.5
Derrick Turnbow	65	39	15	104	119	87.4
Alan Embree	18	86	15	104	119	87.4
Mike MacDougal	44	46	13	90	103	87.4
Jason Motte	54	49	15	103	118	87.3
Matt Lindstrom	45	85	19	130	149	87.2
Jose Veras	26	83	16	109	125	87.2
Darren Oliver	5	97	15	102	117	87.2
Juan Rincon	3	85	13	88	101	87.1
Dan Wheeler	41	120	24	161	185	87.0
Scott Eyre	1	99	15	100	115	87.0
Joaquin Benoit	36	137	26	173	199	86.9
B.J. Ryan	114	25	21	139	160	86.9	13
George Sherrill	56	76	20	132	152	86.8
Billy Wagner	197	7	31	204	235	86.8	14
Damaso Marte	15	110	19	125	144	86.8
Brad Lidge	224	39	40	263	303	86.8	15
Jim Johnson	122	62	28	184	212	86.8	16
Ryan Dempster	87	5	14	92	106	86.8
Brian Fuentes	200	48	38	248	286	86.7	17
Chris Perez	132	23	24	155	179	86.6	18
Matt Thornton	23	170	30	193	223	86.5
Edward Mujica	41	68	17	109	126	86.5
Bobby Jenks	173	5	28	178	206	86.4	19
Ryan Madson	52	113	26	165	191	86.4
Akinori Otsuka	39	74	18	113	131	86.3
Todd Jones	135	28	26	163	189	86.2	20
Bob Howry	17	108	20	125	145	86.2
John Axford	106	25	21	131	152	86.2	21
Brandon Lyon	70	115	30	185	215	86.0
Luke Gregerson	16	132	24	148	172	86.0
Salomon Torres	55	68	20	123	143	86.0
Huston Street	234	18	41	252	293	86.0	22
Jeremy Affeldt	24	111	22	135	157	86.0
Justin Speier	7	85	15	92	107	86.0
Heath Bell	168	83	41	251	292	86.0	23
Jason Frasor	36	97	22	133	155	85.8
J.C. Romero	6	102	18	108	126	85.7
Trever Miller	6	84	15	90	105	85.7
Fernando Rodney	169	69	40	238	278	85.6	24
Andrew Bailey	89	12	17	101	118	85.6
John Grabow	6	95	17	101	118	85.6
Frank Francisco	73	67	24	140	164	85.4
Tom Gordon	48	97	25	145	170	85.3
David Aardsma	69	23	16	92	108	85.2
Guillermo Mota	9	94	18	103	121	85.1
Chad Cordero	127	10	24	137	161	85.1	25
Scot Shields	20	150	30	170	200	85.0
Joel Hanrahan	100	30	23	130	153	85.0	26
Kyle Farnsworth	50	108	28	158	186	84.9
J.J. Putz	189	59	44	248	292	84.9	27
Hideki Okajima	6	84	16	90	106	84.9
Todd Coffey	11	78	16	89	105	84.8
Tyler Clippard	33	110	26	143	169	84.6
Mike Timlin	25	73	18	98	116	84.5
Francisco Cordero	304	23	61	327	388	84.3	28
Matt Guerrier	6	117	23	123	146	84.2
Jesse Crain	4	124	24	128	152	84.2
Jose Mesa	72	27	19	99	118	83.9
Rafael Betancourt	73	138	41	211	252	83.7
Jon Rauch	62	97	31	159	190	83.7
Danys Baez	83	48	26	131	157	83.4
Jason Isringhausen	170	26	40	196	236	83.1	29
Shawn Camp	12	81	19	93	112	83.0
Kevin Gregg	177	18	40	195	235	83.0	30
Ryan Franklin	84	47	27	131	158	82.9
Juan Oviedo	92	29	25	121	146	82.9
Chad Qualls	51	159	44	210	254	82.7
Scott Linebrink	8	148	33	156	189	82.5
Eddie Guardado	71	33	22	104	126	82.5
LaTroy Hawkins	57	129	40	186	226	82.3
Jonathan Broxton	111	84	42	195	237	82.3	31
Joe Borowski	96	20	25	116	141	82.3
Luis Ayala	14	96	24	110	134	82.1
Matt Capps	138	36	38	174	212	82.1	32
David Weathers	61	69	29	130	159	81.8
David Hernandez	19	65	19	84	103	81.6
Armando Benitez	92	14	24	106	130	81.5
Jamie Walker	8	75	19	83	102	81.4
Brandon League	74	51	29	125	154	81.2
Bobby Parnell	36	54	21	90	111	81.1
Kerry Wood	63	39	24	102	126	81.0
Octavio Dotel	81	63	34	144	178	80.9
Matt Belisle	5	94	24	99	123	80.5
Manuel Corpas	34	60	23	94	117	80.3
Aaron Heilman	16	96	31	112	143	78.3

The list above is a familiar list.  All of them, at one time or another, were consider among the elite relievers.  They were players teams wanted at the end of games.  For those reasons, it is important to recognize that the above population is a subset of relievers.  Jim Johnson's 86.8% save rate is in the middle, but it is in the middle of all of these top flight relievers.  In other words, an average value here does not necessarily mean that a pitcher is average.  He is average amongst very good relievers, which is a valuable thing.

Below is a table showing the general percentiles and average for the entire 124 players above.

All Relievers
Percentile	%
50th	86.4
67th	87.4
Average	86.3
StDev	3.1

Johnson's 86.8% is a shade above average, but still well within the standard deviation for this population.  He does not appear to be exceptional so far based on these numbers.  You may contend that Johnson is a special case where he has risen to a new level of ability over the past two years.  In that case, you may prefer to look at his 89.4% success rate from 2012 to the present.  In that case, he is on the cusp of being an exceptional reliever and certainly is a first division closer (I consider anything above the 67th percentile to be a first division closer).

Looking more closely at the closer who accumulated at least 100 saves:

Closers	>100 Saves
Percentile	%
50th	86.8
67th	87.7
Average	86.7
StDev	2.3

For his career, Johnson is at the 50th percentile.  If you look only at the last two years, then Johnson is tied for 5th best amongst the closers (instead of 16th out of 32) and should be considered elite.

Last year, you may remember my concerns about Johnson and how it was sensible to deal him, letting O'Day and Hunter fight out the closer role.  That concern was based on him having a rather low strikeout rate and swinging strike rate.  They were rather historical exceptions for a pitcher with more than 35 saves.  This year, Johnson managed to move toward a near imposing strikeout rate just shy of 20% along with a sizable increase in swinging strike rate.  Perhaps a matter of coincidence, he also saw a near doubling of his home run rate.  The home run rate and a slight decrease in velocity are the only two red flag I see this year.  I'd say that we should expect him to continue to perform as an elite reliever with the potential for being an elite closer (something he has shown at times).

That said, 8 MM or more is a lot of payroll to take up with someone who throws about 60 innings a year and who could be replaced with only a minor impact on games (perhaps an additional blown save or two at most).  Of course, these questions will be addressed by Stuart Wallace later this month in his piece on righties in the bullpen as well as by me again in my final conclusion piece for making the 2014 Orioles into a world champion.