More on Jim Johnson and Elite Closers Status

A couple months back, I wrote about making an argument that Jim Johnson is effectively an elite closer.  This gave rise to a few thoughts from Matt Perez.

-------

Thoughts on Jim Johnson and Elite Closers
by Matt Perez

A couple months ago, Jon wrote a post discussing whether JJ Johnson is an elite closer by looking at JJs conversion rate of save and hold opportunities. While blown holds and saves do get a lot of notice, I was wondering whether they were related to run prevention. Do pitchers with a significantly lower ERA have a higher conversion rate than those pitchers with a significantly higher ERA? 

In order to test this, I looked at individual seasons for all relievers with both a qualifying amount of innings and also ten or more saves plus holds as well as all relievers with eighty or more saves plus holds total from 2003-2013. For each of these relievers, I determined their Save plus Hold conversion rate (in order to save time, I’ll use the abbreviation Sv_Hld_Pct for the rest of this post) as well as their ERA and FIP . Then I determined their correlation coefficient . 

A correlation coefficient is a statistic that determines how strongly specific metrics resemble each other.  In this case, the metrics are ERA and Sv_Hld_Pct. The possible values range from -1 to 1. The closer the value is to -1 or to 1, the higher the likelihood that the two metrics resemble each other. The definitions for when a correlation is strong, weak or non-existent depends on what one is studying and can be subjective rather than objective. However,  a general rule of thumb is that an r with an absolute value below .4 or above -.4 is weak, between .4-.7 or -.4 to -.7 is moderate and higher than .7 or lower than -.7 is strong. 

Listed below is a table with the results.

Individual Seasons

Metric                    r

With ERA:            - .3657

With FIP               -.2715

Career Numbers

Metric                  r

With ERA            -.5038

With FIP             -.4431

These results indicate that there is only a weak correlation between Sv_Hld_Pct and ERA/FIP for individual seasons and a moderate correlation between Sv_Hld_Pct and ERA/FIP for a full career. It would seem that it wouldn’t make sense to judge a closer based on his Sv_Hld_Pct.

It is possible to split up pitchers into two categories; pitchers that have more saves than holds and pitchers with more holds than saves. Determining the correlation between ERA/FIP and the above categories save or conversion rate will determine whether one category has a significant correlation. Listed below is a table with the results.

Individual Season

Metric                  r for More Saves       r for More Holds

With ERA            -.6059                           -.2644                           

With FIP             -.4798                           -.1780

Career Numbers

Metric                r for More Saves     r for More Holds

With ERA          - .6837                        -.3945

With FIP            -.6306                         -.3229

There are drastically different results for these two categories. ERA and FIP have a moderate to high correlation with save conversion percentage but only a low to moderate correlation with hold conversion percentage. 

There are a few reasons why this could be the case. Closers are typically brought into the game at the start of the ninth. In contrast, a setup man is more likely to be called into a game in the middle of an inning. Suppose a reliever comes into the game in the middle of an inning when there is an offensive player already on base. If that reliever allows that runner to score then he wouldn’t be charged the run but he may blow the lead and thus get a blown save. This is because runs are charged to the pitcher that allows an opponent to reach base and not to the pitcher that allows an opponent to score.

However blown saves are charged to the pitcher that allows a runner to score but not to reach base. If a reliever allows many inherited runners to score then he could in theory have a low Sv_Hld_Rate as well as a low ERA.

However, I think the main reason is how closers and setup men are used. The average closer gets either a save, a hold or a blown save 57% of the time. The average setup man gets a save, hold or a blown save 33% of the time. It would make sense that the average closers’ Sv_Hld_Rate would be more likely to resemble his ERA then it would for the average setup man because the average closer simply comes into the game a lot more frequently in one of those situations. 

In order to properly judge Jim Johnson’s save conversion rate we need to compare him to other closers. I looked at all qualified relievers from 2003-2013 who had at least two complete seasons where they had ten saves and had at least three times as many saves than holds.  Any pitcher that fulfilled these conditions was at minimum the closer of a team for most of the year.  There are a total of 57 closers who fulfilled these conditions. Then, I compared how these relievers did in their first year to how they did in their second year. 

It turns out that the median reliever who fulfilled all of the above conditions converted 89.1% of their save attempts in their first year but only 86.3% in their second year. Likewise, ERA goes from 2.54 in their first year to 3.19 in their second year while FIP goes from 3.01 to 3.45. If we look at relievers who have a 90% conversion rate or higher in their first year, we see a drop in their conversion rate from 91.3% in their first year to 88.8% in their second year.  Their ERA drops from 2.50 in their first year to 3.22 in their second year and their FIP drops from 3.08 in their first year to 3.51 in their second year. These differences are significantly high and suggest we could have expected Jim Johnson to be significantly worse in his second season than in his first season. 

Now that we know this, suppose we look at all qualified relievers from 2003-2013 who had all the same qualifications as above except had three complete seasons instead of two. There are a total of 30 closers who have fulfilled these conditions.

It turns out that the median closer who closes for three seasons has a conversion rate of 90%, an ERA of 2.45 and an FIP of 2.79 in his first year, a conversion rate of 89.4%, an ERA of 2.69 and an FIP of 2.78 in his second year and a conversion rate of 88.1%, an ERA of 2.77 and an FIP of 3.23 in his third year. While these differences aren’t significant, they do indicate that the average closer continues to degrade in performance the longer he closes.

The problem is that it doesn’t matter whether a closer was elite his first year. The question is whether he can repeat his performance in future years. Unfortunately, this indicates that even closers like JJ who were successful in their first year become increasingly likely to fail in future years. Just because a closer has one successful year doesn’t mean he’ll be successful in later years.

---