Does a low bullpen ERA result in more 1 run wins?

This past weekend Rick Maese wrote the following in the Baltimore Sun:

Generally, you notice the impact of a manager in the close games, when the outcome might hinge on a single decision. At this point a year ago, the Orioles were 6-15 in one-run games. They finished the year 13-31.

And this year? With last night's win, the Orioles are 15-9 in one-run games, tops in baseball.

Much of that credit goes to the bullpen and to the fact that one of the first things Trembley did after last season was add pitching coach Rick Kranitz to his staff.

I'm not really sure that this is true. Typically, you hear that winning percentage in one run games is the result of luck. I disagree with that, but only slightly. I imagine that one run record is largely luck, but that general team talent is also a contributor. For instance, I expected a team with a .600 winning percentage to do better than one with a .400 winning percentage, but that due to limited sample size you will often see a "noisy" relationship between total team winning percentage and 1 run winning percentage. Today, I am going to try to test this and figure out if a bullpen is largely responsible for a team's record in 1 run games.

Method
I took data from all AL teams from 2005 to 2007 (3 seasons - 42 data points). I calculated their bullpen ERA+. I used ERA+ instead of plain ERA in order to normalize year to year changes in run scoring. I also calculated their 1 run game winning percentage and overall winning percentage for each team-year. I then related 1 run winning percentage to bullpen ERA+ and overall winning percentage. Additionally, I compared bullpen ERA+ to the % difference between 1 run winning percentage and overall winning percentage.

Results
Bullpen ERA+
Maese's basic assumption (and it really isn't fair to attribute this assumption to Maese as you hear this rabbitted about nearly everywhere) is that a good bullpen ERA results in a good 1 win winning percentage record. What we see when we graph these is that there is an incredibly poor relationship between these two measurements (r2=0.06). This basically means that knowing a team's bullpen ERA essentially tells you nothing about what their winning percentage will be in 1 run games. Can you find the Indians in 2005? They are a solitary dot with an ERA+ of 155 and a win pct of .378. I would say that is an outlier, but the general pattern is also rather bunk. True, there does seem to be a pattern, but either there just isn't enough sample size to determine if the bullpen is the deciding factor or not.

Overall Winning Percentage
I assumed this would be far more relevant to the 1 run winning percentage than the bullpen ERA+ metric. This turned out to be the case and overall winning percentage seems to be much better than bullpen ERA+, but still not a great predictor (r2=0.3).






Non 1 Run Winning Percentage
It also bears season that overall winning percentage would obviously have greater significance for the sole reason that the 1 run games are included in that result. Perhaps a better way to determine the effect is to compare 1 run games to non 1 run games. What we see here is a result that is just as worthless as bullpen ERA+ (r2=0.05). We are basically back to where we started as in we have no idea to what we can attribute 1 run game success.




Can Bullpen ERA+ predict difference in Winning Percentage
Finally, I decided to compare bullpen ERA+ to the difference in 1 run winning percentage and >1 run winning percentage. The result is that we have another two statistics that cannot predict each other (r2=0.08).

Conclusion
Maybe small sample size is distorting what we can measure; but, based on these three AL seasons, bullpen ERA+ and non-1 run winning percentage are poor indicators of success in one run games. Overall team record is helpful, but that is most likely due to the inclusion of the 1 run data within that data set.