Buck Showalter's Adherence to Bullpen Roles has Shown Up in the Playoffs Before

Remember when the Orioles had to win a single game to continue in the 2016 playoffs and their manager chose to leave their best reliever - and arguably the best reliever in baseball - in the bullpen? That decision relied on flawed logic, that even if the Orioles got a lead with Zach Britton on the mound, they'd still need another reliever to close the game out and he might give it away. The Orioles eventually lost with Britton sitting alone in the dugout, mostly fresh (he warmed up a few times during the game), thereby creating the new saddest six-word story in the English language: "For sale: playoff jersey; never worn."

The image of Britton sitting on the bench stood in stark contrast to the way Terry Francona and Joe Maddon managed their relentless bullpens throughout the playoffs and into the World Series. The AL Manager of the Year and the NL runner-up each deployed members of their respective bullpens in emergency, high-leverage situations for extended outings regardless of inning. Francona especially garnered a great deal of attention for his unorthodox, win-at-all-costs bullpen management. Journalists and fans around the country began to wonder if this model of bullpen management - firefighting rather than game saving - was key to season-long success (don't be ridiculous; it's incredibly taxing over a postseason, much less 162 games). It was very reminiscent of the Royals in 2014, whose bullpen shut down the Orioles in the ALDS en route to a World Series victory.

Andrew Miller, left handed relief pitcher for the Indians, seemed to be the centerpiece around which Francona deployed his bullpen. Miller was also a member of the 2014 Orioles who were dispatched in 2014, and his short stint in Baltimore made a lot of fans think that the team had a winning model going forward: stock the pen with arms, send them to the mound whenever the team needed an out, and shut down scoring opportunities before they happened. Maybe Francona's plan wasn't novel after all... in fact, FanGraphs highlighted the Orioles, Buck Showalter, and Andrew Miller himself in 2014 as examples of forward-thinking bullpen management.

Looking back at the usage of Andrew Miller in the 2014 bullpen, we find that perhaps Buck Showalter wasn't so forward thinking after all. Like in 2016, Miller was not the team's closer, and he was "free" to appear in any inning in which he was needed. Here's how that usage shook out in each year:

Showalter brought Miller into the game exactly one time before the 7th inning, and never used him for more than 5 outs. He threw 32 pitches in his first outing against the Tigers and then proceeded to throw fewer than 25 in each game afterwards. Some of that is a credit to Miller being good enough to get five outs on 20 pitches; another piece is Showalter adhering to traditional bullpen roles. Miller was used in back-to-back games twice, and only once on back-to-back days.

At face value, this paints Showalter as a more conservative manager who adheres slightly more strictly to defined bullpen roles. Even though Miller wasn't the "closer" or "setup man," he was probably safe in assuming that he would enter the game in the 7th inning if he was available that day. Francona's usage was much more variable over the course of the postseason, with Miller entering the game in each mid and late inning with an even frequency.

To be clear, this is merely one aspect of bullpen management, and only one example of it. Without knowing any context for the leverage of the situation in which Miller entered, it's possible that he was used as a fireman as much in 2014 as he was in 2016.

But such adherence to predictable late-inning entrances in 2014 probably wasn't by chance, and should have tipped fans off (if they weren't already aware) of Showalter's unwillingness to truly break away from defined bullpen roles. At least in the postseason, where every out matters to the team's very near future, the Orioles' manager needs to do a better job of firing the big guns when he needs them and not saving them for later - something he seems to have done consistently.

Don't Trust a Reliever Farther Than You Can Throw Him

Relievers have been in high demand this offseason. Potential closer candidate Andrew Miller received a four-year deal for $36 million while David Robertson received four years and $46 million. Non-closers like Pat Neshek received two years and $12.5 million while Luke Gregerson got three years and $18.5 million. The value of a good bullpen was proven when the Royals rode their top guys to the World Series last season. But many believe that even the best relievers can be highly volatile and therefore teams should be skeptical of offering them large contracts. In order to see whether this is valid, I looked at all of the 301 relievers from 2010-2012 who threw at least 50 innings in relief during that stretch, and compared their performance for 2013 and 2014 to all 265 relievers who threw at least 40 innings in relief using ERA, WAR, and RA9_WAR.

The chart below shows how each of the top 50 relievers according to each of the statistics from 2010 to 2012 performed in 2013 and 2014. 

Only 31.5% of the top 20 relievers and 24.1% of relievers ranked 21-50 according to ERA in 2010-2012 were top 50 relievers in 2013 and 2014. The average top 50 reliever ended up being an asset to the bullpen but wasn’t the star reliever that the team signing him was hoping to receive. The average ERA of roughly 3.15 looks like it's acceptable. However, 100 out of the 265 relievers who threw at least 40 innings in relief from 2013 to 2014 had an ERA under 3.00. An ERA of 3.15 is slightly better than the median ERA of 3.35 for all qualified relievers.

45% of the top 20 relievers and 17.2% of relievers ranked 21-50 according to WAR in 2010-2012 were top 50 relievers in 2013 and 2014. The value of their contributions decreased dramatically from 2010-2012 to 2013 and 2014.

25% of the top 20 relievers and 36.7% of relievers ranked 21-50 according to RA9_WAR in 2010-2012 were top 50 relievers in 2013 and 2014. A number of top relievers according to RA9_WAR from 2010 to 2012 such as Sean Marshall, Jonny Venters, Jason Motte, Eric O’Flaherty, Jesse Crain, Rafael Betancourt, and Joel Hanrahan suffered injuries in 2013 or 2014 that significantly impacted their value. It is questionable whether the large amount of injuries for relievers ranked in the top 20 in this category is typical.

In addition, success rates weren’t impressive when using multiple measures to determine reliever quality. The five relievers ranked in the top 20 using each measure were Craig Kimbrel, Mariano Rivera, Sergio Romo, Mike Adams, and Darren Oliver. Kimbrel has remained excellent and is easily a top five reliever while Rivera retired after 2013, but Romo, Adams, and Oliver were all disappointments. Likewise, there were nine relievers ranked in the top 20 of two of the categories and in the top 50 in the other category. Four of those nine relievers ended up getting hurt and unsurprisingly only four of the relievers in that category were actually successful. Of the nine relievers who were ranked in the top 20 of two of the metrics without being ranked in the third, the only successful ones were Tyler Clippard and Joaquin Benoit.

The problem is that elite relievers have very little room for error. The top relievers had an ERA of about 2.00. If they throw 63 innings a season, then that means they can only allow 14 runs. If they end up allowing just eight more runs per season, then their ERA is closer to 3.14 and they are simply average. The difference between the best relievers and decent relievers is minuscule and can come down to a few good or bad breaks. At the same time those same extra seven runs are more important than the average run because elite relievers are usually used in the most crucial situations.

Some top relievers from 2010 to 2012 were still good in 2013 and 2014. But on the whole a top reliever from 2010-2012 was unlikely to be elite in 2013 and 2014. It doesn’t make sense to pay relievers for past performance because it isn’t likely that they will be able to repeat it. A team that has limited amounts of money should focus on either position players or starting pitching. Quality relief pitching is important, but there are so many variables involved that are outside the pitchers' control that even the best relievers can't consistently provide it.

Should the Orioles Gamble on Andrew Miller's Dominance?

At the trade deadline, the Orioles picked up Andrew Miller from the Red Sox in exchange for Eduardo Rodriguez to bolster the bullpen. At the time, Nate wrote that the O's could have used Rodriguez to acquire a more significant piece (and one that potentially didn't depart after 2014). Meanwhile, Matt Perez analyzed why the O's were willing to part with Rodriguez. Regardless, we all agreed that Miller made the O's at least slightly better and were excited to watch him pitch in Baltimore.

Miller had been a good but not great relief option for Boston the last few seasons, but in 2014 he was able to limit his walks while striking out even more batters. In 42.1 innings with Boston in 2014, Miller struck out 14.7 batters per nine innings while walking only 2.8 batters per nine. But with the Orioles, Miller was even better. In his 20 regular season innings after he was acquired, Miller posted a 15.3 K/9 and a BB/9 of only 1.8. His ERA was about a full run better, and his FIP and xFIP each dropped by about a half run each. Considering how low those numbers already were, that's fantastic. He was also excellent in his five postseason relief appearances. In 7.1 combined innings in the American League Division Series and the AL Championship Series, he allowed only three baserunners (1 hit, 1 walk, 1 HBP) and no runs.

The Orioles targeted Miller because he's a hard-throwing lefty who can pitch effectively against both right- and left-handed batters. He also finishes opposing hitters off with strikeouts (thanks to his phenomenal slider) --  a skill O's pitchers don't have in abundance. Miller was the team's best pitcher out of the bullpen in the regular season after the trade and also in the playoffs, and the Orioles got everything they could have wanted from him. But Miller is a free agent now, and he's positioned himself for quite a payday.

Tim Dierkes of MLB Trade Rumors is projecting that Miller will receive a four-year, $32 million contract. That's a significant chunk of money to pay a pitcher who will likely throw between 50 and 70 innings each season. The Royals' overpowering late-game relievers (along with Miller himself) have demonstrated how shutdown relievers can be even more valuable in the postseason, with the amount of time off between games allowing relievers to pitch a larger percentage of a team's innings than they normally would. But a team has to make the playoffs first for that to even matter, and the O's may be unwilling to shell out nearly twice the amount per season that their current highest paid reliever (Darren O'Day, $4.25 million club option) will make in 2015, especially with so many other moving parts this offseason (Nick Markakis, Nelson Cruz, plenty of arbitration increases, etc.).

Let's double back to Miller's fantastic 2014 for a moment. First, he was one of just 10 relievers with an fWAR over 2. And in the expansion era (1961-2014), only seven other pitchers had a season in which they threw at least 62 innings with a K/9 of at least 14.5 (two accomplished this feat twice).

Rk	Player	SO	IP	SO9 ▾	Year	Age	Tm	G	SV	BB	ERA	FIP	ERA+
1	Craig Kimbrel	116	62.2	16.66	2012	24	ATL	63	42	14	1.01	0.78	399
2	Carlos Marmol	138	77.2	15.99	2010	27	CHC	77	38	52	2.55	2.01	167
3	Aroldis Chapman	112	63.2	15.83	2013	25	CIN	68	38	29	2.54	2.47	149
4	Aroldis Chapman	122	71.2	15.32	2012	24	CIN	68	38	23	1.51	1.55	274
5	Eric Gagne	137	82.1	14.98	2003	27	LAD	77	55	20	1.20	0.86	337
6	Billy Wagner	124	74.2	14.95	1999	27	HOU	66	39	23	1.57	1.65	287
7	Brad Lidge	157	94.2	14.93	2004	27	HOU	80	29	30	1.90	1.97	228
8	Andrew Miller	103	62.1	14.87	2014	29	TOT	73	1	17	2.02	1.51	192
9	Craig Kimbrel	127	77.0	14.84	2011	23	ATL	79	46	32	2.10	1.52	182
10	Armando Benitez	128	78.0	14.77	1999	26	NYM	77	22	41	1.85	2.10	241

Provided by Baseball-Reference.com: View Play Index Tool Used
Generated 10/21/2014.

A few notes: Craig Kimbrel and Aroldis Chapman are awesome. Miller is the only non-closer on this list (he's the only pitcher with fewer than 22 saves). At 29, he's also the oldest.

Unlike many of the pitchers on the above list, Miller has never really dealt with an extreme workload. That can both be attributed to his general ineffectiveness as a starting pitcher and a couple of prior injuries. His 62.1 innings this season were the most he's thrown as a reliever, and the most since his 65 innings in 2011, when he made 12 starts. So it's not like he's been dealing with 70- or 80-inning seasons out of the bullpen.

Considering the price of a win (about $6 million), if Miller is anything close to a two-win pitcher the next couple seasons, a four-year, $32 million deal for him will be a bargain. However, Miller has never been quite this good. As noted, he's only worked exclusively as a reliever for a few seasons; in 2012 and 2013 he combined for an fWAR of 1.1. He also has a career 4.91 BB/9 -- 5.04 as a starter and 4.67 as a reliever. So even though he cut down on his walks in 2014, they're still at least a minor concern.

Miller is extremely talented, and any bullpen could use his unique skills. But relievers are strange. They can be awesome one year, and not quite so good the next. The O's also already have a pretty solid bullpen, and it may be wise to spend that $32 million or however much they would allocate for Miller in another area or two. Perhaps this new-and-improved Miller is the real deal and is here to stay, but that is probably a risk another team should take. It's a tough call to pay Miller like one of the best relievers in baseball when he's really only been in that class for one season.

It's possible to build an effective bullpen without handing out long-term contracts. In fact, Buck Showalter and Dan Duquette have already shown they can do just that. The O's built a strong bullpen with a collection of failed starters (Zach Britton, Brian Matusz, Tommy Hunter), unheralded signings (Darren O'Day, Ryan Webb), a trade (Brad Brach), and the rule 5 draft (T.J. McFarland). It's not easy to build an effective bullpen, but it is at least easier to find pitchers who can succeed in 50-inning relief roles instead of 140-inning starting roles. The O's also may have an extra starting pitcher to dangle in a trade, and there's also no guarantee that Matusz and Hunter aren't traded or non-tendered.

If the Orioles do find themselves in a similar winning situation next year and feel they need to strengthen the bullpen, there's no reason why they can't go out and find a useful relief pitcher on the trade block. Miller was outstanding for the O's, but the price was high. And it's not like he was the only effective reliever acquired at or near the deadline. He was the best, sure, but the Angels got Huston Street despite not having a very strong farm system. The Royals picked up Jason Frasor. The Nationals acquired Matt Thornton on waivers by just agreeing to pick up his 2015 salary. And that's just a few. A single reliever may matter more in the playoffs, but that's just not the case during the regular season.

The Orioles won't be able to re-create Miller's abilities with one or two moves or signings, but they can get close. And then they can use some or all of those savings for some corner outfield help, or maybe a bench bat. That's probably the avenue I'd take.

Photo by Keith Allison

The Orioles Sweep the Tigers: 4 Pivotal Moments

Photo by Keith Allison

As we all know by now, the Orioles swept the Tigers in the ALDS and now move on to face the Royals in the ALCS. Matt had a post yesterday about the Orioles success (or lack thereof) in shutting down the Tigers big bats. Today I wanted to look at a few of the bigger plays, moments, and decisions in the series that allowed the Orioles to advance.

1) The decision to bring in Andrew Miller in the 6th inning of Game 1

Tommy Hunter was warming during the 5th inning, so I assumed he'd be coming in for the 6th. But then Andrew Miller began warming. When Miller emerged from the bullpen to start the top of the 6th, it was pretty clear Buck would be counting on a Miller-O'Day-Britton combination to get the final 12 outs (Yes, that plan changed once the Orioles put 8 runs up in the bottom of the 8th). And why not?

This was great bullpen usage by Buck and a model that I think should be used by managers more often. Miller pitched 1 2/3 of hitless ball before giving way to O'Day. O'Day did give up a home run to Miguel Cabrera, but the Orioles had already added an insurance run at that point, and then they completely ran away with the game in the bottom part of the 8th. In what was the most important game of the season up to that point, Buck used a bullpen strategy different than he had used over the previous 162 games. He extended his best relievers a bit longer than he typically does and it paid off. Looking back to the 2012 ALDS, this isn't a huge surprise, as Buck relied heavily on his setup men (O'Day and Matusz) during that series as well.

2) The double play in Game 2

I was worried about having Ryan Flaherty at 3B. It wasn't so much that I think Flaherty's defense is inadequate, but more so that he isn't named Manny Machado. In the ALDS, Flaherty did plenty to ease the nerves of Orioles fans questioning his defense. There was no bigger example of this than the double play he started in the 5th inning of Game 2 (and yes, Schoop's arm was vital to the play as well).

The Orioles were trailing 5-3 at the time of this double play. A base hit to left would have put runners on 1st and 2nd and nobody out for Victor Martinez. Instead, Martinez comes up with no one on base, Gausman strikes him out, and the inning is over. The inning would have taken on a very different complexion had Flaherty not made the diving stop to start the double play.

3) Delmon Young's double in Game 2

Really, just writing "!!!!!!!!!!!!!" with a video clip of this moment would suffice. I know it was only the ALDS, but I'll remember exactly where I was sitting for this play for the rest of my life. I was lucky enough to be at the game and found myself jumping up and down with the joy of a 5 year old child, high-fiving everyone within reach. The stadium went bonkers, I went bonkers, you went bonkers, we all went bonkers. This was a really cool moment. This is why I watch sports. Down by 2 in the bottom of the 8th and Delmon Young hits a bases-clearing double for the lead. Unreal.



4) The close play at first base in Game 3

The close play at 1st in the bottom of the 2nd went the Orioles way. Had it not, the inning would have continued and the Orioles would have been down 1-0. I've watched this replay a few times and it's so close. If the call on the field had been safe, I'm sure that would have stood. It was that close of a play.

Oh, and despite what people on TBS tell you (people who are paid to talk about baseball), there is no such thing as "tie goes to the runner."

That was a fun series to watch. The outcome made it even more enjoyable. Any of these games (yes, even the game the Orioles won by 9) could have gone either way. However, thanks to many decisions, moments, and plays breaking the Orioles way, including these 4, the team moves on to face the Royals in the ALCS.

Why the Orioles Were Willing to Trade E-Rod for Miller

There’s been a lot of discussion on Camden Depot about the Andrew Miller trade. Most of our writers and writers in general have concluded that Eduardo Rodriguez was worth considerably more than Andrew Miller and that this was an overly expensive move. The poll on the Camden Chat link indicated that readers were split between being cautiously optimistic and feeling that while Miller was a good pickup he was not worth E-Rod.

Before the trade deadline I looked at how traded prospects do compared to average prospects and determined that they did significantly worse.  The interesting thing about specifically looking at Eduardo Rodriguez is that we already know a number of things about him. We know the Orioles were willing to trade him. We know that he wasn't very effective in Bowie. He is now fully healthy and has put together two dominant performances for Portland. I have been told that if he continues being dominant in Portland he is expected to be a top 100 prospect in 2015. I am an analyst and not a scout and am therefore reluctant to discuss whether this is the case. However, I know that as of midseason he wasn't ranked in anyone's top 50 prospects. For this article I am presuming that he will not be ranked as a top 50 prospect in 2015 by Baseball America and therefore will never be considered a top 50 prospect by Baseball America. If he does well in 2015 then I expect him to reach the majors and be ineligible for prospect ranking lists. 

This means that one way to determine E-Rod’s value is by looking at all pitching prospects that were ranked in the top 100 at least once but never ranked in the top 50. Some prospects ranked in the top 100 do make it to the top 50 in a future year. It seems reasonable to presume that those prospects will do better then ones that never make it to the top 100. In addition, E-Rod was traded this year so we can include him in a category with other traded pitching prospects. Here’s a chart showing value by year group and whether a prospect was traded.

Years	Traded	N Rows	Actual Value	Expected Value
1990-1993	N	42	1.283	0.960
1990-1993	Y	17	-0.523	0.960
1994-1997	N	42	2.548	2.481
1994-1997	Y	15	0.402	2.481
1998-2002	N	50	3.754	3.025
1998-2002	Y	19	-0.241	3.025
2003-2006	N	43	2.932	3.318
2003-2006	Y	16	0.852	3.318
2007-2010	N	46	2.156	2.189
2007-2010	Y	13	2.632	2.189

It turns out that traded pitching prospects that never become top 50 prospects were pretty much worthless from 1990-2002. Teams did a very good job at keeping the ones that were successful. They were more valuable in 2003-2006 and especially valuable in 2007-2010. There’s one problem though with this analysis. The best traded pitching prospect that was never ranked in the top 50 was John Danks. He was ranked in both 2003-2006 and 2007-2010 which means his numbers are included in both categories. Looking at valuations from 1998-2010 and 2003-2010 may be more helpful then just looking at 2007-2010 valuations.

Years	Traded	N Rows	Actual Value	Expected Value
1998-2010	N	134	2.944	2.836
1998-2010	Y	44	0.624	2.890
2003-2010	N	86	2.362	2.734
2003-2010	Y	27	1.113	2.816

If I am correct that E-Rod will never become a top 50 prospect then his value should $7 or $8 million as opposed to $15 or $20 million. One of the benefits of a small sample size is that it allows us to look at all the members of the sample in question. Attached is a link to a list of all the traded pitchers from 2000-2013 that were ranked in the top 100 but never in the top 50.

John Danks is the main success story. Guys like Daniel Hudson and Matt Harrison have done well also. Jake Odorizzi and Nate Eovaldi are promising young pitchers although neither looks particularly good according to RA9-WAR. A.J Cole and Trevor May are still young and may be productive. John Maine and Chris Perez had a few good years. Troy Patton was a decent LOOGY. But the vast majority of these guys have been worthless. A closer look shows limited value in this data set.

Jon discussed why Miller can be valued at $3 to $5 million based on fWAR. Using this valuation it appears that the Orioles only overpaid by $2 or $3 million. E-Rod still has more value than Miller* but it becomes more understandable why they’d make that deal. No one wants to pay more than necessary to make a trade but $2 to $3 million isn’t a huge blow. Given that the Red Sox and Orioles are in the same division it makes sense that we’d be forced to pay a “tax”.

* That sentence originally stated that Miller has more value than E-Rod. That was a typo.

Of course, using fWAR presumes that it values relievers properly. If reliever chaining undervalues high leverage relievers and therefore deflates their fWAR values then the Miller deal may be even better than it looks.