16 March 2017

The Orioles Outfield And Catch Probability

The newest tool created by the people that brought us Statcast is Catch Probability. Catch Probability attempts to determine the difficulty to catch each ball hit into the outfield based on how far the fielder had to run to catch the ball and how long he had to get there. Based on the ability to quantify each play, they’ve put many catches into five categories:

5 Star Plays: Converted between 0 to 25 percent of the time and require a speed of 30 feet per fastest second.
4 Star Plays: Converted between 26 to 50 percent of the time and require a speed of 29 feet per fastest second.
3 Star Plays: Converted 51 to 75 percent of the time and require a speed of 28 feet per fastest second. 2 Star Plays: Converted 76 to 90 percent of the time while requiring a speed of 27 feet per fastest second.
1 Star Plays: Converted 91 to 95 percent of the time and require a speed of 26 feet per fastest second.

MLB has also created a 2015 and 2016 Statcast Catch Probability Leaderboard on their data portal at Baseball Savant. For each outfielder in these years, Baseball Savant tells us, the number of opportunities in each of these five categories that an outfielder had and how many times he converted an opportunity. Using this data for each outfielder with at least 50 opportunities, it’s possible to determine the league average in a category and how many catches above or below average an outfielder is presuming he faced an average difficulty of catch in each category.

There are at least four reasons why such a metric will be imperfect. Statcast doesn’t provide position information, so I’m comparing corner outfielders to center fielders even though center fielders likely have better range. This likely overvalues center fielders and undervalues corner outfielders. Such a metric presumes that each catch has equal value, but some catches are more likely to prevent extra bases than others. This metric tells us nothing about a fielders’ performance when fielding a pitch likely to be caught 96 to 100% of the time. Finally, if a fielder faces easier or harder opportunities in a bracket, then their value will not be valued properly. Note that this mostly doesn’t even take into account the positional problem that Statcast says exists. Still, even if this metric is flawed, it’s likely to have some value.

Jeff Sullivan from Fangraphs found average catch probabilities and frequencies for each of the five categories that Statcast created. The highest frequency of opportunities are 1-star plays despite the fact that this category only covers balls that are caught 91-95% of the time. One would expect balls caught 50-75% of the time to have a higher frequency of opportunities. The next highest frequency are 5-star plays which are converted only 8% of the time. These results are similar to the data collected by Inside Edge. If the data continues to be similar, then the overwhelming amount of chances will be in the 96-100% bucket and will be largely converted by all outfielders with minimal defensive competence. Given that in a full season, an outfielder has about 300 putouts, and the highest number of opportunities on this list is around 170, it’s safe to say that the data will continue to be similar.

This suggests that outfield defense follows something akin to the Pareto principle. Elite outfielders that can cover a lot of ground are significantly more valuable than average outfielders because they can convert tough plays. Terrible outfielders that can’t cover any ground are significantly less valuable than average outfielders. But most outfielders are more or less interchangeable --- especially in the corners. This is because there are relatively few plays that a good outfielder can make that just a bad outfielder can’t.

In 2016, out of 162 total outfielders with 50 or more opportunities, 115 (70%) were between -5 to 5 catches above average. Per 77.6 opportunities (average opportunities per outfielder in the sample) ranked in one of the categories quantified by Statcast and shared with the public, 93 were between -5 to 5 catches above average. In 2015, 120 out of 162 outfielders were between -5 to 5 catches above average (74%). On a rate basis, 102 were between -5 to 5 catches above average. Again, elite outfielders are extremely valuable and terrible outfielders are a liability, but there’s little difference between good defensive outfielders and bad ones.

There are a number of interesting players. As Jeff Sullivan stated, Mark Trumbo was terrible in 2016. He only converted 11 of 19 1-star plays good for a 58% rate. On a rate basis, Trumbo was easily the worst outfielder in 2016. It is likely that there was some bad luck involved, as Trumbo had nearly the same conversion rate on 1-star plays and 2-star plays, but Trumbo’s inability to make difficult plays further shows his lack of range. Trumbo wasn’t as bad in 2015, but was still one of the worst outfielders whether considering rate stats or his actual performance. Statcast suggests that Trumbo is best at either first base or DH which is problematic given that the Orioles already have Chris Davis.

Nick Markakis also had bad results using this method. He ranked 154th in rate stats in 2015 and 112th in 2016 but 171st in 2015 and 148th using actual counting stats. This method likely underestimates Nick because it doesn’t take into account his sure hands when fielding balls hit near him nor his strong arm. However, it is another data point that supports the sabermetric consensus that his range is terrible.

Andrew McCutchen ranked 160th in 2016 and 164th in 2015 looking at his counting stats. Part of that is due to the fact that he had a lot of chances, but he was still worse than 5 catches above average in both 2015 and 2016. This could suggest that he’s a worse fielder than normal metrics suggested and therefore his value may be limited. This could be why he is being moved from center field to right field next year.

Adam Jones ranked 28th in 2016 with 4.5 catches above average (actual result) and 10th in 2015 with 11.2 catches above average. This suggests that Adam Jones does indeed have above average range. However, this leader board doesn’t tell us how many of the catches he misses go over his head and therefore result in extra bases for the hitter. Still, it’s likely that Adam Jones has been underestimated by the statistical models.

Joey Rickard ranked 26th in 2016 with 5.3 catches above average. While he struggled to make two-star catches, he converted all of his one-star opportunities and was surprisingly successful with three-star and up catches. With a slight improvement in his route running, Rickard may just be able to become an elite outfielder. Pairing him with Adam Jones next year could pay significant dividends defensively although whether or not he can hit is another question.

Mr. Kim had especially interesting results. His rank was 145th in 2016. However, he was as good as the average outfielder on one-star and two-star plays. He was only slightly below average on three-star plays, which given that he had only eight opportunities may just be due to small sample size. However, he was 0-23 for four-star and five-star plays. This suggests that his fielding skills are acceptable, but his range/speed isn't good enough to play in the majors without significant help from Adam Jones. On the other hand, it may mean that an outfield of Kim/Jones/Rickard may be successful defensively because the other outfielders could cover for Kim's lack of range.

This data seems to suggest that most outfielders perform reasonably similar to each other but a few outfielders can be outliers. Outfielders with elite speed can be considerably more valuable than the average fielder. Outfielders with poor speed can be considerably worse, suggesting that the lower bound for acceptable outfield defense is higher than I might have suspected. This also suggest that trying to use players like Alvarez, Mancini, Walker and Trumbo in the outfield is a bad idea that will likely fail.

Statcast defines a one-star play as one that requires a player to reach a peak speed of 26 feet per second. Players that can’t successfully convert these plays the vast majority of the time will likely be unsuccessful outfielders. If so, it’s safe to say that teams can determine whether a player is fast enough to be an outfielder if he can reach a peak speed of 27 or 28 feet per second when trying to field a ball. If a player isn’t that fast, then using him in this capacity is a waste of time. Obviously, players like Nick Markakis with especially good hands and a strong arm, or players that are able to run good routes can be acceptable with a slightly lower speed. Players that run bad routes need to be able to go somewhat faster than that. It seems fair to argue that slow players could indeed be legendarily bad if they’re put in the outfield.

My understanding is that Statcast will continue to roll out and improve this metric over the 2017 season. If they include this data in their play-by-play datasets, then this will significantly advance our understanding of outfield defense and allow us to learn what it takes to be successful in the outfield.


Roger said...

This still doesn't answer the elephant in the room..... How is Chris Davis in the OF? I recall the discussion yesterday about him being a leadoff hitter that he is not as slow-footed as some others. Whether he is a GG 1B or not, it seems better for the team for 1B to be open for Trumbo/Mancini/Alvarez than to insist on keeping Chris Davis at 1B and a black hole in the OF.

Matt Kremnitzer said...

Why does Chris Davis being an improved and solid defensive first baseman not matter? Does it make any sense to move him to the outfield if he's slightly less bad than the other options if those same options are (even slightly) worse options defensively at first? At the very least, we're talking about a minimal gain, which is far from an elephant in the room.

Also, none of those other players have contracts that are as lengthy and lucrative. It's in the team's best interest to not move him around and do what they can to keep him healthy.

Anonymous said...

I am 56, and I could still outrun Davis. The only slower one was Weiters, who runs like he has a piano strapped to his back!

Matt P said...

This dataset probably has minimal data for Chris Davis. Still, it does have some basic principles that a team can use to determine whether Davis might be acceptable.

I don't see the Os using Davis in the outfield though, which probably means that signing Trumbo was a mistake.

Pip said...

This is an extremely interesting article, I have two questions. The first is does this debt completely discount place for the outfielder didn't have to run at all? He only had to move a few steps to make the catch of those left out of the sample size completely? I would think so, but I don't recall seeing that in the article.
Secondly, Dan and his crew are not stupid. They have access to all this information and much more. Can you discern their logic in deliberately using an outfielder who is so
bad that his defensive value almost canceled out his offensive contribution?

Pip said...

Edit: "does this stat completely discount plays where the outfielder didn't have to run at all."

Sorry about that. I wish there were an edit feature here.

Matt Perez said...

Presumably, plays where the outfielder doesn't need to run at all would be in the 95-100% convert bucket. Those plays aren't in this dataset (and would be highly valuable if they were).

I don't know what Dan and the other guys are thinking. More study may discover a motive.

Roger said...

I disagree. If a Davis OF/ Trumbo 1B provides a measurably better defense even though Trumbo is not as good a 1B as Davis then it makes sense and really improves the value of Trumbo's signing and makes Mancini more valuable too. You should never look at the Davis contract isolated without context. Everyone is stuck on the idea that the Davis deal was bad and the Trumbo deal was worse. Make lemonade for chrissakes. The Trumbo deal had to be because of insufficient confidence in Mancini (or insufficient willingness to commit to him). Is Mancini faster than Trumbo or Davis? I don't know - no one has provided 40y dash times for any of them. Can Mancini play OF?

Matt Kremnitzer said...

Are you assuming that Davis has to be better in the outfield than Trumbo? Why? Being a little faster than someone does not make you a better outfielder. He might be slightly better; he might not. I'm arguing that it doesn't really matter. They've stockpiled a bunch of 1B/DH types, and they don't have room for all of them. Trying to fit Davis in in the outfield does not solve that.

Matt Kremnitzer said...

Also, I've never heard anyone say the Trumbo deal is worse than the Davis deal. Whether you're fine with Davis's contract or not, his deal and Trumbo's aren't even in the same stratosphere.

Jon Shepherd said...

I am unsure if it is my words that are being remembered differently than I wrote them. I think Mark Trumbo's contract is a good deal related to his value as a first baseman, but is an overpay as he is very unlikely to play first base. Davis' presence makes Trumbo's presence less advantageous.

I would not really call it lower than a below average deal simply do to positional considerations. Otherwise, I think it is an above average deal. I think his swing revamp really helped him develop into a more consistently above average bat, but he has a lot of problems as a total player.

Roger said...

The whole base of the article we're discussing is that a certain speed is required to make certain catches. That would inevitably imply that speed is required to be a good outfielder. Who among Davis, Trumbo, Mancini, Smith, and Alvarez is the fastest player and how do they rank? Is Davis faster than Trumbo? Play him in RF. Is this team better with Trumbo 1B/Davis RF or with Davis RF/Trumbo 1B? Whatever the answer is, play them that way and we can quit talking about it. We don't have a choice of who to have on the team. DD provides the players and Buck has to play them. You guys just analyzed lineups and determined which one comes up with the most runs. One of the more productive ones was one Buck is likely to use. Maybe not the best but a good one. The whole point here is to get your best bats into the field without negating their worth. Someone in this blogspot needs to take the players we have and tell us where the best place to play them is. I happen to think Davis is the only one of the bunch that has a decent likelihood of playing a passable OF and there are several that can play a passable 1B. Someone prove me wrong. If Hardy gets hurt, I say leave Flaherty on the bench, play Davis at 3B and give DH or 1B to Mancini. Players should not be played based upon their own best capability if it detracts from the team's potential for success. Maximize the team not the individual.

Jon Shepherd said...

Where is the best place to play the players? We really do not have the data to say that very well. Trumbo is awful in RF and Davis a couple years ago was poor. Davis is very solid at 1B and pre injury Trumbo a while back looked above average there. How do we really assess whether one or the other take RF?

I agree that you maximize the team, but a great deal of assumptions would go into anything that puts Davis in right field. The point of the lineup optimization model is that we have a lot of data to lean on. We simply do not have that in this situation.

RF play requires speed and skill. There are ways to approximate speed from offensive metrics. I cannot think of a way to assess skill in a similar fashion.

Roger said...

I am just basing my argument on the base assumption of this article. Catch probability goes up with speed. Seems like a good way to start would be to have them all run times in the 40 and tell anyone who runs 28fps or faster in their fastest second that they are being assigned to practice for the OF. The only reason we don't have that data is no one has run the tests and released the data. If this is what the data tells us then time to use it to the advantage of the situation which currently tells us we have at least four 1B/DH and a bunch of 4A or unproven OFs or proven terrible OFs. Like I said, time to make lemonade.

Jon Shepherd said...

I think, sure, the data is telling us that speed is important, but I think it is a great leap of faith to say it is all. Reaction time and projectile comprehension obvious play a role as well.

If 40 times were king then we would see a different way of going about things. I think the Royals academy in the early 70s showed that skill matters more than athleticism.

ballgame said...

Roger: I think your basic argument makes a certain amount of sense if we assume that a modest improvement in OF fielding has a significantly higher impact on preventing runs than a moderate improvement in fielding at 1B. (This seems plausible to me; I recall an old Bill James study where he found that the best fielding teams were more likely to boast great fielding outfields than great fielding infields, though his analytical tools back then were more rudimentary than what we have today.)

The fly in the ointment to your approach, though, is you're also assuming that a change in fielding position won't have a negative impact on that player's hitting. I don't think we can assume that, particularly when you're shifting a player from a position he plays well to one he's rated as sub-par.

Jon Shepherd said...

The translation to runs is much better now than what James had to work with. Still some areas could be ironed out more, but it is a bit of a leap to think outfield defense compared to average is more important than infield defense compared to average.

ballgame said...

"… it is a bit of a leap to think outfield defense compared to average is more important than infield defense compared to average."

I find your statement to be somewhat surprising, Jon. Aren't the vast majority of infield-fieldable balls going to fall into the "base hit/out" category (aside from a small percentage of drives down either line being possible doubles or triples)? By comparison, won't a significant portion of outfield-fieldable balls be potential "triple/double/base hit/out" types, with a very small number of HRs falling into the potential "out" column as well? Admittedly, an infielder will have a bigger impact on how many double plays the defense will convert. OTOH, a good outfielder doesn't just convert some hits to outs … he also converts some triples to doubles (or even singles), and some doubles to singles.

With all that in mind, it seems at least plausible to me that an above average outfielder could have a bigger impact than an above average infielder on reducing the total bases/runs produced by the opposing team.

Jon Shepherd said...

Defensive ability is expressed as runs saved and it is all anchored by average performance.

Your event based examples are incorporated into that construct. Run expectancy is based on where a ball is hit, how it is hit. This is also a cumulative statistic, so it adds up. A run saved is a run saved is a run saved.

I do not know what is surprising about that.