The Orioles Outfield And Catch Probability

The newest tool created by the people that brought us Statcast is Catch Probability. Catch Probability attempts to determine the difficulty to catch each ball hit into the outfield based on how far the fielder had to run to catch the ball and how long he had to get there. Based on the ability to quantify each play, they’ve put many catches into five categories:

5 Star Plays: Converted between 0 to 25 percent of the time and require a speed of 30 feet per fastest second.
4 Star Plays: Converted between 26 to 50 percent of the time and require a speed of 29 feet per fastest second.
3 Star Plays: Converted 51 to 75 percent of the time and require a speed of 28 feet per fastest second. 2 Star Plays: Converted 76 to 90 percent of the time while requiring a speed of 27 feet per fastest second.
1 Star Plays: Converted 91 to 95 percent of the time and require a speed of 26 feet per fastest second.

MLB has also created a 2015 and 2016 Statcast Catch Probability Leaderboard on their data portal at Baseball Savant. For each outfielder in these years, Baseball Savant tells us, the number of opportunities in each of these five categories that an outfielder had and how many times he converted an opportunity. Using this data for each outfielder with at least 50 opportunities, it’s possible to determine the league average in a category and how many catches above or below average an outfielder is presuming he faced an average difficulty of catch in each category.

There are at least four reasons why such a metric will be imperfect. Statcast doesn’t provide position information, so I’m comparing corner outfielders to center fielders even though center fielders likely have better range. This likely overvalues center fielders and undervalues corner outfielders. Such a metric presumes that each catch has equal value, but some catches are more likely to prevent extra bases than others. This metric tells us nothing about a fielders’ performance when fielding a pitch likely to be caught 96 to 100% of the time. Finally, if a fielder faces easier or harder opportunities in a bracket, then their value will not be valued properly. Note that this mostly doesn’t even take into account the positional problem that Statcast says exists. Still, even if this metric is flawed, it’s likely to have some value.

Jeff Sullivan from Fangraphs found average catch probabilities and frequencies for each of the five categories that Statcast created. The highest frequency of opportunities are 1-star plays despite the fact that this category only covers balls that are caught 91-95% of the time. One would expect balls caught 50-75% of the time to have a higher frequency of opportunities. The next highest frequency are 5-star plays which are converted only 8% of the time. These results are similar to the data collected by Inside Edge. If the data continues to be similar, then the overwhelming amount of chances will be in the 96-100% bucket and will be largely converted by all outfielders with minimal defensive competence. Given that in a full season, an outfielder has about 300 putouts, and the highest number of opportunities on this list is around 170, it’s safe to say that the data will continue to be similar.

This suggests that outfield defense follows something akin to the Pareto principle. Elite outfielders that can cover a lot of ground are significantly more valuable than average outfielders because they can convert tough plays. Terrible outfielders that can’t cover any ground are significantly less valuable than average outfielders. But most outfielders are more or less interchangeable --- especially in the corners. This is because there are relatively few plays that a good outfielder can make that just a bad outfielder can’t.

In 2016, out of 162 total outfielders with 50 or more opportunities, 115 (70%) were between -5 to 5 catches above average. Per 77.6 opportunities (average opportunities per outfielder in the sample) ranked in one of the categories quantified by Statcast and shared with the public, 93 were between -5 to 5 catches above average. In 2015, 120 out of 162 outfielders were between -5 to 5 catches above average (74%). On a rate basis, 102 were between -5 to 5 catches above average. Again, elite outfielders are extremely valuable and terrible outfielders are a liability, but there’s little difference between good defensive outfielders and bad ones.

There are a number of interesting players. As Jeff Sullivan stated, Mark Trumbo was terrible in 2016. He only converted 11 of 19 1-star plays good for a 58% rate. On a rate basis, Trumbo was easily the worst outfielder in 2016. It is likely that there was some bad luck involved, as Trumbo had nearly the same conversion rate on 1-star plays and 2-star plays, but Trumbo’s inability to make difficult plays further shows his lack of range. Trumbo wasn’t as bad in 2015, but was still one of the worst outfielders whether considering rate stats or his actual performance. Statcast suggests that Trumbo is best at either first base or DH which is problematic given that the Orioles already have Chris Davis.

Nick Markakis also had bad results using this method. He ranked 154th in rate stats in 2015 and 112th in 2016 but 171st in 2015 and 148th using actual counting stats. This method likely underestimates Nick because it doesn’t take into account his sure hands when fielding balls hit near him nor his strong arm. However, it is another data point that supports the sabermetric consensus that his range is terrible.

Andrew McCutchen ranked 160th in 2016 and 164th in 2015 looking at his counting stats. Part of that is due to the fact that he had a lot of chances, but he was still worse than 5 catches above average in both 2015 and 2016. This could suggest that he’s a worse fielder than normal metrics suggested and therefore his value may be limited. This could be why he is being moved from center field to right field next year.

Adam Jones ranked 28th in 2016 with 4.5 catches above average (actual result) and 10th in 2015 with 11.2 catches above average. This suggests that Adam Jones does indeed have above average range. However, this leader board doesn’t tell us how many of the catches he misses go over his head and therefore result in extra bases for the hitter. Still, it’s likely that Adam Jones has been underestimated by the statistical models.

Joey Rickard ranked 26th in 2016 with 5.3 catches above average. While he struggled to make two-star catches, he converted all of his one-star opportunities and was surprisingly successful with three-star and up catches. With a slight improvement in his route running, Rickard may just be able to become an elite outfielder. Pairing him with Adam Jones next year could pay significant dividends defensively although whether or not he can hit is another question.

Mr. Kim had especially interesting results. His rank was 145th in 2016. However, he was as good as the average outfielder on one-star and two-star plays. He was only slightly below average on three-star plays, which given that he had only eight opportunities may just be due to small sample size. However, he was 0-23 for four-star and five-star plays. This suggests that his fielding skills are acceptable, but his range/speed isn't good enough to play in the majors without significant help from Adam Jones. On the other hand, it may mean that an outfield of Kim/Jones/Rickard may be successful defensively because the other outfielders could cover for Kim's lack of range.

This data seems to suggest that most outfielders perform reasonably similar to each other but a few outfielders can be outliers. Outfielders with elite speed can be considerably more valuable than the average fielder. Outfielders with poor speed can be considerably worse, suggesting that the lower bound for acceptable outfield defense is higher than I might have suspected. This also suggest that trying to use players like Alvarez, Mancini, Walker and Trumbo in the outfield is a bad idea that will likely fail.

Statcast defines a one-star play as one that requires a player to reach a peak speed of 26 feet per second. Players that can’t successfully convert these plays the vast majority of the time will likely be unsuccessful outfielders. If so, it’s safe to say that teams can determine whether a player is fast enough to be an outfielder if he can reach a peak speed of 27 or 28 feet per second when trying to field a ball. If a player isn’t that fast, then using him in this capacity is a waste of time. Obviously, players like Nick Markakis with especially good hands and a strong arm, or players that are able to run good routes can be acceptable with a slightly lower speed. Players that run bad routes need to be able to go somewhat faster than that. It seems fair to argue that slow players could indeed be legendarily bad if they’re put in the outfield.

My understanding is that Statcast will continue to roll out and improve this metric over the 2017 season. If they include this data in their play-by-play datasets, then this will significantly advance our understanding of outfield defense and allow us to learn what it takes to be successful in the outfield.