Is Defense Overrated?

Fangraphs recently released some Inside Edge fielding data to the public. As stated in the article, Inside Edge scouts watch every play and determine how difficult it is to field on the following scale:

• Impossible (0%)
• Remote (1-10%)
• Unlikely (10-40%)
• About Even (40-60%)
• Likely (60-90%)
• Almost Certain / Certain (90-100%)

Unlike zone-based applications like DRS and UZR, Inside Edge defensive system results are determined primarily by scouts. This data gives insight to how scouts grade defense and determine similarities and differences between zone-based and scout-based systems.  

The Inside Edge fielding data available to the public states how many chances and the success rate for each fielder in every category listed above. For example in 2013, Nick Markakis had 112 impossible chances of which he converted zero, five remote chances of which he converted zero, five unlikely chances of which he converted two, six about even chances of which he converted all six, eighteen likely chances of which he converted fifteen and 289 almost certain chances of which he converted 288.

I used this data to create a metric I think of as Catches over Average (COA). It is derived by downloading all Inside Edge fielding data for each position and determining the average success rate for each of the six categories described above. Next subtract the average conversion rates for each category from the players performance and multiply by the number of chances. Finally, sum the results for each category. Many concepts incorporated in this metric were originally discussed here.

This metric has a number of shortcomings. It presumes that each chance in a category is of equal difficulty. It doesn’t measure how successful an infielder is at turning a double play. It doesn’t measure how successful an outfielder is at ensuring singles don’t turn into doubles or outfielder arm strength. Nor does it measure how well a catcher frames pitches or prevents passed balls or throws out potential base stealers. Inside Edge collects this data but doesn’t share it with the public. Despite these shortcomings this metric still can be used to compare Inside Edge fielding data results to UZR and DRS.

This table shows how some of the Orioles main players performed defensively in 2012 and 2013.

There are many similarities between COA and UZR or DRS. Matt Wieters, Manny Machado and JJ Hardy are considered excellent defenders while Ryan Flaherty and Chris Davis are considered above average. Each indicates that Mark Reynolds and Wilson Betemit are unable to play third base very well while Adam Jones is a poor defender in center field. The main difference is that Nick Markakis is a good defender according to COA and a bad one according to UZR or DRS.

In 2012, our outfield defense ranked 16th in the majors with a -1.73 COA while our infield defense ranked 7th with a 9.13 COA. In 2013, our outfield defense ranked 26th in the majors with a -2.40 COA while our infield defense ranked 2nd with a 30.9 COA. Having a full season of Manny Machado at third base instead of using Wilson Betemit and Mark Reynolds had a significant impact on our infield defense. This statistic indicates that the Orioles have excellent infield defense and mediocre outfield defense which is the common consensus.

The difference between the best and worst infield defense was 48 COA in 2012 and 52 COA in 2013 while the difference between the best and worst outfield defense was 30.5 COA in 2012 and 20 COA in 2013. These numbers seem low when compared to UZR especially when one considers that a catch doesn’t equal a run. In order to do a full comparison it is necessary to determine how many catches equal a run.

Tom Tango states that each catch is worth .8 runs. This implies that the amount of runs saved can be determined by multiplying the value of a catch by a players COA. I’ll refer to this stat as Defensive Runs over Average or DROA. *

*  It was brought to my attention that Tom Tango has discussed the value of saving a play. Originally, I quoted Michael Lichtman who stated that a typical outfield hit is worth .83 runs and simply estimated the value of an infield hit. I believe that estimate was incorrect.  

It is possible to compare DROA to UZR. The table below shows the ranges for UZR and COA for each position.

Position	COA Range	Value Of Catch	DROA Range	UZR Range	UZR/DROA Range
1B	16.56	0.80	13.25	30.90	2.33
2B	25.94	0.80	20.75	31.70	1.52
3B	26.01	0.80	20.80	48.00	2.31
SS	25.22	0.80	20.18	45.20	2.24
CF	14.60	0.80	11.68	42.50	3.64
LF	12.15	0.80	9.72	31.30	3.22
RF	13.31	0.80	10.65	47.10	4.42

The range for UZR is about 2.25 times larger than DROA for infielders (except second base) and 3.75 times larger than DROA for outfielders. Instead of a top defender like Machado being worth 3 wins defensively this stat indicates that he’s worth one win. This implies that defense is less valuable than one may have thought based on UZR. **

** Due to the new data, I've updated the ranges quoted above.

The good folks at Retrosheet attempt to document every baseball game played. They share their data with the public provided that the following disclaimer is used:

Information used here was obtained free of charge from and is copyrighted by Retrosheet. Interested parties may contact Retrosheet at www.retrosheet.org. 

I downloaded the data from the 2012 and 2013 seasons from their site and determined how many balls in play allowed by each team. I split the balls in play into outs and non-outs (including hits and bases reached on error). Then I did the same thing with the Inside Edge Data. Below are the results.

The numbers of outs for balls in play are different in each dataset by a minimal amount while there are more than twice as many balls in play in the Retrosheet dataset for each team than in the Inside Edge dataset. The average team had a BABIP of .298 using the Retrosheet data and a .188 using the Inside Edge data. The Inside Edge data is omitting a large number of hits.

UZR and DRS split up the whole field into zones. Each ball in play has to fall into a zone which is the responsibility of at least one but potentially more fielders. If a ball isn’t caught then it is someone’s fault and it impacts their UZR/DRS rating. The Inside Edge data seems to indicate that part of the field is no fielders’ responsibility. Other parts of the field may be a fielders’ responsibility but an uncaught ball hit there has no impact on his defensive rating if it’s considered an impossible chance like most non-out balls in play. Balls hit into play that do have an impact of a fielders rating are usually ones that every fielder can field with nearly perfect accuracy. As a result, there is less variation in defensive ratings and an elite player according to UZR may be worth 3 wins defensively while an elite player according to DROA is worth 1 win defensively.

It is possible to determine whether UZR or COA is more likely to predict future performance by doing a correlation analysis. I found that UZR had a correlation of .393 from 2012 and 2013 while COA had a correlation of only .249. This suggests that UZR is more likely to predict future performance though neither is particularly accurate. I also found that while range runs above average is the best factor to predict UZR that error runs above average is the best factor to predict COA which supports the above paragraph. This explains why players similar to Nick Markakis that have limited range but make few errors have much higher defensive ratings using Inside Edge data rather than UZR while players like David Lough that have excellent range but make errors have much higher defensive ratings based on UZR rather than Inside Edge.

The average AL team allowed 696 runs in 2013. According to UZR, the Royals defense (best in the AL) was worth 68 runs. This means that a team with average pitchers and the Royals defense would have allowed the fourth fewest runs in the American League. Using DROA, the Orioles defense (best in the AL) was worth 16 runs. A team with average pitchers and the Orioles defense would have given up the ninth largest amount of runs or still been about average. The amount of importance that UZR places on defense as opposed to Inside Edge is huge.

If UZR is correct in its rating of defense then focusing on defense should be a cost-effective way to improve performance. If Inside Edge is correct then excellent defense players offer smaller contributions than UZR and WAR would suggest. I believe that the Inside Edge data suggests that defense is overrated.