Bad Outfield Defense or a Bad Fielding Metric?

When I decided to write that title I did not mean that fielding is unfair between teams.  Rather, I wonder whether the way fielding is measured is fair.  I was wondering this because there has been discussion about Carl Crawford's defense not translating well to Fenway Park.  It made me wonder whether there was anything peculiar about Camden Yards.  My only resource (FanGraphs) informs me about career UZR home and away, so I decided to use Oriole outfielders who have been nearly exclusively part of the Baltimore Orioles:

• Nick Markakis has played 7874 innings for the Orioles in RF.
• Adam Jones has played 3405 innings for the Orioles in CF and 227 for the Mariners.
• Felix Pie has 852 inning for the Orioles in LF and 13 for the Cubs.

Now from this we can basically declare that all three of these Orioles have overwhelmingly played at these positions for the Orioles.  With these players, we can attempt to measure if there are any unaccounted park factors.

Here is a short description of UZR if you need to be refreshed on how it is calculated.

How to calculate UZR: The baseball field is divided into 78 zones, 64 of which are used in UZR calculation. (As Lichtman explains, infield line drives, infield pop flies, and outfield foul balls are ignored. Pitchers and catchers are not included.)

Here's what is calculated for each zone: the out rate and the percentage of balls in that zone that turn into outs. The league average out rate is then subtracted from the player's out rate — if this number is negative, it means the player is worse than league average. If it's positive, it means he's better than league average.

That rate is then multiplied by the number of balls that hit in that player's zone. This yields a Zone Rating. To obtain the run value, it's multiplied by the Zone Ratings that are calculated for each zone the fielder covers, and then summed. This sum is a simple, unadjusted UZR. It is then further adjusted for park factors, batted ball speed, which side of the plate the batter was hitting from, the pitcher's groundball/flyball ratio and the number of baserunners and outs at the time. The adjustments are made because each of these variables can significantly affect the average out rate in a particular zone. Using run expectancy charts, these rates can be converted to runs.

UZR / 150

Click to Enlarge

Each player does remarkably worse at home than on the road.  It is actually a pretty remarkable finding.  Pie has only about 135 games worth of innings in left field, which is not nearly enough to get a good idea of how dependable a fielder he is.  UZR typically requires 2-3 years of data to get a good read on a player.  Adam Jones has about 546 games worth of innings.  With that amount split between assumed home and road games, it is arguably just enough to be usable.  Markakis has about 875 games worth of innings and is perfectly fine as a data source.  It seems clear enough that this is a real effect.  Playing in Camden Yards in any position in the outfield decreases your defensive metrics.  This means that as useful as UZR might be, it appears to do a poor job characterizing what normal means for Camden Yards.  The alternative explanation is that the Orioles outfielders are actually rather poor defensive players at home.

The question now lies as to whether we can discern what part of UZR has the problem and whether it makes any sense.

Arm

 

 No pattern appears with arm values.  This makes sense as throws are contained within the playing field.  It would be unlikely if a stadium could play much havoc with throws outside of wind issues, which apparently is not the case with Camden Yards.

Incidence of Errors

Incidence of errors also does not appear to be greatly affected by Camden Yards.  This is also expected as grounds crews do a fairly good job ensuring that each stadium has an excellent field.

Range

Range is where we see the issue.  I am not entirely sure what the problem is.  Range is basically determined by how plays are in a players' area and how many he winds up catching.  Somehow, Camden Yards is a difficult place to track down baseballs.  I am not sure if there is an issue with see the ball come off the bat, if high flies are greatly affect by wind, or something else.  Unfortunately, I do not have any data for how visiting teams perform here.  UZR does account for park factors and one would think such a shift in fielding would be figured into the final number.

What if the numbers are correct?

It just might be that UZR is actually accurately measuring defensive ability at Camden Yards. The Orioles may be horribly position themselves and/or are inadequately instructed in how to play in their own ball park.  This reminds me of an article from a couple years ago, but I fail to remember who exactly wrote it.  Peter Gammons, I think, mentioned how the Red Sox did not care that Jacoby Ellsbury had a -9.7 UZR after the 2009 season.  They said that their internal metrics measured defense better at Fenway Park better than UZR did.  Ellsbury looks like a good centerfielder, so maybe they were correct.  Likewise, maybe UZR measures defense in Camden Yards as well as it does in Fenway Park.

Conclusion

I do not have a solid conclusion after looking at this data.  If the Orioles are doing this poorly at Camden Yards and UZR adequately adjusts for park factors then it would mean that every other team  on average is playing about 2 WAR better defensively.  While also meaning that when the Orioles are on the road, they outperform various home teams about the same.  I just have a hard time understanding how the numbers can be accurate here.  I inclination is to think there is a significant failing in UZR in the outfield at Camden Yards.