The way I decided to test this was pretty simple. I took all draftees listed as a catcher from 2004 through 2008. I decided to ignore any position switches because what we are interested in are catchers at the MLB level. Whether or not they hit well enough to assume another position (e.g., Gaby Sanchez) does not resolve the issue of catching depth in a system. I gave them credit for service at a level (e.g., MLB, AAA, AA, A, Hi A, A, Short A, Rookie) if they had over 100 games played at that level or accumulated at that level and a above. For instance, if a player had 70 games caught at AA and 40 at AAA, that player would be credited with having reached AA with 110 games at or above that level. All this was how the study measured performance, nothing more.
To determine how a catcher's ability was viewed at the time of being drafted, I consulted Baseball America's draft database scouting reports. I decided to be conservative in what I labeled as a defensive catcher. The following are several of the terms used that would result in a player being classified as such:
best defender in the class, pure catch and throw, solid average, at worst fringe average, best defensive catcher seen since Charles Johnson (a label that adorned Matt Wieters' scouting report, which is contrary to my memory)Draftees labelled as poor defenders included labels like these:
needs improvement behind the plate to stick, defense remains his nemesis, passable, adequate, no scouts see him at catcher, deserves a look behind the plate, will move to thirdThese designations left us with 37 catchers labeled as defensive drafted in the first five rounds and 28 labeled as non defensive. This may not be the most robust data set, but it might be able to yield some answers if the differences between these two groups are large.
How well do defensive catchers advance through the levels?
If you simply count each level attained as a single point, sum them up for each player individually, and group them by their defense, then defensive catchers significantly (p=0.001) rack up levels played as a catcher in professional baseball. The table below shows the total number of players in each group that are credited with time spent (100 games or more at that level and above) and the percent of that group that made it to that level.
As you can see, defensive catchers are likely to make it to the Majors 2.4 times more likely than non-defensive catchers. Keep in mind that this success rate does not include defensive catchers Kyle Skipworth, Austin Romine, and Travis D'Arnaud who all stand a very reasonable chance of achieving that 100 game MLB mark in the next year or two. That would raise the success rate to 51% and increase of probability to 2.8 times more likely to make the big show than non-defensive catchers. In other words, if you are an organization looking to improve catching depth within your system then you should stick to the amateurs who already show proficiency in the position. However, if you really love a bat and the player just happens to be listed as a catcher then feel free to select him knowing that it is highly likely that he has to exchange his catching mitt for a fielding one.
One question that may be asked would be whether or not non-defensive catchers are more likely to be selected in the later portions of the five round database of drafted catcher that I put together. In fact, the average round players were drafted in was higher for the non-defensive catchers (2.5) than the defensive catchers (2.8). For the most part, defensive and non-defensive catchers drafted in the first two rounds were in similar proportion in their groups. The non-defensive catchers attained a higher average draft round because teams drafted few of them in the fifth round in comparison to defensive catchers.
Below is a table parsing out the round data for each group as well as the MLB attainment rates for those groups.
A first round catcher with existing professional defensive skills is 4.5 times more likely than a non-defensive catcher. If you add Skipworth and D'Arnaud in that then the percent goes up to 82%, 5.9 times more likely than a non-defensive first round catcher. Romine's addition would put the second round at 71%. Keep in mind that these are small sample sizes, but these are rather impressive differences that suggest that there is a trend here.
The conclusion is this: if you want a catcher, then draft an individual who has already displayed professional catching skills. If you really like a bat and the player just happens to have played catcher in high school or college, then do not talk yourself into dreaming about him as a catcher and respect him simply for his bat. Otherwise, an organization may just find itself a few years down the road with no internal solutions behind the plate.