Do the Orioles have the 4th worst farm system in baseball?

Victor Wang, I assume, is a pretty smart guy.  He began to break out on the sabermetric scene when he was 16 years old, worked his writing and analytical skills for a few years, joined the Cleveland Indians, and is now the Assistant Director of Professional Scouting.  He is one of the guys made good.  One of many, many baseball analysts who have presented their work on forums, blogs, and journals and then wound up employed in Major League Baseball.  While the mainstream has been arguing the value of modern qualitative and quantitative measures against traditional qualitative and quantitative measures, baseball moves along slowly, but surely.  Success (or perceived success) produces mimicry and we now have a flooded market of front office analytical types that seem to greatly benefit their teams.

Back to Victor Wang and, perhaps one area where analytics seem to have some trouble, the valuation of minor league systems.  The August 2007 edition of the Sabermetric newsletter, By the Numbers, included a piece by Wang where he attempted to measure the value of prospects.  The idea was to use established sources, Baseball America's top 100 prospects and John Sickels' yearly prospect grades, to determine not only the likelihood of production by being a recognized prospect, but also how much money that meant to the team in control of said prospect.  This article had a great deal of fanfare attached to it as we could then use his work to calculate how productive a team's minor league system might be.

This is quite an important concept because by changing over the murky qualitative expressions of how good certain prospects are, even the most attentive follower of baseball can be overwhelmed of the scouting process and their myriad determinations of a player's current and future worth.  The confusion is a good reason why so much white noise has been generated in the past five years or so with everyone trying to generate prospect rankings and scouting reports that are often based on a few sources (e.g., Baseball America, Baseball Prospectus, Keith Law, John Sickels).

There is nothing inherently wrong with a freelance blogger trying to figure out the scouting game.  Former writer, often podcast guest, and friend of the Depot, Nick Faleris (now with Baseball Prospectus) made his way from forum evaluator to Camden Depot writer to BP writer through great skill and sheer determination in a way that was very transparent.  That transparency is something I think is essential and was the main reason why Nick and I eschewed our forum handles on this site a few years back.  That transparency in process and in identity is important when the audience is trying to gauge the utility of the information.  It is a major reason why I tend to ignore (and probably you should, too) prospect sites that simply give lists or short evaluations without any description of the process or clear identification of the contributors.

So, in this prospect evaluation fog, Wang's work appeared to be a possible light house to lead the way to some concept of reality.  By using trusted sources, with clear track records and some semblance of methodological explanation, you could determine just how good your team's best prospects likely were and what the whole system could mean to the success of the parent club.  Beyond the Box Score was one of a few who used the system to devise a way to express organizational worth back in 2009.  I was not able to find any more recent, so either this one off was do to simple lack of future interest or it may well be that the crew at BtBS determined that maybe there were issues using Wang's work in this manner.

If you read the article in By the Numbers (do it now if you have not), his conclusions were basically that prospects are worth a lot and that position players are more likely to achieve a higher level of performance.  What it does not say is that position players are worth more than pitchers.  How can one player perform better and not be worth more than another player?  Scarcity.  If one players is part of a population that is needed, but has a higher rate of failure, then that player may still be worth the same or more than another player who is part of a group that has a much lower level of failure.  Though the 2009 thoughts of Wang seem to indicate that he did not follow this perspective at some point in time.

Anyway, these are not exactly new sentiments here at Camden Depot.  I have addressed, somewhat briefly and quite clumsily, this approach on two occasions.  In 2009, I used it to assess the 2008 draft and how the Orioles' and the Camden Depot Shadow Orioles' drafts differed from what I suggested would have been an approach more in line with what Wang suggested.  That is it is more effective to build with elite hitting prospects and second tier pitching prospects.  Wang's paper showed that a top 100 hitter performed better than a top 100 pitcher.  However, a Sickels' B level, non-Baseball America top 100 pitcher performed better than the categorically equivalent hitter.  This meant that a draft consisting of 1B Justin Smoak, RHP Tim Melville, and LHP Tim Murphy would have trumped the actual draft of LHP Brian Matusz, OF Xavier Avery, and OF L.J. Hoes.  I think we can all agree that the actual Orioles draft has been superior.  Although, one comparison does not prove the Orioles having taken the right approach just that their approach appears far more useful in this single instance.

A year later, I wrote about the approach again on the Depot.  It was spawned from a reader who asked me to comment on an evaluation similar to Beyond the Box Score's evaluation that appeared as a fan post on John Sickels' Minor League Ball.  In my response, I attempted to articulate what I mentioned before: scarcity.  People certainly like to overstate TINSTAAPP (There Is No Such Thing As A Pitching Prospect).  However, it is not like we can simply ignore pitchers and simply expect the baseball to find its own way from the mound to home plate within the vicinity of the strike zone.  Pitchers simply are a frustrating necessity in that teams must put in a great deal of effort (just as much as acquiring and developing a hitter), but wind up with considerably more failure (reviewing 2004 data for prospects I found that 44% of ranked pitching prospects suffer significant career altering injuries within 6 years of being ranked versus 14% for position players).  That scarcity to me means that even though pitchers are likely to produce less value than a hitter, pitchers are worth more than their production would seem to suggest given this methodology.

With that said, I created the following graph using an adjusted methodology from the initial paper.  The changes simply are accounting for the increase in free agent salaries and collapsing price differences between C level prospects and their age.  The latter change was simply one that I felt would vastly speed up my analysis and would not greatly alter the final tallies.

The following is the organizational values for each team based on the 2013 top 100 prospects ranking from Baseball America, the prospect grades assigned by John Sickels, and using the adjusted Wang methodology.

Click on graph to increase size.

As expected, teams that are deep in positional talent (e.g. Minnesota with Miguel Sano, Byron Buxton, and Oswaldo Arcia) are valued much higher than those who have considerable pitching talent (e.g., Baltimore with Dylan Bundy and Kevin Gausman).  Additionally, teams with incredibly deep B level reserves of pitching (e.g., Tampa Bay) are valued much higher than teams with shallower systems (e.g., Baltimore).  Again, this methodology would place great importance on elite positional prospects as well as teams with deep second tier pitching prospects.

As mentioned earlier, I find this approach does not adequately consider scarcity.  In response, I decided to multiply pitching prospect worth by a coefficient to reduce the effect of injury flameout.  To equate the two groups, I multiplied pitching prospect worth by 1.55.  This is not a perfect methodology as it assumes that our ability to assess the future impact of pitchers and position players are equivalent.  Nor does it more greatly define scarcity.  To summarize, this is simply an injury adjustment.

Click on graph to increase size.

There is a slight readjustment in organizational value by measuring in this way.  For instance, Baltimore received more credit for having two elite pitching prospects, but the shallow nature of their system works against them from jumping ahead too far with them placing 25th instead of 27th.  The team that moved forward the most were the Atlanta Braves who had five B level pitching prospects and two pitchers ranked in the top 100.  San Diego went from 10th to 6th based on seven B level pitchers and two top 100 ranked pitchers.  Both the Chicago Cubs and the Detroit Tigers fell back five slots due to their positional heavy systems.

As it stands, I do not think that the second graph is the "reality" of organizational value, but it does provide a decent starting point to address other potential issues in order to harmonize the statistical perspective and the way in which Baseball America, John Sickels, and others (including MLB) values prospects.  Some have suggested that teams like Baltimore are being unfairly labeled as a mid to low quality minor league system because of the supposed value of having two potential aces on the farm.  This small effort in this post does not reconcile that view with what various publications have been reporting.

As it stands, I think we are still limited in our knowledge about the true value of these players.  Simple evaluation of the success of past prospects is inadequate in determining the true value of players because position scarcity plays a major role in that value.  Future studies need to more completely define that scarcity to determine if, in fact, there is a significant population of MLB teams that inadequate evaluate their players in a way that other teams could exploit.