For Top 100 Prospect Lists, F Is A Passing Grade

Executive Summary:

• Team controlled players contributed 36% of total wins from 1997-2006 and roughly 52% of wins from 2007-2015 indicated the increased value of a successful farm system.
• Team-controlled position players and team-controlled pitchers contribute roughly the same percentage of production.
• The top twenty-five team-controlled pitchers produced 60% of total production and the top twenty-five team controlled position players produced 50% of total production.
• Baseball America successfully identified fewer top ten team-controlled position players and team-controlled position players ranked 26^th to 50^th than they have in the past. 
• Ranked team-controlled position players produced 72% of production from 1997 to 2005 but only 61% from 2006 to 2015.
• Baseball America has volatile results when predicting pitching performance. They were especially successful from 2008-2011 but have largely failed to maintain that success.

There have been a number of articles over the years attempting to determine the value of ranked prospects.  These articles, with the exception of my own, have largely followed the same methodology: determine how much production prospects ranked  for given years at a given spot produced over six years. My work at Camden Depot on the subject differs because I determine the actual time that each prospect is under team control and use actual salaries instead of approximated salaries for each prospect.

The major problem with these methodologies is that it requires most players ranked in a given year to have completed their team controlled years so that their performance can be assessed. This means that it doesn’t allow the audience to determine an idea of prospects’ current worth, but rather only a retrospective view. It’s helpful to know how prospects performed in the mid-2000s, but it would be far better to know how they performed in 2015.  

Therefore, instead of judging prospects by when they were ranked, I decided to determine how much production team controlled players have provided in each year from 1996 to 2014 provided that they were still under team control. A prospect that was ranked #8 in 2011 and didn’t make it to the majors until the middle of 2013 would have his 2013 contributions calculated in 2013 and his 2014 contributions calculated in 2014. In 2020, when he used up his six years of service time, his contributions would not count towards this ranking because he would no longer be a team-controlled player even if he signed an extension. This allows the reader a more current view.

I use RA9_WAR to determine production, Lew Pollis’ documentation of the value of a win while valuing a 2014 win at $7.5M and a 2015 win at $8M, determine actual salaries via Cot’s and the Lehman Database and determine service time also via Cots where available and by my own algorithms and Baseball Reference when not available.  

The chart below shows how much production that ranked and unranked position and pitching prospects contributed each year from 1997 to 2015. While ranked position players have contributed the most production, the graph is reasonably volatile. There was a period from 2008 to 2011 where ranked pitching prospects were at their peak value. Before that, from 2003 to 2006, ranked pitching prospects produced less value than usual. Meanwhile, unranked position prospects did well in 2014 and 2015 and poorly in 2003 and 2004.  All in all, ranked position team-controlled players have contributed the most value, followed by ranked pitchers, unranked position players and unranked pitchers as one would have expected.

 

This next graph shows how much production team controlled players have produced when ranked, unranked and total by both year and whether they’re position players or pitchers. This graph measures production as a percent and divides the production produced by players in a given category by either all pitchers or position players.



The first thing that is that team controlled position and pitching players have produced a similar percentage of total pitching and position production. One problem when measuring pitching and position production is that position players can earn 570 WAR in a year while pitchers can only earn 430 WAR. This makes sense because position players catch pitches and therefore can frame them as well as field balls put into play, but it does mean that comparing position and pitching production can be like comparing apples and oranges.

The second thing is that team controlled players, both pitching and position, have become more valuable over time. From 1998-2006, team controlled players produced only 36% of total production while from 2007-2015, they’ve produced over 50% of total production. It has certainly become more important for teams to be able to successfully develop their top talent due to the value that younger, team controlled players provide.

This next graph shows the production that ranked players have provided divided by total production for both pitchers and position players by year. This graph shows that ranked position players aren’t producing as much value as they once did. For whatever reason, Baseball America seems to be doing a worse job adequately grading position players than they have done previously. It also shows that Baseball America has periods where they accurately grade pitching prospects and where they do less well.

The margin of error appears to be small for Baseball America and this is because most production is produced by a few top players. On average, the top ten team-controlled pitchers contribute 30.4% of total production while the next fifteen contributes another 30.6%.  Team controlled pitchers ranked below 100 contribute -37.9% of total production due to being primarily below replacement level. It should come as no surprise that the Gini coefficient for team-controlled pitchers is .911.

As a result, team controlled ranked pitchers were extremely successful in 2008-2011 because on average they successfully ranked 8 of the top 10 team-controlled pitching prospects instead of the average 6. They only hit on seven of the top ten in 2008, but they also ranked the top five in that year. In 2011, they only missed on the #4, #6 and #10 pitching prospect and this also inflated their results.  The chart below shows the percent of pitchers ranked whose production was ranked between 1-10, 11-25 and 26-50.

Likewise, the top ten team controlled position players contributed 25.2% of total production while the next fifteen contributed 26%. Team controlled position players ranked below 100 contributed -18% of production due to being below replacement level. The Gini coefficient is slightly smaller for team-controlled position players, but still high at .871.

As a result, Baseball Americas numbers look considerably worse because they were only able to rank 5 of the top 10 position prospects in 2014 and 2015 compared to their average of between 7 and 9. It doesn’t help that they’ve also been unable to rank more than 50% of position players that produced between the 26^th and 50^th total WAR for position players.

To sum up, team controlled players are becoming more valuable now than they have been in the past. They contributed 36% of total wins from 1997-2006 and roughly 52% of wins from 2007-2015. This indicates that youth is becoming more important.

Pitchers contribute only 430 WAR per season while position players contribute 570 WAR per season. Therefore, in absolute terms, team-controlled position players contribute more production than team-controlled pitchers. However, in relative terms, team-controlled position players and pitchers contribute roughly the same percentage of production, indicating that it’s important to develop both position players and pitching.

Relatively few team-controlled players contribute the vast majority of total team controlled production. The top ten team-controlled pitchers produced 30% of total production and the next fifteen produced another 30%. The top twenty five position players produced half of total production.

Baseball America isn’t doing as effective of a job ranking position players. They’ve predicted fewer top ten position players in recent years and have done a poor job ranking position players ranked 25^th through 50^th over the past two years. As a result, ranked team-controlled position players produced 72% of total production from 1997-2005 and only 61% from 2006-2015. They only produced roughly 50% in 2014-2015.

Baseball America’s performance is very volatile when it comes to pitching prospects. They were successful ranking pitching prospects from 2000-2003 and 2008-2011 and less successful during other periods. This is because missing on a few top pitching prospects can have drastic impacts. It isn’t clear whether Baseball America has gotten worse at predicting pitcher performance.

It seems that baseball prospects are becoming more and more valuable as players are expected to be productive at younger ages. This makes publications like Baseball America more important and means that reliable rankings are important to adequately assess the status of baseball teams.