A Look At The Distribution Of Prospect Values

Over the past few years, there’s been a movement to grade prospects by not only overall rank and organization rank, but also by future value. Future Value is not very well defined, but per MLB is a 20-80 scale in which a ranking of 20-30 is well below average, 40 is below average, 50 is average, 60 is above average and 70-80 is well above average. A player with a 65 FV is someone who could develop into a future impact Major Leaguer, perhaps an All-Star-caliber standout. Fangraphs defines future value, using a similar grading model and provides a chart showing how future value can be defined using WAR.

In general, I’m not a fan of using such definitions as Top #2-3 player or #2/#3 starter because it’s subjective and can’t be easily determined and prefer using WAR values because that at least is easily definable and somewhat objective. WAR may undervalue a strong player that is hurt for a few years, but any clear definition will have its limitations.

In order to learn about future value, I decided to build a metric called actual value. Future value attempts to predict how much production a prospect will produce using WAR, and actual value attempts to measure how much production a prospect actually did produce. In essence, I defined actual value in such a way that future value attempts to predict actual value. 

To do this, I looked at players from 1990-2016 and measured each prospects’ production while they were under team control or until they had six or more years of service time. Once players have six or more years of service time, they are eligible for free agency. Then, I determined their rookie year and measured their average value per year or actual value using the following scale.

With this distribution of actual value illustrating how prospects performed, we can understand what a reasonable distribution for future value might look like for the future. I looked at the performance of prospects over two time periods, 1990-2016 and 2004-2013 defined by a prospects rookie year. Here were the valuations this process returned from 1990-2016.

Each year from 1990-2016, there were roughly 18.5 prospects that had an actual value per year of 2 WAR or more. This may seem surprising at first glance, but one should consider that from 2000-2017, there were roughly 130 offensive players and 90 pitchers that were worth more than 2 wins or above average. This includes players that were one year wonders like 2009 Jason Kubel (2.5 fWAR) or 2016 Michael A Taylor (3.2 fWAR). They may have been worth 2 fWAR in a single season, but weren’t above average players over an extended period of time. When looking at players that have staying power, there are significantly fewer than 220 at any given time – maybe around 150 total. If there are few average and above average players in the league, then there can be few prospects each year to replace them. If 20 prospects each year are above average, and there are 150 players total above average, then the replacement period would be about 7.5 years. This seems to suggest that 20 average prospects per year is within reason.

There were significantly more prospects each year that ended up being below average and the vast majority were either 25s or 30s. This makes a lot of sense because teams have injuries and need to call up their prospects to make it through the season. In addition, teams would prefer to give their prospects chances to be successful because they’re cheaper than free agents. However, the vast majority of prospects either fail or struggle to be better than replacement level.

The story is reasonably similar when looking at prospects that became rookies from 2004-2013. There were about 20 rookies per year that because average or better players in the majors. Another 22.5 per year were below average, but still worth over 1 WAR per season. The vast majority of rookies, or roughly 184 per year, ended up being worth less than 1 WAR per season or clearly below average. Both sets of data tell the same story: being successful in major league baseball is difficult.

If only twenty players per year have above average actual value, than prospects ranked outside of the top 100 should not be expected to have a high future value. After all, future value is trying to predict and should have similar behavior to actual value. Prospect lists are tricky because they grade all prospects and not just those that should be expected to enter the league each year. So, I would argue that it’s reasonable to presume that all prospects considered to be at least average should be in the top 100, and likely all prospects with a future value of 45. I would say that all prospects with a future value of 40 should be at least in the top 125 prospects and all prospects with a future value of 35 should be in the top 200 prospects. Players ranked worse than 200 should be seen as slightly better than replacement players at best and ideally will be either bench players, middle relief or minor league depth pieces.

Fangraphs provides a board with its top 850 prospects as ranked at the beginning of the season with both future value grades and estimated time of arrival. We can use this board to determine the time period that their prospect list measures as well as see whether their distribution seems reasonable.

If they still use the future value definitions defined by McDaniel, than their distribution is difficult to comprehend. In their defense, it appears that they consider their FV to be their peak level rather than their average level. In addition, they deserve some leeway for injuries. Still, it is highly unlikely that there were will be 54 prospects in the 2018 rookie class that end up being average or better or 110 prospects that are worth 1 fWAR or more. It seems that a 55 in the Fangraphs distribution is the same as a 50-55 in the historical distribution, a 50 in Fangraphs is the same as a 35-45 in the historical distribution, a 45 in Fangraphs is the same as a 30 in the historical distribution and a 40 in Fangraphs matching a 25 in the historical dataset. Perhaps this is why Fangraphs only gives valuations for players that are ranked up to 45?

Even prospects graded as a 30 have a surprisingly high amount of value using a WAR linear model. These players typically produce about .2 fWAR per year and last for about 4.5 years. Using our predicted arbitration rates would suggest that these players produce $9M in value (at $10M per win) and cost $2.5M for a total surplus value of $6.5M. This seems surprising at first glance, but consider that players like Caleb Joseph and Ryan Flaherty rate as having an actual value of 30. These players aren’t great, but Joseph is certainly a decent backup catcher option while Flaherty was a decent utility player. Despite the fact that these players are valuable, it is impossible to build an average team with players just like those two, explaining why teams are willing to trade them away for marginal improvements. Treating WAR as a linear model does have its limitations.

Players ranking 35, 40 and 45 in actual value have a preposterous amount of value using this metric. 35 actual value level players produce roughly 4.2 WAR of production over their six years of team control or produce $42M in value (at $10M per WAR) but only cost about $10.6M putting their value at $30M. 40 level players produce roughly 7.2 WAR of production over their six years of team control or $72M in value but only cost $15.6M putting their value at $56M. 45 level players produce roughly 10 WAR of production over their six years of team control, but cost roughly $24M putting their value at $76M. Presumably, Orioles fans would not have been happy if the Orioles received just one 45 level player (top 60-90 prospect) in return for Machado.

Part of the reason why these players have such high values is that non-arbitration player salaries are unfairly low. If a win is valued at $10M, and the salary for a non-arb player is $550k, then it is relatively easy for a non-arb player to be underpaid. A reasonable minimum salary of $1.5M would rectify the situation where near replacement players have significant value. Part of the reason is that having a slightly above replacement player play instead of a below replacement player can have significant value. The fact is that teams are willing to pay a premium to avoid having to use a player like David Hess for 65 innings.

Future value may be the attempt to determine actual value, but that doesn’t mean they are the same. There’s a significant amount of uncertainty when predicting future value, and the distribution of prospect performances suggest that it is far easier to overestimate a prospect than underestimate a prospect. Most prospects that make it to the majors aren’t successful, and all prospects that don’t make it to the majors aren’t successful by definition. This uncertainty will mean that a player with an actual value of 40 (as determined in hindsight) is far more valuable than a prospect with a future value of 40. In a player with an actual value of 40 is worth $56M, than a prospect with a future value of 40 is worth significantly less. This is one reason why prospects aren’t valued as actual value suggests. In contrast, measuring the value of top 100 prospects take failed prospects into account and therefore provide a reasonable baseline for value.

In addition, a team filled with 35 AV players that are under team control, may be fiscally successful, but will also be terrible. In some roles, a 30 or 35 AV player can be acceptable, but in other roles (starters) they’re not particularly ideal. Obviously, it’s important to be fiscally responsible, but teams are in the business of maximizing their win total and not their excess value total. Others believe that teams would rather win now rather than later, and therefore future wins should be discounted.

Going forward, looking at how many prospects historically fit into each category of actual value should help analysts build future lists using future value. It doesn’t make sense to predict that hundreds of prospects will have a future value of 45 if few players ultimately have an actual value of 45. Using historical data to accurately assess the value of prospects will lower these rankings and make it more difficult to excite casual fans about their teams’ future. Still, even if it’s bad press, it makes sense to admit the reality of prospect value. Having a better idea how prospects historically perform can help teams plan for their future, and ultimately fans don’t really want to see how their prospects develop but rather watch their team be successful at the major league level.