15 August 2018

Prospect Rankings Are More Precise Than Prospect Values

One difficult aspect of assessing the value of prospects is trying to figure out just how much is one worth.  When I was reviewing the trades this past season or in years before, I have mainly relied on top 100 prospect lists and, sometimes, John Sickels B and C age based ratings for those who do not make the list.  However, recently FanGraphs, Baseball Prospectus, MLB.com, 2080 Baseball, and others have begun using the 20-80 scale to designate prospect values.

Fangraphs performed an exercise to determine value based on rankings and some peculiar associations came up.  The linearity of the values seemed off and I wanted to take a look by directly comparing them with top 100 rankings, which the methodology is a bit more robust than going back in time and retroactively applying grades.

I went ahead and looked at pitchers.  In the table below you see pitcher rankings in the first column, 2018 attributed value for those rankings, and in the third column is the averaged MLB pipeline scouting grade.
2018 Grade
Pi 1-10 73 64
Pi 11-25 46 58
Pi 26-50 32 55
Pi 51-75 20 55
Pi 76-100 15 55
If you apply the values derived from the FanGraphs venture, you would get a 62 MM value in the first ten, 34 MM in the 11 to 25 rankings, and 22 MM throughout the rest of the list.  However, we do find significant variation from 26-100 on this list.  Variation that the grades are unable to discern, but that the rankings do.

What this suggests to me is that using grade values for prospect worth is viable only after exhausting top 100 rankings.  Additionally, that value should be limited by the value associated with a prospect ranked on the backend of the top 100 list.

For instance, Dillon Tate and Luis Ortiz are rated as 50 prospects.  Using the FanGraphs value system, the associated value would be 14 MM (slightly more as this number is for 2017 and not adjusted for 2018 values).  The lowest value in the top 100 ranking is 15 MM for a pitcher, but those values (55s) would be considered 22 MM.  There is a good bit of contradictory information and potential issues with cross site evaluations.

What I would suggest is to value prospects off the top 100 in this way: in 2009, the initial study was to look at Baseball America rankings and fill in with Sickels' rankings for those player who did not appear on the top 100.  For pitchers, a quality pitcher who was not on the top 100 was considered worth 70% of what a backend top 100 prospect pitcher was worth.  A position prospect was worth about  just off the top 100 had an associated value about 45% of that backend value.  These kinds of players are what MLB pipeline would refer to as 50s.

For MLB pipeline 40/45 prospects, young pitchers (22 yo or below) would be worth about 21% of that backend value while those 23 and older would be worth about 16%.  For hitters, the younger bracket would be 6% and the older group 4%.

So let us go back and take a look at one of the deals.  The Gausman and O'Day deal brought back Jean Carlos Encarnacion (45, 20yo, 1.2 MM), Brett Cumberland (45, 23yo, 0.8 MM), Bruce Zimmerman (unrated), and Evan Phillips (unrated).  This provides a traumatically different view on the take than in my original column.  In that column, I put forward the notion that because a lot of handwaving occurred when putting together the final batch of the top 100 that the different in value between the ones on the list and just off it is not all that much.  

Maybe that still is true though.  Maybe guys like Encarnacion or Cumberland are, for some, in the conversation in that next batch.  And, maybe, 45 is a huge bucket where players fall.  When you look at the rankings, there are about 400 known prospects in baseball, according to MLB pipeline as 45s.  That is stunning and probably wrong.  A 45 generally means someone is good enough to sit on the bench and I doubt there are 400 players (ignoring the 200+ that are at 50 and above) that will eventually be dependable bench players.

This makes me think that just knowing that a player is a 45 is not very useful information.  We know that a 55 ranking in the top 100 has less resolution than the ranking, so the a generic 45 means fairly little as well.  This all means that while we have a good base to evaluating deals on a top 100 prospect basis that deeper dives into the prospect world are far more difficult to evaluate and require us to have strong scouting skills or to rely on those with strong scouting skills.


Unknown said...

Do you know if those future value numbers are the player's expected ceiling or most likely outcome (or perhaps it varies depending on the outlet)? If the former, then the 400 players with a 45 value is more believable. If the latter, then I agree that that is way too many.

Jon Shepherd said...

There is some variation about what it means, but it basically means the value of a player given the market, risk, and time to prime time. That is why relievers slide down the scale. A 45 on a position player prospect means a bench player.

Matt Perez said...

Fangraphs still links to this article. https://www.fangraphs.com/blogs/scouting-explained-the-20-80-scouting-scale/

It claims that a '45' player is a #5 starter, low setup or utility/platoon player and a '40' player is a swing/spot RP, middle relief or bench player.

I suppose this grading system could make sense. I mean, I guess all we care about is how the prospects would do in the majors. So, it wouldn't be crazy to just have a very few 65+ guys, and a bit more but not very many guys that go from 40-65, with pretty much everybody lower than 40.

A few hundred is not quite what I have in mind though.

Unknown said...

From my understanding a 45 rated 25yearold triple A player and a 45 rated 19 year old have differnt meanings to the 45 rating. The kid has a ton of risk but a chance to be a better player so the 45 is the median rating. The 25year old is a 5th/6th starter or bench/platoon bat.

Unknown said...

Its not perfect but theres not really any other way to compare players of different ages/floors/ceilings in a quick and easy manner.

Unknown said...

I prefer the current fangraphs team work over the other free prospect websites since they do a better job of avoiding the bloat on the 40 to 45 rating players and generally explain specific ratings much more in depth.

Jon Shepherd said...

Thanks AJ.
Yeah, I have buddies in scouting and front offices. I comprehend the scale and variations of it. It is difficult to implement further refinement because you have so many personnel used to an older system. It can take four or five years to master even a slight change and during that time you have aspects of reports you may not quite understand.