30 December 2014

Free Agents Will Break Your Heart

Everyone knows that prospects will break your heart. Many prospects touted as the next big thing have ultimately failed. But top free agents that are unsuccessful with their new team don’t seem to have the same stigma attached to them. It seems the common perception is that top free agents are more likely to be productive than the top prospects. Certainly, casual fans get excited when they see their team sign a top free agent and usually are willing to trade prospects for top players.

I wanted to see whether this perception is accurate so I looked at all prospects ranked in Baseball America’s Top Hundred list from 1998-2007 and all free agents that earned an annual salary of at least five million, have completed their contract, and whose contract information was either in MLBTR or ESPN’s transaction tracker. These free agents were primarily players that signed a contract starting from 2007 to 2014. A more detailed description of my methodology can be found here.

Position prospects ranked 1-10 produced an annual average of 2.08 fWAR while free agent position players earning $15 million annually produced 2.09 fWAR per year. In addition, 56% of the prospects and 60% of the free agents produced more than 1.5 fWAR per year. This indicates that these position prospects were about as successful as the top free agents.

Free agent position players that earned between ten and fifteen million and those that earned between five and ten million performed similarly. Position players earning between ten to fifteen million produced on average 1.24 fWAR per season while those earning between five to ten million produced 1.13 fWAR. The position players that earned between ten to fifteen million did have slightly higher success rates as 41% were successful and 22% were stars. In contrast, 34% of the position players that earned between five to million were successful and 19% were stars. But in general, there’s little difference between position players that earned ten to fifteen million and those that earned five to ten million. This suggests that the best strategy for teams is to go after top position free agents or to stick with the ones that are affordable.  

Free agent position players that earned between $5 and $15 million annually performed similarly to position prospects ranked 51-100. The prospects produced an annual average of 1.06 fWAR while the free agents produced an average of 1.17 fWAR. 32% of the prospects and 36% of the free agents produced at least 1.5 fWAR per year. They also performed similarly to position prospects ranked between 26 and 50. Position prospects ranked 26-50 produced 1.34 fWAR per year while 35% produced at least 1.5 fWAR per year. It is worth noting that position prospects ranked 26-50 from 2003 to 2007 averaged 1.5 fWAR per year while 46% produced at least 1.5 fWAR per year.

Position prospects ranked 11-25 produced 1.63 fWAR per year while nearly 47% produced at least 1.5 fWAR per year. These prospects are easily better than free agents earning between $5 and $15 million annually but not as good as the top position free agents.

The chart below shows position category results.



Pitching prospects ranked 1-10 weren’t as successful as the top free agent starting pitchers. These prospects produced 1.98 fWAR on average per year while the top free agent starters produced 2.74 fWAR per year. 61.11% of the prospects and 70% of the free agents produced at least 1.5 fWAR per year. However, pitching prospects ranked 1-10 from 2003-2007 performed similarly to the top free agent starting pitchers. These prospects produced 2.36 fWAR on average per year while 81% of these prospects ultimately produced at least 1.5 fWAR per year. Pitching prospects ranked between 1 and 10 from 2003 and 2007 may not have had the same ceiling as the top starters but they arguably had a higher floor. Still, it appears that the top free agent starters are better than the top pitching prospects.

Pitching prospects ranked 11-25 performed similarly to free agents starting pitchers that earned between $10 and $15 million annually. From 1998-2007, the pitching prospects produced an annual average of 1.58 fWAR and 43% produced at least 1.5 fWAR per year. However, from 2003-2007 the prospects produced an annual average of 2.18 fWAR and 58% produced at least 1.5 fWAR per year. Free agent starting pitchers earning between $10 and $15 million annually produced an average of 1.85 fWAR and 53% produced at least 1.5 fWAR per year.

Pitching prospects ranked 26-50 from 1998-2007 produced an annual average of 1.01 fWAR produced an annual average of 1.05 fWAR while 31% produced at least 1.5 fWAR per year. Those ranked between 26 and 50 from 2003 to 2007 produced an annual average of 1.05 fWAR while 35% produced at least 1.5 fWAR. Free agent starting pitchers that earned between $5 and $10 million annually produced an average of 1.04 fWAR while 30% produced at least 1.5 fWAR per year.

Pitching prospects ranked between 51 and 100 from 1998 to 2007 produced an annual average of .73 fWAR while 20% produced at least 1.5 fWAR per year. There was no difference between the 1998 to 2007 and the 2003 to 2007 results.

If one believes that Baseball America is getting better at predicting pitching prospect performance over time then it makes sense to use the 2003-2007 prospect results. In fact, it makes sense to presume that current prospects will be even better than those ranked from 2003-2007. Therefore, it suggests that pitching prospects ranked from 1-10 have similar production to free agent starters earning $15 million or more, pitching prospects ranked from 11-25 have similar production to those free agent starters earning $10 and $15 million and those pitching prospects ranked from 26-50 have similar production to free agent starters earning between $5 and $10 million.

If one believes that Baseball America simply had good luck when selecting top prospects from 2003-2007 then this suggests that pitching prospects ranked from 1-10 are similar but better than those free agents that earn between $10 and $15 million and prospects ranked from 26-50 are similar to free agents that earn between $5 and $10 million.  

The chart below shows pitching category results.


Ultimately, the following groups have similar production:

  • Free Agents earning at least $15 million per year = Prospects Ranked 1-10
  • Free Agent Starting Pitchers earning between $10 and $15 million annually = Pitching Prospects Ranked from 11-25
  • Free Agent Starting Pitchers earning between $5 and $10 million annually = Pitching Prospects Ranked from 26-50
  • Free Agent Position Players earning between $5 and $15 annually = Position Prospects Ranked from 51-100
Prospects ranked by Baseball America have become significantly more productive and valuable over the years. Today the best free agents have similar success rates and production to the best prospects. Maybe it is time to start saying that free agents will break your heart.

8 comments:

Anonymous said...

Really nice thorough analysis! I, like you and many people, have wondered about this over the years, and now you have shown very interesting results to think about. Thanks!

Mike Bonsiero said...

This is really good work. I was talking with a friend of mine earlier this off-season about the Cruz and Markakis deals and we went through and (non-scientifically) looked at all the 4 year or longer contracts that were signed just last off-season. Only about 20% of them can you look at, just a year later, and say "yeah, I'd take on the rest of that deal right now".

The MLB service time/arbitration/FA system, coupled with the typical player's career arc means that a big majority of players have already reached their maximum usefulness before they get near the open market.

Matt Perez said...

Glad you liked it.

In my last post I noted that about 70% of wins come from players that are still under team control. There's enough talent in free agency to add 10-15 wins over the average club. That's enough to turn an average club into a wild card or division leader with average team controlled talent. But a team with only minimal team controlled talent will struggle to win 80 even if they're spending $200M.

The scary thing is that I'd say that probably 7 of the 12 players that meet your criteria have probably performed as well as should be expected or better.

Unknown said...

Good stuff Matt. Any idea how the 2003-2007 pitching prospects shake out in terms of coming from college or high school? It may be nothing, but you mentioned that they could have been a lower ceiling/higher floor aspect going on. This made me think of one of the early 2000 A's (aka "moneyball") strategy to draft college players over high school players, and the possibility of other teams copying that strategy.

Again, just curious and probably isn't anything there, because as you mentioned, it's possible that BA has just gotten better at ranking prospects.

Matt Perez said...

Thanks Nate.

Ranked high school pitchers are usually better. But the sample size is tiny. There were 13 pitchers ranked between 1-10 from 2003 to 2006. Only five went to college.

I discussed the impacts of high school vs college somewhat in this post.

http://camdendepot.blogspot.com/2014/02/the-impact-of-age-on-prospect-success.html

Erik said...

The problem with a high-priced free agent bust is that it locks the franchise in financially for a long time. That is not true of the top prospects. When Markakis flattened out it left the Orioles spinning their wheels for years, and he did not "bust." Neither did Wieters, except if you go by the estimates of his potential at draft time.

What we are seeing is a failure to get true stars out of top draft picks. A damn shame that. But not as bad as $150 million dollar men not panning out.

Pat Holden said...

This is great work, Matt

ZenRoto said...

As an Indians fan who welcomed Nick Swisher and Michael Bourn two years ago, I can vouch for the truthiness of this.

Free agents tend to be older (given that they need six years of experience), so the likelihood of injury is that much greater. Plus, I tend to think that players in the last year of their contracts tend to play through the injuries that they nurse a year later (when richer).