Showing posts with label Model. Show all posts
Showing posts with label Model. Show all posts

26 February 2014

Orioles Are Locked for 90+ Wins

The title is an arrangement of the tweet I received below when discussing how Ubaldo Jimenez and Nelson Cruz affected Clay Davenport's projections along with my educated guess as to how Ervin Santana would change those projections:
This perspective as well as the bounty of tweets, follows, and unfollows the Depot collected in response to us being a bit nonplussed about the path chosen by the team this off season as well as to what these additions actually meant with regard to meaningful September and October baseball.  Sort of related due to the similarity in numbers between what he earned and what Nelson Cruz will earn, this Orioles Hangout poll from 2011 was something I also found interesting. When Andy MacPhail signed an old, broken down Vladimir Guerrero, it was done with an incredible amount of fan fare.  That masterstroke, according to that poll of 255 Orioles faithful, resulted in over 67% of them giving him an A- or better (89% gave it a B+ or better).

The Guerrero signing was memorable for me because of two things.  One, I had a series that year that followed Vlad's attempt up old DH mountain.  He finished with a bWAR of 0.4, good for 20th out of 25 all-time.  Two, it resulted in this article railing against my pessimistic view and suggesting that it might well be Vlad's curtain call.  In a series of tweets (of which I have no idea how to find), the author stated that me equating Vlad's 2010 offensive output with Matt Wieters' 2010 offensive output given the context of their respective defensive positions made me a "liar".  Of course, even if my opinion was faulty, that would make me simply misguided as opposed to being a liar.  Really, though, I think what the author was really trying to express was that he was very much emotionally involved with the team and highly invested to see them succeed.  That can be difficult to explain or even comprehend about oneself, so strangely calling someone a liar may suffice.

So where this leads me is about emotional expectation and the use of rather unaware projection modeling.  Why are projections unaware?  They are unable to adequately assume player usage, past (to some extent) or future injuries, weight training, etc.  Basically, all of the reasons why many folks claim that there projections are useless.  However, their inability to clearly predict the future does not mean that are worthless in terms of projecting the future.  In other words, a team projected to win 55 games will not make the playoffs.  The models know enough about the histories of player populations to realize that this event is literally almost impossible.At a projected talent bases increases, then those probabilities grow larger and should give some hope to fans (along with a dose of realism).

In order to show this, I took a projection (devised with PECOTA, ZiPS, or MARCEL) and compared that with the actual results from (2003-2011).  I did not double count years.  From 2003-2009, I used PECOTA projections I had on hand.  From 2010-2011, I used MARCEL.  From 2012-2013, I used ZiPS.  The PECOTA projection model was reported by Baseball Prospectus.  The MARCEL and ZiPS projection models were reported by Replacement Level Yankees Blog.  It may look messy to take things from so many sources, but the point here was not specifically to test a specific model.  It was to casually use models blindly under the assumption they perform rather similar.

Year        St DEV Model
All 9.3
2003 8.8 PECOTA
2004 11.7 PECOTA
2005 7.6 PECOTA
2006 7.5 PECOTA
2007 6.4 PECOTA
2008 9.7 PECOTA
2009 11.7 PECOTA
2010 9.6 MARCEL
2011 10.1 MARCEL
2012 10.9 ZiPS
2013 8.7 ZiPS
So, what does the table above mean?  Hopefully, the graphic below helps.  Each standard deviation includes a certain amount of the population.  If we assume that win deviation is normally distributed, then we would assume that a team will perform within 9.3 games better or worse about 68% of the time.  To cover 95% of all events, a range of 18.6 games better or worse would be expected.  Using this approach, you would expect a team to perform 27.9 games better or worse would happen about 1 times in about 11 seasons.  In our data set of 11 seasons, this has indeed happened only once (2004 Arizona Diamondbacks, 81 projected wins, 51 actual wins).

http://rchsbowman.files.wordpress.com/2009/01/010309-1504-statisticsn2.png

The last two years there has been some grumbling from the fan base that ZiPS has been unfair to the Orioles in its projections.  Both seasons, the Orioles have, as a team, outperformed the projection using ZiPS.  Of course, a sample size of two is not a powerful sample size and it would make more sense to assume that it was a statistical anomaly unless we identify some mechanism that ZiPS and/or the team projection model has issues with.  For instance, if Buck Showalter is the difference between a 69 win team and a 93 win team then neither projection system will be able to pick that up.  Additionally, Buck needs to talk to his agent because if he was worth 24 wins then he needs to be paid about 144 MM a year.

Below is a sampling of the last two seasons the Orioles enjoyed as well as Clay Davenport's current projection of the team winning 83 games after adding Ubaldo Jimenez and Nelson Cruz.



exWins range n stdev low high % to 93 % to 96
2012 69 66 to 72 42 8.9 -15 24 2.4 0
2013 79 76 to 82 100 9.4 -30 19 7 2
2014 83 80 to 86 123 9.5 -30 19 16.3 6.5

For better or worse, I expanded the projected win totals in order to get larger sample sizes to work with.  In that first line, the Orioles were projected to win 69 games in 2012.  The Orioles outperformed that mark by 24 games.  To make the Wild Card (93 wins is a decent number to use for that), a team at 69 wins needs to outperform by exactly 24 games.  The Orioles are the only team in that group to perform so well.  Historical events suggest a 2.4% possibility.  In 2013, the Orioles were projected to win 79 games and outperformed that mark by 6 games.  That was not good enough for the playoffs.  What they needed was in the neighborhood of outperforming their mark by 14 games.  In that data set, only seven out of 100 teams have manage to do that.  Of those seven, two did well enough to improve to a point with the divisional crown was a likelihood.

What does the history of teams in the 80 to 86 win bracket look like with respect to under and over performing their projected wins?



The above is a weighted distribution graph.  Just based on this one grouping, it appears that teams that crash, crash to varying degrees.  Perhaps, this has to do with increased play of prospects, dealing of players, or something along those lines.  Still, it holds up pretty well as data that appears normally distributed.  The Orioles would be looking to improve by 10 games over this projection, which has happened about 16% of the time in the past.  Greedy for a division crown?  That number drops to 6.5%.  Those odds would be 1 in 6 and 1 in 15, respectively.  Keep in mind that in Davenport's projection that the Orioles would need to leap frog several teams.  Briefly, it is more likely for the Orioles to over perform and another team to under perform than it is for them to over perform and two teams under perform.  That whole concept though will not be addressed in this post.

Going back to the original tweet suggesting that 90 wins are a lock, a team must to projected to win 98 games or more to have not fallen below 90 wins.  Six teams have been described as 98 win or better teams.  Six out of 330.
Proj. 90+ wins n
98+ 100% 6
97 50% 2
96 50% 2
95 67% 3
94 50% 4
93 43% 7
92 43% 7
91 50% 10
90 80% 5
For projected 83 win teams, four out of ten won 90 games.  In other words, it is possible for the Orioles to be a 90 win team.  History suggests that.  However, that same history also suggests that it is not likely.

Addendum (Model Projections)
Davenport 83-79
FG (STEAMER) 78-84
PECOTA 78-84

19 May 2008

Revisiting the Season Prediction

Several weeks back I predicted the number of runs the Orioles would give up and the number of runs the Orioles would score. The basis of this prediction depended on a few assumptions:

1) ZiPS/Morong Formula (my arrangement) would properly predict offensive and pitching performance.
2) Offensive replacements would cause a 10% reduction in run scoring while unearned runs would be ignored for pitchers.
3) Top 5 starters would start every game and provide an average of 6 innings pitched.
4) Relief pitchers would be league average.

1. ZiPS/Morong predicting performance.
ZiPS actually overpredicted the runs scored (with the run reduction application). ZiPS predicted that 193 runs would be scored. In actuality, 179 were scored. Even more of an issue was prediction of pitching performance. 222 runs were predicted, while 184 were actually scored. A major contribution to this error was the unexpected development of Daniel Cabrera and a bullpen that was much better than expected. It should be mentioned that my placeholder of a league average bullpen was actually somewhat optimistic. This formula under predicted the Orioles success.

2. Offensive reduction and static pitching.
My educated guess of a 10% reduction was pretty apt. Plugging in the actual OBP and SLG of each player resulted in a coefficient of 0.927 to reach the actual runs scored. The pitching prediction appeared a bit too kind. After plugging in the actual SP and RP era, the system predicted 177 runs, where there were actually 174 runs scored. The application of a coefficient (1.057) would have been appropriate to account for unearned runs.

3. Top 5 Starters would remain so and would average 6 IP.
It was to be expected that a starter or two would be injured. It was known this was a weak assumption. Loewen's injury made it so. The 6 IP prediction is actually almost right on the button.

4. RP would be league average.
Orioles RP are actually pitching 13% better than the league average bullpen.

New Adjustments


1. ZiPS is being replaced by PrOPS and xFIP.
As the season continues, in-season statistical methods may actually predict future performance better than season beginning predictions. The reason for this is that certain growth or degradation may not be apparent prior to the season. PrOPS takes in peripheral batting data to predict OBP and SLG. xFIP takes peripheral pitching performance data and predicts future ERA. Current relief pitching ERA will be multiplied by the coefficient factor mentioned in the next paragraph.

2. Performance Coefficients

The batting performance reduction coefficient will be changed from 0.9 to 0.927. The pitching performance coefficient will be changed from 1 to 1.057.

3. Record Calculation
The current record is considered a given, so the new predicted winning percentage will be applied to games yet to be played. The number of wins determined by the formula will then be added to the current total.

Team Used for Calculations
PrOPS/Morong
2 Roberts.........374obp/424slg
3 Mora............348/454
R Markakis........414/521
D Huff............324/441
L Scott...........328/395
1 Millar..........354/444
C Hernandez.......326/332
C Jones...........293/354
S Placeholder.....300/330

xFIP
S Olson...........3.75
S Guthrie.........4.19
S Cabrera.........4.19
S Trachsel........5.92
S Burress.........4.64
R Bullpen.........3.42

Results

In the games left, this method predicts the Orioles will score 542 runs and give up 537 runs. The season ending run totals would be 722 runs scored and 721 runs given up. In the remaining games, the winning percentage would be .505, which would end with us having a .518 winning percentage at the end of the year (Pythagorean Win Expectancy). This means that the current rendition of the model predicts we wind up with a 84-78 record, which is 2 wins above the PWE.

Discussion


The current model is placing a great amount of worth on the ability of PrOPS and xFIP to accurately predict future performance. In addition, the new coefficients are assumed to remain constant. Finally, expecting the current rotation to remain as the final rotation is, again, quite a weak assumption. Anyway, things look a lot brighter for the O's than it did a few months back.

10 April 2008

24-Sided Die Determines Season


Batting performance is often simplified by people new to sabermetrics as OPS. Others take that a step further and define things by OBP and SLG. These numbers are used to describe the worth of a batter generically. They fail to recognize that batting position also plays a part in run production. Certain skills are often put to better use in certain positions. A simple analysis of this was done by Cyril Morong. He used raw data from 1988-2002 and determined the value of OBP and SLG by position in the batting order. It should be noted though that OBP and SLG are also rather generic, but it is the best data I have. Of course, hit-derived OBP is worth more than walk derived OBP except in extreme chances. Anyway, from these numbers we can determine how well the Orioles are likely to do based on their lineup and projections from ZiPS.

Assumptions
1. The starting nine players will play every inning of the entire season.
2. The starting pitchers will remain the starting pitchers over the course of the season and average 6 innings per start, equally.
3. The bullpen will be league average.
These are rather prominent assumptions, but I frankly do not care to go deeper into it.

We will compare two lineups. The most common one employed by Trembley along with the ideal lineup based on Morong's formula. Each lineup will be compared to the runs expected in the pitching performance. The run tallies will be converted into wins and losses by the pythagorean theorem.

Results

Pitching
ZiPS is not high on the Orioles pitching and assigns ERAs to them as such: Loewen 4.55, Cabrera 4.85, Guthrie 4.84, Traschel 5.20, and Burress 6.12. With each averaging 194.1 IP at an ERA of 5.11 and a bullpen with a 4.35 ERA, the team will give up 4.86 runs per game. That comes to 787 runs. I am ignoring unearned runs, which makes this an optimistic projection.

Trembley's Lineup

Trembley's typical lineup so far has been Roberts, Mora, Markakis, Millar, Huff, Scott, R Hernandez, Jones, and L Hernandez. This lineup should be weak with regard to underutilizing the top of the order by have Luis Hernandez bat last as well as having Millar and his lack of power in the 4 hole. This lineup is predicted to average 4.86 runs per game and a total of 788 runs scored. That results in a record of 81-81.

Ideal Lineup
The ideal lineup as defined by Morong's formula would be Roberts, Markakis, Huff, Jones, Scott, R Hernandez, Mora, L Hernandez, and Millar. Millar bats last to take advantage of his OBP as well as lessening the damage from his projected lack of power. This lineup brings in a predicted 5.07 runs per game and a total of 821 runs. This results in a 84-78 record.

Discussion
I'm not sure this exercise accomplished much, but it does kind of show what can be expected if many things broke our way offensively. It is likely that any backups to our starters would result in a significant decrease in offensive production (oh, wait, Luis, I didn't mean your projected offense). So, in a way . . . this projection should cover the event that several players outperform their expected production level. In a conservative sense, I would probably decrease offensive production by about 10% to account for days our main offensive players take off. Doing that would result in a Trembley lineup record of 71-91. This looks pretty accurate based on what most projection systems use with playing time considered.

I think the best case scenario would be if the offense improves 10% of what should be expected (787 runs) and the pitching improves 10% of what is expected (720 runs). In this scenario we would have an 88-74 record. I think expecting anything more than that would be tragically optimistic. I think to expect 88 wins is tragically optimistic. Probably expecting 81 wins would be inadvisable. Anyway, that is that. On the flip side, 10% decreased performance on each side would result in a 58-104 record. Expecting anything close to that would also be inadvisable.

Further Reading
Frost King Baseball did a quick study back in mid-March, apparently. It is a slightly more optimistic appraisal of the Orioles offense. The site uses a standard base runs method to determine runs scored.

06 April 2008

Today's Links

Today's Links:

Aaron Crow Biomechanics Report



Kiley McDaniel breaks down potential Orioles draft pick Aaron Crows over at SaberScouting. Crow has great mechanics, but there is a kink in his delivery that decreases his tempo. As he brings his arm fully cocked behind him, he has this weird hitch as he comes back across that slows down his movement and increases stress on his elbow. It probably means nothing, but, yeah, that looks like the only issue with his mechanics. Very sound, but he could improve his speed to the plate by correcting that hitch . . . if possible without affecting his performance. Weight transfer and front-side mechanics are near perfect.
Discussion at the Baltimore Sun

Chorye Spoone Biomechanics Report



Alex Eisenberg over at the Hardball Times breaks down Orioles minor leaguer Chorye Spoone:
Spoone is a workhorse. He'll walk his share of batters, but if he can maintain a solid K rate to go along with the many ground balls he is going to give up, then there is a possibility that he can reach his upside as a No. 2 starter. His mentality and work ethic give him an even better chance of reaching that upside.
Discussion at the Baltimore Sun

Composite Model Prediction for 2008 Season
In 6000 seasons, the O's never won the East. TB have a 1 in 20 chance of winning it. We do not have a 1 in 6000 chance. Yikes. No other team failed to win their division once in 6000 seasons.