Why the Projection Systems Can't Get Bryce Harper Right

"History will little note, nor long remember, what we say here." --President Abraham Lincoln at Gettysburg battlefield, considered the worst prediction in human history.


The futility of predicting the future in baseball has been amply documented and only weakly lamented. Bless their hearts, the various Sabermetric websites continue to offer the old college try, annually tweaking their systems to finer and finer levels of mediocrity. Fangraphs, Bill James, Baseball Prospectus, Tom Tango and The Hardball Times all provide projections whose mathematical formulas do a 5% better job than you would making educated guesses.

As I've mentioned before, understanding a few simple rules will give you all the tools you need to pull even with, or even out-guess, the projections. The reason is that these systems are prisoners of their calculations, whereas you know something or two about the players.

Take Bryce Harper, who flummoxes PECOTA, ZIPS, Steamer, Marcel, Bill James and all the rest. Harper was a young phenom who flashed wads of unrefined brilliance in his first three seasons but also got himself injured with regularity because of his over-exuberance. Then last year, having learned how to stay on the field and harness his innate power, he blasted 42 homers and got aboard safely at a 46% clip, en route to the MVP.

What the projections systems see is a good player who scuffled along for three seasons batting .270 with 20 home runs before out-performing himself in 2015. Consequently, they are regressing him to the mean. Here is PECOTA's projection of Harper:

2015 (Actual)	22	.386	11.2
2016 (Projected)	23	.313	5.1
2017 (Projected)	24	.322	5.5

(That's age, True Average and WAR listed in the three columns above.)

That's generally the way to treat players, but this is Bryce Freakin' Harper. When it crunches the numbers and spits out its projection, PECOTA doesn't know Bryce Harper from Valerie Harper. It's never seen this prodigy play.  It thinks he's just a guy whose pants were on fire last season.

You know better. You know that's the Harper we've been waiting for, and that putting it all together at 22 makes him a superstar not in spite of three previous seasons of .270 and 20 home runs but because of what we saw during them. 

I'll bet my Pope Francis Fan Club membership that Harper posts a TAv north of .350 this season and, if he stays upright, hands at least another eight wins to the Nationals. It may not work out that way, but that's what PECOTA would say is a reasonable 50% percentile projection if it had the ability to discern the man behind the statistics.

It doesn't, as none of the projections systems do, which is one of the reasons they aren't of much use.

MLB Demonstrates the Uselessness of Projection Systems

Major League Baseball is touting today the Steamer baseball projections for 2016. Steamer is one of several Sabermetric projection systems -- including Fangraphs' ZIPS, Baseball Prospectus's PECOTA and the Hardball Times' Oliver -- that offer a crystal ball of sorts on future player and team performance.

These systems generally work the same way, projecting the accomplishments of players by comparing their careers to similar players in history, averaging the performances of those similar players in their next season and adjusting that to the player in question.

The MLB.com story is not a particularly interesting read. It projects that Miguel Cabrera will win another batting title, Clayton Kershaw will again top the pitching charts and Giancarlo Stanton will once more outslug the field. 

Shocking.

This is what you get with projection systems, which have no ability to predict the unpredictable. The 2015 accomplishments of Jake Arrieta, Dee Gordon, AJ Pollock, Dallas Keuchel, the Kansas City Royals and New York Mets were no less a mystery to these projection systems than to you.

They also have no ability to predict outlying performances. They will never project any player will slam 58 home runs, bat .360 or accrue 21 pitching wins unless he has done that regularly in the past. 

Moreover, Steamer does not attempt to project playing time, so even though Stanton has missed 40 games a season in his career, the projection assumes he plays 150 games. (Some of these systems do take playing time into account, which means they are wildly wrong slightly less often.)

These systems gave us the Angels, Dodgers and Nationals as baseball's best teams in 2015. They got the AL East and West almost exactly wrong. They offer some mild insights that can be almost completely encapsulated in 13 simple rules.

So don't bother reading the article. (Read this one about Statcast instead.) And let's just enjoy the raging unpredictability of the game that served us so well last year.

An Un-Evan Projection for Longo

What do the projection systems have against Evan Longoria?

The All-Star third-sacker has manufactured a .298 lifetime True Average (TAv) in a seven-year career, dropped below .300 only by last season's struggles following off-season surgery. That not withstanding, Longo has been automatic 30 homers and a .351 lifetime on base percentage.

The projections systems say that at 29 all of Longoria's best days are behind him. Three prominent systems -- PECOTA, ZIPS and Steamer -- say he is more like a .255-.260 hitter, with OBP in the .330s and low-20s home runs. A seven win player his first six seasons, Longoria is grading out as just a three-win player in 2015, almost all of it because of defense.

This suggests that players with Longoria's profile -- medium-speed power hitters with fancy leather -- don't age well starting in their late 20s. Or it suggests that a new $100 million contract will emotionally short-circuit the ordinarily Evan-keeled Longoria and dampen his skills in 2015.

Neither of these premises rings particularly true. Indeed, the most likely event, barring further injury, is a bounceback from last year's scuffles that sustains itself for a couple of seasons before real decline sets in. The former Rookie of the Year has too much talent, demonstrated too consistently, to suddenly fall off the cliff in his prime.

Wherefore Art Thou Justin Verlander?

From 2009 to 2012, Justin Verlander was the best pitcher in baseball. In those four campaigns, the 6'5" Virginian averaged 34 starts, a 20-8 record, 2.95 ERA, more strikeouts than innings and a 4-1 K-BB ratio. He finished first, second, third and 11th in the Cy Young voting those four years and was worth 26 wins to the Tigers.

He did a little less of everything in 2013, and then last year, at age 31, he cratered. His 4.54 ERA was well-north of average (that is, worse) and he fanned just seven batters per nine. He was worth about a win to Detroit, about the same as Zach Duke, who hurled all of 59 middle innings for Milwaukee.

Will the real Justin Verlander please stand up? The Tigers, recently shorn of their new-now-old ace Max Scherzer, need a clue about which Verlander they will get.

There's tempered good news, according to Baseball Prospectus's PECOTA projection. It sees Verlander bouncing back to 2013, with a 3.30 ERA, nearly a strikeout an inning and 3.5 wins against replacement. That might not be ace material, but it'll play.

But not so fast, my friend. The Steamer projection system forecasts continued scuffling for the Detroit righty. That system has him pitching fewer innings at a 4.05 ERA, continued diminished strikeouts and just two wins against replacement.

See, they're as bad at this as you are. We know one thing for sure about these two projections: at least one of them is wrong. One system says he'll return to a diminished version of his old form; the other thinks there's a new norm. What you're seeing is one system valuing recency more than the other. And systems are just a series of assumptions that get tested and re-tested thousands of times. 

Let's examine some other intriguing projections:

Corey Kluber
Corey Kluber slashed nearly three runs from his ERA over two years en route to a Cy Young in 2014. PECOTA is unimpressed, mostly because of a mediocre six-year Minor League journey that preceded his rise to the Bigs. Its forecast: 12-12, 4.08 and 0.7 WAR.

Steamer is more sanguine, though it's regressing only to the mean: 13-9, 3.21, 4.0 WAR. (Worth noting: there isn't a seamhead in America who wouldn't take the over on the PECOTA line.)

Prince Fielder
Pitchers are puzzles inside riddles wrapped in shoulder tendonitis. Hitters are easier to project. So let's take a look at Prince Fielder, who blasted 38 taters a year from '07 to '12, while getting aboard at a 40% clip. In 2013 he slipped to 25 homers and a .362 on base before being shipped to the hitting zone in Texas. Alas, Fielder succumbed to a herniated disc after a 42-game power outage last season.

PECOTA both gives Fielder a pass but also projects the whompin' days are over at age 31 and weight 275. It projects just 23 homers and three WAR. Steamer agrees: 24 homers and two-and-a-half WAR.

ARod
You're probably wondering about Alex Rodriguez. Projecting him is easy. That's because ARod, no matter what any projection system thinks about his talent, is certifiably 40 years old. Debilitations of the body and the batting line are a guarantee. PECOTA says .248, 10 home runs in 300 at bats. Steamer projects .235, 11 home runs in 400 at bats. He needs but six to catch Willie Mays. Yankee brass may have something to say about all those at bats.

Matt Harvey
The Dark Knight of Gotham breezed through the minors and wowed the baseball world in his 36 lifetime starts (2.30 ERA, 4.5-1 K-BB ratio) before Tommy Johnning his elbow in '13. Both systems project Harvey will continue to light it up but over less than a full season.
PECOTA: 10-8, 2.91, 151 innings, 153 K
Steamer: 11-8, 3.13, 163 innings, 175K

Matt Kemp
Kemp's career has been an elevator ride. His past four seasons have gone like this: 

• Up! MVP-type season.
• Down! Excellent but injury-shortened.
• Down! Lost season on offense and defense.
• Up! Very good at the plate. Fielding not so great.

Now he moves from the offensive hole in L.A. to the offensive black hole in San Diego, plus he has to face Clayton Kershaw a couple of times a year. The statistical crystal balls see the power and average returning, but not the glove. 
PECOTA: .270, 25 HR, 14 steals, 3.1 WAR
Steamer: .266, 20 HR, 8 steals, 1.9 WAR (in only 128 games)

So there are six of many projections. We'll look at the team projections over the next few days and return occasionally to individual players.

Statheads Have To Lighten Up About Their Projections

When a kid gets a shiny new toy he always wanted for Christmas, it's not surprising that he plays with it endlessly. And when it ceases to work, it's not surprising that he puts it away.

The seamheads turned out shiny new projection toys in the 2000s variously called PECOTA, Steamer, ZiPS, Marcel, FANS and Oliver, depending on whose proprietary system you prefer. (For example, Baseball Prospectus created PECOTA, Fangraphs introduced ZiPS and Tom Tango developed Marcel.) They all compare ballplayers of today to their counterparts in history to determine what awaits the current crop in the coming year. All that is then compiled into team projections.

These toys have some value for a couple of reasons: they are objective and they have built into them some simple rules that humans fail to recognize or choose to ignore when analyzing player performance. Beyond that, though, they offer little insight. Most significantly, they don't know anything about the people who play the game. They are blind to who learned a new pitch, is struggling through a wrenching divorce, picked up a hitch in his swing or is finally healthy.

As a result, they can never tell us who will suddenly break out, Jose Bautista style, or who will Dontrelle Willis all over himself. None of them could pinpoint the 2014 demise of the Red Sox or the Orioles' pre-eminence. And yet the sabermetric world talks about these tools as if they're atom splitters.

So, for example, Steamer predicts the top 10 players by WAR next year are Mike Trout, Andrew McCutcheon, Giancarlo Stanton, Troy Tulowitzki, Miguel Cabrera, Josh Donaldson, Evan Longoria, Adrian Beltre, Jose Bautista and Paul Goldschmidt. (Cano, Machado and Puig are right behind.) Well, duh. right? (Clayton Kershaw and Felix Hernandez are its top pitchers. Who knew?)

It says Joey Votto and Bryce Harper are stars with injury risks. You don't say. It projects Chris Davis bouncing about halfway back to 2013 with a .242 batting average and 30 home runs. About what you'd predict. It sees Justin Verlander returning to about league average pitching. And it's not really buying Rookie of the Year Jacob deGrom. Well, who outside of the outer boroughs is?

So the toys have some entertainment value, but the saber community plays with them as if they're iPads.  They quote chapter and verse on players, particularly when analyzing trades and signings. The truth is, exercise a few simple guiding principles, like Warren Buffet investor tips, and you too can lord over the average baseball fan.

And be right 60% of the time.

In subsequent posts, we'll discuss those principles, examine some interesting player projections and the swim among the team projections. (Spoiler alert: the Phillies will suck.)