One Less Stat to Not Understand

Today, one of the premier sabermetric websites, Baseball Prospectus, ditched one of its signature stats, saying it wasn't doing the job. Luddites will, of course, crow about this, but they should be eating crow instead.

What BP is doing is exactly what the hidebound can't abide: moving forward to improve the state of the art. The BP crew spent a year-and-a-half tweaking SIERA, the ERA predicting tool, until they finally decided that it was not a significant improvement over other tools already in use.

SIERA attempted to measure how much of a pitcher's ERA was his pitching and how much were non- repeatable factors outside his control, strip out the noise and then project what the pitcher would do going forward without luck's finger on the scale. There are already simpler statistics that strip out fielding and fluctuations in BABIP from a pitcher's work. And there are already a plethora of sabermetric crystal balls that perform marginally better than an educated person's estimate. So BP is leaving in the starter and consigning SIERA to the eternal bullpen.

Learning is not linear. We gain knowledge in fits and starts, by trial and -- yes -- error. Another seamhead mixing up a new statistical brew will sprinkle SIERA concepts into his cauldron and we'll all benefit collectively.

b

Past Performance May Not Be Indicative of Future Returns

Imagine if you could tease out from a player's performance the skill-wheat from the luck-chaff and make a more nuanced prediction of his future results. The imagination is not so very boggled here; we have many tools in the sabermetric toolkit that help us do this. On the hitting side we have, first and foremost, BABIP, now so widespread in its use that it has its own Wikipedia entry.

Pitching performance is dependent on a larger array of variables, and so requires a more robust distillation. Seamheads around the world have been nose-to-the-grindstone on that, to the point that many of the rough edges (on pitching stats, not on noses) have been smoothed. There exist several calculations that are independent of defense, BABIP and other noise.

Last year, Baseball Prospectus put its SABR light ray to use on the obverse proposition: rather than strip out the obfuscations, why not focus on the core elements of a pitcher's work. Pitchers can control their their walks and strikeouts, their home runs allowed, and the ratio of ground balls to fly balls allowed, so why not simply measure those things to understand their core skill sets? After developing the statistic, called SIERA (skill-interactive ERA), they tested its predictive ability and found that it was indeed superior to other measuring tools.

In other words, if you see a pitcher who went 20-8, 2.97 last year, you probably assume he pitched spectacularly. In fact, what you're seeing is a pitcher with spectacular results. The correlation is imperfect. If that hurler benefited from great defense and run-scoring support, fabulous relief pitching, an unusually low BABIP, unsustainable success with runners in scoring position and a good home pitching park, you could conclude that fortune has smiled upon him and may be looking elsewhere next season -- what we commonly call regressing to the mean. Conversely, if he served up a lot of strikeouts, limited his walks and HBPs, and kept the ball in the park and on the ground, he's got the requisite core skills to repeat, rinse and shine.

As soon as SIERA was introduced last year, I thought how interesting it would be to examine the 2010 pitchers whose SIERAs varied dramatically from their actual ERAs and then observe their 2011 performances. BP has provided just that list, so I present to you the hurlers whose performance will be worth noting this year. (Of course, these predictive tools have enormous Fantasy Baseball value, and in fact, BP has become more of a Fantasy Baseball site.)

Just one note of caution: Judging a player on a single season can be like judging a city's climate by one day. The weather in Fargo might be nicer than in Miami on a given day, but not over the long haul. So seamheads ordinarily use a rolling three-year average when measuring player performance and BP is doing that with the gentlemen listed below.

The four unlucky moundsmen capable of dramatic rebounds in 2011 are Josh Beckett, Brandon Morrow, Aaron Harang and Dan Haren. Beckett, slowed to 128 innings by a balky back, went a miserable 6-6, 5.78 for the Red Sox. But SIERA saw his performance more generously and believes he's more like a 3.84 ERA starter, which is among the league's elite. If he's healthy, expect his ERA to be as well.

Morrow, a rookie for Toronto last year, shoves strikes down batters' throats for a sterling 3.15 SIERA. An out-of-kilter .344 BABIP pushed his ERA to 4.45, but the SIERA suggests he could be a star in 2011. Aaron Harang's SIERA of 4.44 was 88 points better than his actual ERA in Cincinnati in 2010. Now toiling in a pitching-compliant Petco Park in San Diego, his actual ERA could plummet. The same for Haren, who spent half of last year in Arizona's hitter-haven park.

The four sons of serendipity in 2010 who might be best avoided in your fantasy draft are Brian Duensing, RA Dickey, Livan Hernandez and John Garland. Hernandez is a walking cautionary tale. Knuckleballers like Dickey are often immune to the Laws of Sabermetrics, so he might not belong on this list. Garland is a flyball pitcher, but at Petco that wasn't a demerit. He's employed by the Dodgers this season and he'll lose baseballs if he keeps his pitches up. Duensing, who pitched half in relief and half in the rotation for the Twins, had a 2.62 ERA with 4.22 skills. His inability to miss bats may come back to haunt him.

Over the years, SIERA, like all baseball metrics, will be whittled and honed as more information is revealed and more smart people contribute to its improvement. It'll be interesting to see how well it identifies the outliers in its first year.

b

Deconstructing His Ubaldoness

What in the name of Bob Gibson, 1968, is Ubaldo Jimenez up to? Two months into the season, he has already amassed a 13-1 record and a 1.15 ERA. In fact, it's even better than that, because his RA (which neglects the largely irrelevant distinction of unearned runs) is the same other-worldly 1.15.

Is UJ really this good? Can he maintain this pace?

It's worth noting that Jimenez is no flash in the pan. He has been considered a prime cut since his signing out of the Dominican at age 16, and has posted a solid first four seasons of work, sporting an ERA around 3.70 and nearly eight strikeouts per nine IP. He's big, coachable and fires mid-90s darts. Scouts have always felt that his control and health were the only things standing between him and stardom.

That said, 13-1, 1.15 with Coors as home base is pretty special. Below-average major league starters sometimes shut down their opponents for two or three turns, but rarely this well, this long. Usually, exploding out of the starting blocks like this takes a combination of skill, luck and good teammates. And Jimenez is no exception.

Ubaldo is responsible for a goodly portion of his results. First, he's averaged more than seven innings a start. That's both a cause of good outcomes and a result. As the best pitcher whom opponents are going to see in a given game, Jimenez helps himself by remaining effective as long as possible into the start and minimizing the use of relievers.

Second, he's fanned 88 batters in 101 innings, which is about his career rate, but he's shaved a walk per nine off his career average. He's surrendered just three taters all year, and six hits per nine, which are both superb, and about half his former rate. But as we shall see, we now have the tools to see that both those figures are a tad misleading.

MLB now records the location, velocity and horizontal and vertical break on every single pitch, and their result. With that information, researchers have been able to tease out correlations that suggest when results are whacky. For example, we know that pitchers have little control over the batting average on balls in play (BABIP) against them. So when we see batters scuffling at .235 on balls in play against Ubaldo, we know that he's got a bill coming due. (Average BABIP in the NL is .298. Some pitchers can sustain a BABIP down around .280, but not much lower.)

We can break BABIP down much more granularly to see how exactly Jimenez has been surfing a wave of serendipity. Although Jimenez serves up line drives about as often as everyone else, the batting average against him on those balls is .600, compared to a league average of .722. So before congratulating him on keeping guys off the bases, he should be offering obeisance to the Patron Saint of Right At Someone. The same for ground balls, which reached the outfield at an 11% rate, a mere shadow of the league rate of 17%. That could be the result of a Rockie infield slick with the leather, which appears to be a fact, but not sufficient to reduce the hit rate on worm-killers by 64%.

It breaks down further. Those three meager home runs? They smack of a visit to Flukeytown. Batters have poked just 5.8% of their fly balls off Jimenez into the stands, about half the normal rate. This is possible if those sky-balls present as pop-ups, but Ubaldo has no particular pop-up proclivities.

There's more. Hitting with runners in scoring position is often cited as a key to run scoring. It's largely a matter of luck that tends to regress to mean for a player, team or pitcher over time. Ubaldo is surrendering just a .186 BABIP with runners on base. It's certainly possible that he bears down more, digs deeper and makes his best pitches when the enemy threatens, but that would show up in strikeouts and walks, not BABIP. Fortune tends not to smile so broadly as .186 BABIPs for whole seasons.

Finally, you may have noticed that Jimenez has 14 decisions in 14 starts. That's unusual for a pitcher who's left 25 innings for his relief corps to handle. But handle they have, watching carefully over those 1-0 leads he's left to them. The bullpen trio of Joe Beimel, Matt Belisle and Manny Corpas are all in the top 20 of reliever effectiveness.

Given all that, how great has Ubaldo Jimenez actually pitched? Baseball Prospectus has created a new yardstick that strips out luck, ballpark and defensive efficiency and measures what a pitcher's ERA would be, all things being equal. In other words, given a pitcher who walks and strikes out at his rate, and who induces grounders, flies and line drives at his rate, what should Ubaldo Jimenez's ERA be? (BP calls this SIERA, for "Skill-Interactive ERA.")  Their analysis suggests that Jimenez has been the 16th best pitcher in MLB with a SIERA of 3.43.

Before you scoff at the difference, remember that a 3.43 ERA is pretty sweet. Based on BP's analysis, 20 more starts at a 3.43 ERA would leave Jimenez with a 2.49 ERA for the year and somewhere between 20 and 22 wins. On the other hand, it's hardly historical, or even the best in the league. It suggests, as does all the information above, that Jimenez is a top 20 hurler on a top five run of luck.

b