Luck Is Not the Residue of Design In Baseball or In Life

Branch Rickey famously said that "luck is the residue of design." Branch Rickey was a baseball man and luck is part of baseball so it's worth examination in a baseball blog.

You might have noticed that only lucky people make this statement. They want to take credit for their good fortune. Partisan politics aside, it's not possible to view the life of George W. Bush -- who screwed up in college, over-imbibed in alcohol and cocaine in his 30s, bankrupted two companies, got himself removed from a corporate board in his 40s and traded away young Sammy Sosa as president of the Texas Rangers -- without recognizing the immense power of luck. Had he been George Tree, instead of the rich scion of a politically connected family, it seems pretty obvious that he wouldn't have been accepted to Yale, installed as a corporate CEO, served on well-paying boards, been installed atop an MLB team or won election to the local zoning board, much less the presidency.

Indeed, luck and design are utterly independent. Good design can mitigate bad fortune and exploit good, but overwhelmingly bad luck will ruin all but the most adept. Even the best horse-and-buggy makers were put out of business by Henry Ford. 

Which brings us to baseball.

Tigers' third baseman Nick Castellanos, a lifetime .258 hitter, leads the American League in batting average so far this season at a .378 clip. No doubt he is hot: he's already pounded half as many home runs (7) as last year in a fifth as many at bats. He's also batting .449 on balls in play, even though the distribution of the balls he's hit is only marginally better than in the previous two seasons. 

In other words, Catellanos's grounders are dribbling through a hole. His lazy flies are dropping between charging fielders. His line drives are not getting snared. He has been lucky.

There's a fella named Pujols whose batting average (.183) is less than half of Castellanos's. He's not ripping as many line drives as Castellanos but he's killing many more worms, the kind of batted balls that metamorphose into singles at a high rate. Combine his relative turtle-speed with a snoozing patron saint of serendipity and Albert's hitting just .160 on balls in play. That's horrible luck.

Perhaps Branch Rickey would blame design, but Pujols has fanned half as many times as Castellanos, while walking twice as often. That suggests that Castellanos is just plain lucky. We should think twice, for example, before voting him to the All-Star team.

And while on average these trends will regress to middle ground, sometimes they fester all season. In 2002, Cubs infielder Jose Hernandez batted .404 on balls in play for no apparent reason en route to a .288 batting average and an All Star selection. He never again came within 50 points of OPS over a whole season and retired with a .252 batting average.

We can overcome bad luck and allow good luck to slip by unnoticed. But refusing to acknowledge its mighty influence is just silly.

A Twin Killing

You might recall, back months and months ago when the NBA playoffs were in their second round; or third, I can't keep track; Memphis led Golden State two games to one and supposedly knowledgeable sports analysts began their Chicken Little dance around the Warriors.

Steph Curry & crew proceeded to win the next three games, take the series in six and earn their way to the league championship for the first time in 40 years.

It's easy to do the same thing with the long baseball season. Who could blame a St. Paul resident for thinking that the first 60 games of the season are representative of something for their beloved Twins.

Alas, there is nothing in the record, besides 33 wins and 25 losses, to suggest the Twins are even average. A peak behind the curtain, the kind of thing we couldn't do in the halcyon days before advanced metrics, suggests Minnesota has taken a long ride on a carpet of smoke, mirrors and just plain luck.

When you're among the best teams in the league, you tend to be particularly adept at some element of the game, or generally adept at all elements. The Twins are neither. Here is their offense:

• middle of the pack in batting average
• fifth worst in OBP
• third worst in slugging
• with the fourth fewest home runs and
• fourth worst in steals and caught stealing

The Twins lead all but three AL offenses with 12 triples, but I doubt that's the reason that they've scored the fifth most runs.

Huh?

Yes, a pretty substandard offense has scored more runs than 10 of the other 14 teams. How? Good timing -- what the geezers in the retirement home call "clutch hitting," bless their hearts. But if clutch hitting is hardly a thing among individual players, it's certainly a passing wind at the team level.  The sequencing of Twins hits have produced runs so far. Expect karma to start making amends in Minneapolis right soon.

The formula is similar on the defensive end. Pitchers have allowed the fewest walks in the league, but registered the fewest strikeouts and allowed the most hits. They keep the ball in the park reasonably well (sixth out of 15), and the result is the eighth best ERA, which is dead center for the league, about what you'd expect.

So the pitching isn't carrying them.

On defense, Baseball Reference rates the Twins just slightly below average. Baseball Prospectus isn't that sanguine. Either way, the gloves haven't transformed weak pitching into shutouts.

Is there some magic formula at Target Field? The team has prevailed in one-run games, but not enough to dress a talent-challenged team in Wild Card clothes. Closer Glen Perkins has turned out the lights consistently (five runs in 27 innings and a league-leading 21 saves) but the rest of the pen is a mish-mash that fails to scream "secret sauce."

In short, the Twins are nothing special, despite the special status in the standings. Those with deadlines and column-inches to fill will credit manager Paul Molitor for teaching his charges "how to win" and settling them down after a poor start. But real people know that managers, even great managers, even Hall of Fame former players, even really smart people with spectacular emotional IQs, can't turn the 2015 Twins into a pennant contender.

Reporters can't interview luck, can't describe it, and won't be received well in the clubhouse if they refuse to credit their sources with the good results those sources have produced. But luck is the reason Minnesota began the season 30-19 and luck is fickle.

The fall won't be pretty because, like fire, luck is a great servant but a terrible master.

Why Seamheads Keep Under-Rating the O's

In the last three seasons, the Baltimore Orioles have punished their AL East brethren, tallying 93, 85 and 96 wins, and snagging two division titles.

And so this year, once again, the stat-based projection systems are showing the franchise less respect than Aretha Franklin. PECOTA pegs the Birds at 78 wins and a basement finish for the fourth straight season. Not a finished basement, mind you.

The Birds have unintentionally uncovered one way in which projections systems and the seamheads who worship them are for the birds themselves. (But not for the Birds.) Because of pretzel logic baked into the projection system by stat guys who call everything they can't quantify "luck," PECOTA holds a deep and abiding hatred for everything Baltimore, including crabs, Terrapins, Edgar Alan Poe and being called "hun."

In this article for Baseball Prospectus, Zachary Levine explains what the Orioles did to PECOTA (pulled its hair and made it cry in fifth grade) to consistently earn a subterranean ranking. How does PECOTA suggest the Orioles will shed 18 wins from last season? Here's the critical segment of the explanation:

The first five wins are easy. That’s the luck that the Orioles had last year that you would expect to be close to independent year-to-year. A 96-66 team, they had a Pythagorean record based on runs scored and allowed of 94-68 and a third-order record based on the underlying stats that go into run scoring and prevention of 91-71. 

Translation: the O's presented a unique profile in 2014, in which the close matches tilted their way and the blowouts blew in the other direction. Generally, that's more likely luck than skill. Bill James discovered in a previous century that teams are much more like their run differential than they are like their won-loss records; the run differential is a better predictor of future record than past record.

The discrepancy between the won-loss record suggested by run differential (technically, the difference between the square of runs scored and the square of runs allowed, hence the notion of a Pythagorian record) and the actual won-loss record is commonly referred to by sabergeeks as "luck."

But sometimes what appears to be luck is skill lurking in the corner of the dugout. The Os under Showalter have vastly out-performed this measure, which suggests that they know something Bill James doesn't . (Is that possible?) In 2012, Baltimore posted an astonishing 29-9 record in one-run games and 16-2 record in extra-inning affairs, and they repeated the feat, albeit less dramatically, in 2014.

This could be a run of luck akin to flipping heads 17 of 19 times. Or it could be that the team knows something that gives them an edge in close contests. A sharp manager, a good bench and a deep but unheralded pitching staff, all of which Baltimore possesses, could convey that needed margin in close games without tilting the balance in less evenly contested skirmishes.

This is a bit like the "pitching to the score" argument people make for Jack Morris and his pedestrian 3.94 lifetime ERA, so we have to be careful. But PECOTA and its brethren have been wrong for three straight years. Maybe it's time to acknowledge that just because we don't know what it is doesn't necessarily make it luck.

Bet the over.

BABIP: From the White House to the Outhouse

Batting Average on Balls In Play (BABIP) burst onto the baseball landscape like a meteor, illuminating life on the diamond in its wake. By measuring the part of the game largely outside the control of the pitcher (and somewhat outside the control of the batter) it appeared to separate skill from luck.

If the Royals' Billy Bulter hits .341 on balls in play in 2012 and Shane Victorino hits .278, might that not suggest that Butler was crazy lucky and Victorino not so much? If we were making predictions about the following year, might we not temper Butler's prediction and enhance Victorino's on the notion that luck will average out?

The answer, it turns out, is: well, maybe. After cuddling up beside BABIP for several years, seamheads started to notice that it had some idiosyncracies. BABIP just likes some guys better than others. Speedsters are particular BABIP sweethearts because they can get on base without hitting the ball hard. Ichiro rocked a .357 BABIP through 2010, but as his speed has waned, he's suffered three sub-.300 BABIPs since.

Worm killers romp with BABIP because ground balls tend to find holes more often than fly balls. That's offset by the fly balls that do make safe landing, which tend to result in extra base hits. Pop-ups are BABIP death and line drives are BABIP medicine, for obvious reason. And batters can control their BABIP to some degree. The aforementioned Ichiro didn't just hit the ball; he often placed it, much to BABIP's delight.

BABIP can start the narrative, but not complete it. Certainly Butler doesn't have foot speed on Victorino, but he launched many more line drives and many fewer pop-ups than Victorino in 2012. That suggested there might be some staying power in those numbers; i.e., there might be some skill involved.

In 2013, Butler did, in fact, maintain an above-average BABIP, losing just 15 points from his 2012 rate. But Victorino, rejuvenated in Boston, "regressed" beyond the mean and up 43 points. Some of that was the result of doubling his line drive rate and much of it was pure serendipity, from a lot of bad in 2012 to a little good in 2013.

It's even more pronounced for pitchers, whose results are affected by the defense behind them, defense that remains largely intact over the course of a season. Rookie of the Year Jose Fernandez allowed a .240 batting average on balls in play, despite fairly pedestrian rates of line drives and pop-ups induced. Fernandez looks highly promising, but BABIP strongly suggests a return to earth in 2014. 

Conversely, all-world starter Justin Verlander played second fiddle to Max Scherzer in Detroit last year, in part because of a high BABIP-allowed (.316). Verlander allowed a lot of line drives, but probably not enough to explain all of BABIP. Likely the Tiger statues at third, short and first contributed to balls getting through. A bounceback may be in order, particularly with Miguel Cabrera moving to first, Prince Fielder and Jhonny Peralta leaving town and Ian Kinsler joining the infield.(Why didn't this matter for Scherzer? A. Luck. B. An 11% higher strikeout rate. Dr. Strangeglove can't flub your strikeout.)

So BABIP is useful, particularly at the extremes, but it isn't quite the revelation originally envisioned. Pure luck is hiding in every facet of the game,but in the nooks and crannies, not out in the open for BABIP to see in its entirety.

Lifting the Veil of Luck

It's appropriate on the weekend of Black Friday to discuss shopping frenzies. To some extent, every team is in one now as they kick the tires on free agents who might help them win in 2014 and beyond.

One of the most important advances in the world of baseball analytics is the ability of franchise brass to evaluate personnel realistically. When baseball GMs determine which free agents might help their teams in 2014, they will view 37-year-old Bruce Chen with a jaundiced eye.

Teams will know that Chen's 9-4, 3.27 effort for Kansas City in 2013 was a chimera. Nothing in his pitching profile suggested that he had a special season except the ultimate results. In 121 innings he fanned just 78. He relinquished just 107 hits, primarily because opponents batted only .255 on balls in play (BABIP) against him, 25 points lower than his career rate and 40 notches lower than league average. The chasm opens wider when examining only Chen's 15 starts; he made 19 relief appearances to begin the season before the Royals summoned him back into the rotation.

In addition, Chen  stranded 78% of runners who reached base against him, significantly above his lifetime and league average rates. Strand rates are rarely a matter of skill. Finally, an unusually low percentage of fly balls against him left the park, often a sign that a pitcher got lucky with wind, field configuration, and the like.  Taken together, the snapshot of Chen's results provides strong evidence that regression troops are arraying on the border and preparing to attack next season. Caveat emptor.

Likewise, Cubs brass won't be relying on Travis Wood to deliver another stellar (literally: he made the All-Star team) 3.29 ERA season in 2014. Wood also enjoyed a low BABIP (.248), high strand rate (77%) and low fly ball/HR ratio. Advanced statistical models suggest his actual performance looked more like a 4.50 ERA.

Instead, Cubs management might hope for bounce-back from Edwin Jackson. Despite an 8-18, 4.98 campaign in '13, Jackson was actually a strong performer. A .322 BABIP against and a frighteningly low (63%) strand rate conspired to denigrate his work. Fangraphs estimates Jackson was worth two wins above replacement, something an ERA of nearly-five would not suggest.

And don't look now, but 2014's Tiger pitching phenom could be Rick Porcello. The 25-year-old righty managed a 13-8, 4.32 line when everything was going against him; it'll be interesting to see what happens when he basks in the sunshine of serendipity. Like Jackson, Porcello suffered a high BABIP and low strand rate in '13, but big flies are what really did him in. One of every seven balls in the air against him cleared the fences, a miserable rate for which he might not be completely responsible. Fangraphs thinks Porcello was more like a 3.50 ERA hurler who was worth 3.5 wins last year.
Over the last four years, Porcello's strikeout rate has risen while his walk rate has held steady. If he can keep the ball in the park just a little better he has a chance to to some damage in 2014.

Remember, regression is a trend, not a guarantee. Jackson could stink up the joint again and Wood could again leave the bases full of runners. It's just not the way to bet.
 

Kill the Umpire!

Umpiring has been a central element of the 2013 World Series, but in a turnabout of the usual, it's because the umps have worked scrupulously to get it right.

In Game 1, the second base arbiter gakked up a call and was immediately over-ruled, with his consent, by his five brethren. A wrong was quickly righted and we could all move on with our lives.

With two outs in the ninth of a tied Game 3, the third base umpire immediately, forcefully and correctly called fielder interference and was supported by the plate ump. They made the call even thought the interference might have been unintentional and resulted in an awful conclusion to a scintillating game.

This is (yet) another example of how critical luck is to the outcome of any single game. Little accumulations of luck -- the hardly-noticed variety -- can literally turn a 5-3 loss into a 7-2 win. If luck can alter the outcome of one game so significantly, it's going to play an outsize role in a four-win series.

The 800-pound gorilla of luck favored the Cardinals in Game 3. Where the smaller serendipities fell, it's hard to know. (I thought a generous strike zone forced the Red Sox to swing at unhittable low and outside pitches they otherwise would have ignored. The implications of that, if correct, ripple across the entire game and swamp one game-ending play.) But the bottom line is that they're part of baseball and teams just have to try to deal with them. 

Fans and analysts should read the same memo.

Congratulations, You're Fired

Jim Tracy and Mike Scioscia were named Managers of the Year yesterday. Which means, of course, that they'll be fired. Nearly every manager is, eventually.

Sportswriters voting for manager of the year is like frequent fliers voting for pilot of the year. They can measure whether landings equals takeoffs, but beyond that, how can they possibly assess the work of the nominees when most of it takes place in private?

The primary job of a team manager is to be a leader, to keep his team productive and on-task over 162 games.(Or as Casey Stengel famously observed, to keep the two guys who hated him from the 23 who weren't sure.) While managers make some strategic decisions, most of them are rote. Over the course of a year, the difference between a good strategist and a bad one is probably two or three games. That means a league's best manager -- whatever "best" means -- could be hiding behind a 60-102 record.

Were Scioscia and Tracy better leaders of men this year than Bobby Cox, Joe Torre, Tony La Russa, Joe Girardi, Jim Leyland and Hugo Chavez? This award suggests that sportswriters know the answer. In fact, what they know is how managers' teams performed compared to expectations, and vote accordingly. But the correlation between managerial skill and performance versus expectation is pretty weak.

In truth, if the award were for great managing, the same guys would win every year. If Tony La Russa is a genius, he's more or less a genius every year and should be Manager of the Year as often as Albert Pujols, the best player, wins the MVP. (Actually, Pujols has 400 competitors, La Russa just 15, so the skipper should pretty much monopolize the trophy.)

There's no question that Jim Tracy made some key moves that transformed the team from rocks to rockets when he inherited the Rockies from Clint Hurdle this summer. He also had to have caught lightning in a bottle, for which he got undue credit and now sports some nifty hardware. Players say they'd run through walls for Scioscia, and he obviously handled the aftermath of Nick Adenhart's death deftly, but it doesn't hurt that Jared Weaver, John Lackey and Joe Saunders can hurl.

Three of the last six Managers of the Year have been fired, so apparently the smartest skippers turned to dolts in two years' time. Either that or sportswriters don't really know what they're voting for.
b

The Ascendancy of Luck: #1 In A Series

It's a new year, time to reconsider the old and start afresh. In that vein, today begins a wintertime series of discourses on common baseball misconceptions currently in the possession of most baseball fans and media.

You can probably guess some of the subject matter. RBIs are not king. Speed is not the top quality for a leadoff hitter. Most of what you think is "clutch" is just luck or timing. Analyzing playoff series game-by-game is a fool's errand. Players don't win championships; teams do. Managers hardly matter. Pitching wins are such a weak barometer of pitching quality that we should cease tracking them. Et cetera, et cetera (which is not pronounced eck-setera.)

Today, maybe the single most important concept: the ascendancy of luck.

Luck is ubiquitous, significant and often determinative. This is true not just in sports. Luck is the reason both George W. Bush and Barak Obama attained the presidency. Imagine how their scenarios might have changed had the weather in Florida or the overall economy been different on election day. Luck gave a marginal talent like Tom Brokaw the opportunity to reach the top of TV news while far better journalists and broadcasters toiled elsewhere for far less money. Fitness guru Jim Fixx lost his life to bad luck. Good luck has kept the rest of us alive.

Luck is a factor on every play of every baseball game. A gust of wind, a bad hop, a fortuitous carom, a close call, an aggressive fan, a freak injury, a quarter-of-an-inch when bat meets ball: any one of these can affect the outcome. If you want to find an over-performing team, find one that wins a pile of one-run games. The pundits will say they're "clutch," but they're generally lucky. The proof is, those teams have lesser records the following year. Conversely, teams that lose close contests in bunches over a season tend to improve their performance the next season.

That's not to say that skill, hard work, practice and dedication are irrelevant; they are so important that no matter how many times I bat against Johan Santana, I will never homer off him. (However, I'm quite likely to run home.) In the Major Leagues, where every player and every team is magnificently skilled and focused, the five percent of the game that luck inhabits can make all the difference in the world.

Branch Rickey said luck is the residue of design. This is nonsense perpetrated by lucky people. Rickey himself had the massive good fortune to be born white and rich in America. Certainly people who work hard and smart exploit the good breaks and overcome the bad ones better than those who don't. But no measure of ability or commitment could have catapulted Jackie Robinson's father to Branch Rickey's career.

In the course of a baseball season, serendipity visits every player in one form or another. One hitter's bloops drop in, while another's line drives get caught. These natural vicissitudes add up to what the seamheads call BABIP -- batting average on balls in play. (In other words, not counting strikeouts, walks, HBP and home runs.) Some of BABIP is skill -- how hard you hit the ball and how fast you run can affect it -- but BABIP fluctuates as much as 100 points after accounting for those things.

That 100 points is primarily luck. The same is true for pitchers. In 1999, after his amazing seven-year run, Greg Maddux surrendered 39 more hits than innings and his ERA spiked 50%. Asked what happened, Maddux shrugged that the balls being hit off him were just falling in and that it would even out. Sure enough, in 2000 Maddux threw 30 more innings and relinquished 33 fewer hits as his ERA dropped 57 points. One reason Maddux was so transcendent is that he intuitively understood the hidden skills of pitching.

On average, luck evens out. On average, the wind blows 10 mph. But sometimes it's a hurricane and sometimes the air is still. Most players live in the fat part of the bell curve, but a handful spend a whole season shacking up with Lady Luck while another handful endure her year-long enmity. Total it up and you can see how a player who bats .280 with 20 home runs one year is the same one who bats .250 with 14 or .310 with 26 the next, or a pitcher with a 4.00 ERA one year is the same as one with a 3.40 or 4.60 the next. In fact, these are rather conservative examples.

Now multiply that by team. Whole clubs have enjoyed a heaping teaspoon of nature's smiles that lead to victories in the close games and hot runs through the gauntlet to the World Series. The players, coaches, media and fans can be forgiven for seeking ex post facto reasons for their success -- "chemistry," commitment, team effort -- when a simpler but less morally uplifting explanation is obvious. As King Charlemagne is given to say in Pippin, "It's smarter to be lucky than it's lucky to be smart."

Soldiers of Serendipity

What do CC Sabathia's one-hitter and instant replay have in common? They're both controversial because people don't understand how incredibly important luck is in sports.

Sabathia's mates are miffed that the official scorer in last night's Brewers' game credited Andy Laroche with a hit when the pitcher muffed his swinging bunt. It turned out to be the Pirates' only hit of the game, denying the big righthander a no-hitter. Milwaukee management actually appealed the hit call to the MLB front office.

At the same time, MLB has instituted instant replay to help umpires determine whether balls hit out actually leave the ballpark in fair territory. Newfangled stadiums, with their outfield nooks and crannies, apparently hamper umpire calls.

It's hard to argue that umpires and official scorers shouldn't get calls right as often as possible. But am I the only one who notices that we agonize over calls in sports that are determined by the most granular margins when twists of fate affect every play.

There's the gust of wind that catches a long fly ball and drops it into an outfielder mitt. There's the double-play grounder that hits the rubber, bounces into right field and scores two runs. There's the 0-2 slider that fools the batter into flailing desperately -- and blooping a two-out, two-run double into right field. There's the bad-hop grounder, the swinging bunt, the Texas Leaguer, the broken bat, the rain, the slick field, the shadows, the gnats and a hundred other soldiers of serendipity that turn a 2-1 win into a 9-4 loss. That doesn't even account for the freak injuries and other cosmic anomalies that transform whole seasons.

It's easy to parse one play out of a game and blame it for ruining a no-hitter, but every no-hitter is a lucky five-hitter. Unless they strike out 27 batters, pitchers don't pitch no-hitters so much as put themselves in position to do so and reap the rewards when luck shines on them. Likewise, every game is a stew of skill, execution and chance, and if an ump happens to call a home run foul when it was actually two feet inside the fair pole, well, it's not like the hitter planned to hit it that way. He was just two feet lucky in the first place.