Wednesday, March 1, 2017

The Devil in the White City

The Devil in the White City: Murder, Magic, and Madness at the Fair That Changed America
Erik Larson (2003, Vintage)

     The World's Columbian Exposition of 1893, otherwise known as the Chicago World's Fair, was an undertaking of colossal scale, covering a square mile of lakeside with all manner of exotic attractions and novelties, and beautiful white buildings that rose from the mud in just a couple of years. Erik Larson's history of its conception and construction pulls together all kinds of things we now take for granted, from skyscrapers to Shredded Wheat. Larson also weaves in a contemporary true-crime story about a man whose architectural ambitions are considerably less exalted.

     In 1890, Congress awarded Chicago the right to celebrate the four-hundredth anniversary of Columbus's voyage, which left only three years for a tremendous feat of planning and building. Chicago architect Daniel Hudson Burnham was an inspired choice to lead the effort. Not only was he a talented architect in his own right, he had the powers of leadership and persuasion he would need to pull together such other architectural lights as Charles McKim, Augustus St. Gaudens, and Frederick Law Olmstead.

     The fair's construction faced "a legion of obstacles, any one of which could have–should have–killed it long before Opening Day." Maddening delays in choosing a site wiped out most of the first year; a worldwide financial panic and unexpected weather exacerbated the ordinary difficulties that come with any project, let alone one that proposes to entertain 700,000 people in a single day, as the fair did at its peak. 'Entertain' might be too strong a word for the fair itself: Burnham intended to awe visitors with the beauty and grandeur of the buildings, which were all painted white. Entertainment, as such, was more likely to be found at neighboring sideshows, such as Buffalo Bill's Wild West show, the belly dancers in the Egyptian village, and the world's largest Ferris wheel, at 264 feet high. (That it was also the world's first Ferris wheel makes that figure even more impressive.)

     The 'Devil' in Larson's title was a con man turned murderer named H. H. Holmes, though he took other names any time it suited his purposes. A mile or so from the site of the fair, he built a large, dark building with offices and apartments over street-level storefronts. With its soundproof rooms and a coffin-shaped kiln, the setup was just right for making young women disappear, and disposing of their bodies. He did this sometimes for profit, selling the skeletons as medical specimens, and sometimes for convenience: he possessed a dangerous charisma that led him to make promises to the young women in his thrall. When he reached a stage where his victim started to expect him to follow through, off they went. Larson is scrupulous to give us only what is known to be fact, but some of the gaps tell their own story.

    The Columbian Exposition lasted only six months, through the summer of 1893. The labor boom that had built it gave way to unemployment and homelessness in the following year, and arson took down many of the most magnificent structures. Like a magic kingdom, the White City was gone almost as quickly as it had come. It sounds like a fairy tale, in a way, but I trust Larson: it's all true.

Any Good Books
March 2017

Wednesday, February 1, 2017

The Undoing Project

Any Good Books
February 2017

The Undoing Project: A Friendship That Changed Our Minds
Michael Lewis (W.W. Norton, 2017)

    Amos Tversky and Daniel Kahneman would have had brilliant careers in psychology if they had never known each other. Amos worked in mathematical psychology, and helped write the principal reference on measurement. He was as lively and outgoing as his chosen subject was dry, and he could get to the heart of other subjects with shocking speed, able to appear the smartest physicist in a room full of them. Danny was the most interesting teacher at Hebrew University, with a seemingly limitless supply of ideas and questions about why people act the way they do. At 22, he designed an assessment system for the Israeli military which helped it make better decisions about assignments and promotions. He was cut from a different cloth personally, feeling an acute sensitivity to the opinions of others where Amos felt none. 
    But when they started working together, in 1969, they hit it off like peanut butter and chocolate. Both had studied at the University of Michigan before returning to Hebrew University as professors, and one day Amos came to a seminar Danny was teaching. He came to discuss some work being done at Michigan about how people judge probability; the working assumption was that people have an intuitive grasp of statistical theory, though they sometimes make mistakes in expressing it. Danny, who had taught statistics, thought that was ridiculous, and said so; he won the argument that day. 
    That fall, Danny and Amos renewed the argument, from the same side. They devised a test requiring statistical reasoning and tried it out at meetings of psychologists. "The resulting paper dripped with Amos's self-assurance, beginning with the title he had put on it: 'Belief in the Law of Small Numbers.'" Their point was that the Law of Large Numbers doesn't actually apply to small numbers: "Even people trained in statistics and probability theory failed to intuit how much more variable a small sample could be than the general population–and that the smaller the sample, the lower the likelihood that it would mirror the broader population." Were psychologists committing such errors in their own work? Certainly they were; if their experiments gave different results on different trials, they were far more likely to rationalize the difference away than to conclude that their samples had been too small.

    The next year, they moved on to the obvious next question:"If people did not use statistical reasoning, even when faced with a problem that could be solved with statistical reasoning, what kind of reasoning did they use?" Danny and Amos identified several rules of thumb, which they dubbed 'heuristics', that seemed to be operating when people tried to generalize from partial information. The heuristic of representativeness applies to things like choosing athletes in a draft - the front office knows what a pro basketball player 'looks like', which is a useful rule of thumb, most of the time–but it would have missed Jeremy Lin and Steph Curry. (It has since been adjusted to account for them, but who knows what else is still missing?)

    Another heuristic they called 'availability'. "Any fact or incident that was especially vivid, or recent, or common–or anything that happened to preoccupy a person–was likely to be recalled with special ease, and so be disproportionately weighted in any judgment." It's why we're always fighting the last war, and doubling our fire insurance when our neighbor's house has burned. Like other mental shortcuts, it isn't always wrong–evolution made us this way for a reason, so to speak–but when it is wrong, we have trouble catching ourselves. The previously prevalent notion that people are inherently rational has great big holes in it.

    The hundreds of days, over eight years, that Danny and Amos spent in closed rooms, arguing in Hebrew and English, shouting and laughing, gave rise to published work that made everyone think differently. Their findings were disruptive to academic psychology, not surprisingly, but also to economics, medicine, history, and sports. Their lives and academic fortunes diverged in the late seventies, and they began collaborating with other people, though probably never as fruitfully as they had together. Michael Lewis has captured a little bit of their lightning in a bottle, and you may never mistake Man for a rational animal again.

Monday, January 2, 2017

The Only Rule Is It Has to Work

The Only Rule Is It Has to Work: Our Wild Experiment Building a New Kind of Baseball Team
Ben Lindbergh and Sam Miller (Henry Holt and Company, 2016)

   Ben Lindbergh and Sam Miller are baseball experts, after a fashion. They have written for Grantland and Baseball Prospectus, whose daily podcast they host together. But their playing experience peaked in Little League: they are statheads, sabermetric geeks, all theory and no practice. In 2014, they had a chance to try out some of their theories with the Sanoma Stompers of the Pacific Association, an independent league which they estimate to be four levels below the affiliated minor leagues, which is to say ten levels below the majors. The general manager brought them in as the two-headed head of baseball operations, which meant they were the guys wearing corduroys and hoodies, carrying clipboards and stopwatches.

   The gulf between theory and practice is only the first paradox they encounter. The next is that they want to be both bystanders and participants. They are in an experimental situation that seems to call for trying to fade into the background, but they're also going to try to make their suggestions stick; at least part of how decisions get made comes down to who is most willing to pitch a fit, which is not Ben and Sam's strong point.

   From a purely baseball point of view, some of their ideas are really good. Why can't you have a five man infield, or some other defensive shift that puts fielders where the ball is most likely to go? Some batters may adjust, but others will be flummoxed. And: it is demonstrable that the seventh inning or so is most dangerous for your starting pitcher, because the top of the order is seeing him for the third time by then. If your best pitcher is in the bullpen, you might as well use him, because if your starter gets shelled, there will be no save, anyhow. 'The other teams will laugh at you' is not at all a good enough reason.

   But that runs into another version of the contradiction between the geeks and the old-timers. For the first half of the season, they are managed by a crusty 37-year-old player-coach named Fehland Lentini. From where he sits, 'the closer is the closer,' which means he doesn't pitch the seventh inning, because he's the closer. Obviously. Lentini is biased toward the players he's friends with, which counteracts the Corduroy Crew's biases toward players they chose out of row R of their giant spreadsheet. Everybody's right sometimes, and in baseball, time and chance happen to them all.

   It's a beautifully quirky indy-league season up there in Sonoma, including a weekend cameo by Jose Canseco. The 2014 Stompers, somewhat accidentally, featured two things that were new to all of organized baseball, with the first Japanese-born manager running the second half of the season, and the first openly gay active player, a pitcher named Sean Conroy. His start on Sonoma Pride Day draws a modicum of national press attention, in addition to being a move in the game of 'how can we get our best pitcher in the game earlier?'

   Such layers of meaning and motive run all through the season, and the book. Ben and Sam started out imagining a laboratory for experimental baseball. "But once we started signing players and getting to know them, and especially once we saw them in spring training, we realized that they were not in our story so much as we were in theirs." How much of a prospect can any Pacific Association player can be, ten levels below the bigs, and without even the level of daily instruction found in A ball? The odds are terrible, but it's not up to baseball operations guys to make them worse. And as Sam concludes, some of those players would be better off if they just viewed baseball as the most fun way to spend a particular summer, because the odds of that working out are excellent.

Emailed January 2, 2017

Wednesday, November 30, 2016

Weapons of Math Destruction

Weapons of Math Destruction: How Big Data increases inequality and threatens democracy.
Cathy O'Neil (Crown, 2016)

   Cathy O'Neil is a math nerd who, in 2007, left her academic career to do math for a major hedge fund. She thus had a front row seat for the financial catastrophe of 2008, which was in large part a product of the kind of mathematical processes people like her were engaged in. "The housing crisis, the collapse of major financial institutions, the rise of unemployment–all had been aided and abetted by mathematicians wielding magic formulas.What's more, thanks to the extraordinary powers that I loved so much, math was able to combine with technology to multiply the chaos and misfortune, adding efficiency and scale to systems that I now recognized as flawed."

   After leaving D. E. Shaw, O'Neil recast herself as a data scientist, working on models in the Big Data economy, and once again privy to the magic formulas that are taking over in so many areas of our lives, from college admissions to parole recommendations. She started her blog, MathBabe, "to mobilize fellow mathematicians against the use of sloppy statistics and biased models that created their own toxic feedback loops."

   In Weapons of Math Destruction, she expands that warning to general readers, including the mathematically challenged. In fact, for the kind of algorithm she describes as a WMD, the math is usually opaque, anyhow: it's proprietary to the company that is profiting from it, whether by sorting your resume by the ZIP code you come from, or selling your search clicks to on-line marketers. The black-box quality is part of what makes WMDs so dangerous: the numbers they spit out are nearly impervious to challenges, even though the numbers that went into them may be biased, false, or completely spurious.

   Another dangerous aspect of WMDs is its potential to damage people's lives. O'Neil has a case study about the Washington, D.C., school department's program of rewarding teachers whose students improve, and firing those whose students don't. Such a practice can (and did) lead to the firing of good teachers, if the previous year's teachers cheated by padding their students' scores. The school hierarchy got what it wanted, to be seen as weeding out underperforming teachers, but since they didn't do any external checking to see if that's actually what they had done, they don't know how many good teachers they lost in the process. A healthy model would have an independent way of looking at the results to see if they made sense.

   That's unlikely to happen, however, when the third destructive effect starts to work: scale. If your credit report were managed by your own bank, you could speak with them about it face to face, and presumably establish that you were not the same John Bradshaw who had defaulted on that electric bill three years ago. But scale it up to the size of the big three credit bureaus, over billions of data points, and you are unlikely to find a person who can fix mixups; but you may very well pay for the errors not only in higher interest rates, but also in difficulty getting a loan, or a job, at all.

   And that's to mention only the official credit bureaus, which are governed by requirements that let you see the data they're using, and challenge it. E-scores generated by studies of internet use, or assumption about the street you live on, are under no such constraint, and their feedback loops tend to make unfairness worse. "There's a very high chance that the e-scoring system will give the borrower from the rough section of East Oakland a low score. A lot of people default there. So the credit card offer popping up on her screen will be targeted to a riskier demographic. That means less available credit and higher interest rates for those who are already struggling."

   Most WMDs embody corporate goals such as efficiency and profit; if corporations are persons, they tend to be sociopathic ones. Human beings are much better at thinking about justice than computers are, still, and perhaps always–if we choose to, and if we know what we're up against. Weapons of Math Destruction is disturbing, and distressing, but I couldn't put it down. Cathy O'Neil is a warrior for economic justice, and we ignore her at our peril.

Tuesday, November 1, 2016

Bad English

Bad English: A History of Linguistic Aggravation
Ammon Shea (Perigee, 2014)

   The t-shirt that says "I'm silently correcting your grammar" has my name on it. I come by it honestly: I have linguistically sensitive forebears on all sides. My father winced at sentence adverb 'hopefully'; my maternal grandmother loathed the word 'tasty'; and I have recently felt utterly compelled to fix the number disagreement in the last sentence of the Lord's Prayer.

   But I really don't want to be a jerk about it, so I'm delighted to add Ammon Shea's Bad English to my 'language wars' shelf. Shea has taken a serious historical look at a the usage rants and grammar guides of the past century and a half. English, it turns out, changes so quickly that no guide can hope to be the last word. The peeves of the nineteenth century very often look strange to us now; the expressions that were then considered beyond the pale have either sunk out of sight or become commonplace and unobjectionable.

   Such a change doesn't even need centuries. Remember 'Winston tastes good, like a cigarette should'? "Here we have an extraordinarily clear line of linguistic demarcation. People either feel that using like as a conjunction marks one as essentially subliterate or they have absolutely no idea what you are talking about and fail to see why this would be a problem of any sort." There were many more of the first group around when I was little, and there are many more of the latter now. 
   We will always have doomsayers, and yet, says Shea, "English is not dying. It is behaving exactly as it should, which is to say that it is changing. All living languages change–it is one of the things that indicate that they are still in use by a large number of people. The problem is that, while many people accept that our language is subject to change, they want to dictate what sort of changes will take place and that is a very difficult thing to do."

   The rules and roadblocks set up by fourth-grade English teachers frequently have perverse effects, as either the teacher or the student remembers the rule but not the principle. Years of drilling students not to say 'Jimmy and me are going to the pool' has led to generations of people who say 'Between you and I.' Shea makes an amusing example of George Orwell's famous essay, "Politics and the English Language," which promulgates six reasonable-sounding rules, and breaks five of them with abandon. There must be some sort of rule about that...

   Shea is not out to stop you speaking English the way you want to. If you prefer never to split an infinitive or strand a preposition, more power to you; but don't imagine that you are defending rules handed down on stone tablets. And feel free to set a picket fence around unique and perfect, but you may also want to "accept that certain words... are used by some people in a less semantically exact manner than you would yourself employ and hope that they have some other redeeming qualities that make up for this lapse."

   Shea is being contrarian here, and a little argumentative; but I think I prefer that attitude to the certainty and superiority of the self-appointed guardians of the language. English is doing just fine; it can look out for itself.

Email edition 11/1/16