It helped explain why a simple algorithm is often better than the most experienced doctors at diagnosing stomach cancer, why so many financial experts failed to foresee the implosion of the housing market, and why professional basketball teams make costly errors when picking players — in short, why people’s instincts are often so wildly wrong.
In 2002, Mr. Kahneman won the Nobel in economic science, a field he had no formal training in, for demonstrating how people make decisions when faced with risks and uncertainty. (When asked if their work had any application to artificial intelligence, Mr. Tversky, who died in 1996, countered that they were more interested in exploring “natural stupidity.”)
When he began digging into their research, Mr. Lewis found an even more compelling story about a fertile intellectual partnership that ended too soon, when Mr. Kahneman and Mr. Tversky had a falling-out over who was receiving more credit for their discoveries.
“It’s a love story,” Mr. Lewis said during an interview in Manhattan this fall. “It’s this bromance of such great intensity, with such fertility, and the children are ideas, the children live on.”
“The Undoing Project,” which will be released on Tuesday, seems like a departure for Mr. Lewis, whose best-known works have largely been character-driven narratives that reveal something unexpected about the way markets work. Often, they feature eccentrics and visionaries who see things that others are blind to — like Mr. Beane of the A’s, or Michael Burry, the misanthropic hedge fund manager in “The Big Short,” who made a fortune by betting against the housing market.
“The Undoing Project” centers on psychology, a field Mr. Lewis, a financial journalist who once worked as a Wall Street bond salesman, knew next to nothing about. “I had cold feet about this, for very good reasons,” he said.
On closer inspection, though, it becomes obvious that Mr. Kahneman’s and Mr. Tversky’s research shaped many of the subjects that Mr. Lewis has explored, to an almost unsettling degree.
“Danny and Amos’s work has something to say about why baseball players get misvalued, or why people can’t see the value of Michael Oher in ‘The Blind Side,’ or why people can’t see what mortgage-backed securities really are,” he said.
In a roundabout way, their work influenced Mr. Lewis’s career. Their research demonstrating how people behave in fundamentally irrational ways when making decisions, relying on their gut rather than available data, gave rise to the field of behavioral economics. That discipline attracted Paul DePodesta, a Harvard student, who later went into sports management and helped upend professional baseball when he went to work for Mr. Beane.
“Without their basic work, Michael Lewis doesn’t get to write ‘Moneyball,’” Starling Lawrence, Mr. Lewis’s editor at W. W. Norton, said of Mr. Kahneman and Mr. Tversky.
It wasn’t until Mr. Lewis read a 2003 review of “Moneyball” in The New Republic, which mentioned the connection between Mr. Kahneman’s and Mr. Tversky’s research and the data revolution in baseball, that he realized the extent to which their work had shaped the book. He became fascinated with their research and their unique partnership. Four years later, he finally worked up the nerve to call Mr. Kahneman and request a meeting.
It is hard to imagine Mr. Lewis, 56, feeling skittish around anyone. A New Orleans native, he comes across as someone who is at ease in any situation, whether he is shadowing professional athletes and coaches, foul-mouthed derivatives traders or bigwigs at the International Monetary Fund. After studying art history at Princeton and getting a master’s degree from the London School of Economics, he worked at Salomon Brothers selling bonds. He left to write “Liar’s Poker,” his 1989 memoir about his experience on Wall Street, which he sold to Mr. Lawrence at W. W. Norton. They have worked together ever since, on more than a dozen books.
Unlike many nonfiction writers, Mr. Lewis declines to take advances, which he calls “corrupting,” even though he could easily earn seven figures. Instead, he splits the profits from the books, as well as the advertising and production costs, with Norton. The setup spurs him to work harder and to make more money if the books are successful, he says.
“You should have the risk and you should enjoy the reward,” he said. “It’s not healthy for an author not to have the risk.”
Mr. Lewis’s books have sold more than nine million copies, and three have been adapted into successful feature films. Norton is printing nearly 500,000 hardcover copies of “The Undoing Project.”
Other authors speak about him with admiration bordering on reverence.
“His willingness to jump in and take on these topics that are incredibly difficult to represent on the page has always left me awe-struck,” the New Yorker writer Malcolm Gladwell said in an interview. “There’s not a single book he’s written that I thought I could have pulled off myself.”
“The Undoing Project” — which tackles abstract concepts like utility theory, prospect theory and heuristics — may be Mr. Lewis’s most arcane and ambitious undertaking to date.
“I don’t know how you write a book about a couple of psychologists,” Mr. Lawrence said. “Let’s just say, it’s not promising in another writer’s hands.”
When Mr. Lewis first met Mr. Kahneman in 2007, over coffee at Mr. Kahneman’s house in Berkeley, Calif., where Mr. Lewis also lives, he had no intention of writing a book about him. Instead, he found himself offering writing advice to Mr. Kahneman, who was struggling with his own book, which laid out his theories about human cognition and decision making.
“He would say, ‘It’s going to ruin my reputation. I don’t know why I agreed to do this,’” Mr. Lewis said. “At some point I said, ‘If you’re not going to do it, I’m going to write something.’”
Mr. Kahneman’s literary agent vetoed that idea. In 2011, Mr. Kahneman reluctantly published his book, “Thinking, Fast and Slow.” It sold about 1.5 million copies.
Despite the book’s success, Mr. Lewis felt the story was incomplete. Mr. Kahneman had largely left out a huge piece of the narrative — his own biography and his complicated relationship with Mr. Tversky. Mr. Lewis continued to lobby Mr. Kahneman to let him write about their work.
Mr. Kahneman was ambivalent, arguing that the book would read like a textbook. “I said, ‘You don’t understand what a good character you are, and the fact that you don’t understand it is part of what makes you a good character,’” Mr. Lewis said.
It took a few years, but Mr. Kahneman finally acquiesced. “He’s a pessimist, so he thought it would come out terribly,” said Richard H. Thaler, a professor of economics at the University of Chicago and a longtime friend of both Mr. Kahneman and Mr. Tversky. “Danny needed to be convinced and charmed, and Michael was up to the task.”
Over the next eight years, as he completed other books, Mr. Lewis spent countless hours with Mr. Kahneman, hiking in the hills surrounding Berkeley, traveling to Israel with him twice, and meeting at his apartment in Manhattan. He also dug through Mr. Tversky’s archives and correspondence, with the help of his widow, Barbara Tversky.
People close to both men, including Mr. Thaler and Ms. Tversky, say Mr. Lewis captured the intensity of their relationship and their individual quirks. Colleagues described how the pair would finish each other’s sentences and could often be heard cackling from behind an office door as they wrote dense academic papers. Mr. Tversky was the bold one who delighted in undermining well-established dogma within psychology. Mr. Kahneman was cautious, sensitive and deeply pessimistic.
A few of the more colorful details in the book stray from reality, said Ms. Tversky, who praised Mr. Lewis for getting the “broad strokes right.” For example, Ms. Tversky disputed a delightful anecdote about how her husband was so immune to social convention that he would strip down to his underwear and run outside when he felt like getting exercise. Actually, he would on occasion run on the treadmill in his underwear, but never outside, she said.
Mr. Kahneman, now 82, declined to be interviewed for this article, but offered a statement through his publisher.
“Naturally, there are points on which I would quibble, but this is Michael’s book, not mine,” he wrote. “Some things in the book surprised me, some hurt, but I am glad it was written, and grateful to Michael for writing it. In the course of many conversations and countless email exchanges I came to understand the defining chapter of my life better than I had understood it before.”
Mr. Lewis said he expected Mr. Kahneman to have a complex reaction to it.
“It would be out of character for Danny just to like it, because he doesn’t like much,” he said. “It would not be out of character for Danny to be kind and come to terms with it in a condescending way, like, that’s the best he could do, and I admire him for trying as hard as he could.”