A hundred years ago, an unknown and austere looking Austrian medical professor set up a private practice where he could closely observe and study the human condition. From that humble beginning, the field of psychoanalysis was born and terms Freud invented, such as "ego" and "subconscious," are now widely used.

Genomics, a century from now, will probably be as mainstream as psychology is today. We already use the word “DNA” to describe an inherent nature of a corporation or a person. And I have a feeling that sometime in the future, the duo Watson and Crick, and Craig Venter, will be revered by society as Darwin is admired by ours.

Two issues

But I am uncertain whether the field of behavioral finance will grow in popularity with time. First, the conclusions are stagnant: we act irrationally because we are driven by emotions; we are often overconfident in our assessment of reality; we value things irrespective to their actual cost (also known as the Ikea effect); and we hate to lose more than we love to win.

Second, the lessons are stale. Above a certain point (I believe it is $78,000 for a household in the U.S), more money will not make us happier or contribute to our overall wellbeing. We should aspire to take trips and seek adventures as opposed to accumulating stuff. And when we interact with our financial advisors, we should remember what Darwin warned us: ignorance begets confidence.

In short, the doctrine of behavior finance provides great lessons that are hard to follow.   

To me, the height of behavioral finance lies in its past as it successfully undermined neoclassical economics. Just as Humanism countered evil ideas, such as survival of the fittest (read: scientific racism), fascism and racial hierarchy, behavior finance countered (far less wicked) ideas such as: we are all individuals that seek to maximize utility and profit; we have rational preferences between outcomes; and we act independently on the basis of full and relevant information.

The third and fourth generation of behavioral finance

Last week I heard a lecture by Professor Meir Statman. In his talk, he explained that the first generation of behavior finance PHDs focused on disproving the theories of the rational, utility-seeking economic person. But now, in the second generation, researchers have shifted their focus. They are now redefining what was perceived as irrational behavior to be a perfectly rational behavior. Or as Andrew Lo, author of Adaptive Markets, explains: "we are not rational actors with a few quirks in our behavior. Instead, our brain is a collection of quirks." 

In the third or fourth generation of behavior finance, using discoveries in brain research, evolutionary biology and artificial intelligence, we will surely have better tools to model the behavior of the typical investor. But irrespective as to how the future researcher will model our behavior, the challenge in understanding our financial behavior will remain intact.

The challenge

The challenge is that our behavior is flexible and is dependent on the environment and culture in which we live in. And our mental behavior does not follow physical laws or principles.

Let us walk through a game thought. Let us take a group of recent undergraduates and observe their TD Ameritrade trading activity over the next twenty years. After two decades, we will have enough data to hypothesize, for example, that (1) when the stock market declines by X percentage, we expect Y percentage of investors to sell Z percent of their stock portfolios. Explicitly, we will describe an observed behavior based on twenty years of trading activity and implicitly, we will claim that we understand a fundamental principle in the investor mindset.

Yet our observations would be entirely different if the sample of students was from the prior three centuries. In the early 18th century, in a world absent of modern ideas - such as social security benefits, education for all and meritocracy, for that matter - our financial behavior would be materially different. In the world prior to John Locke and Thomas Hobbes, one had to depend on personal relationships (and luck) to survive, not on a 401(K) account.   

And if you would like to further expand your imagination, I suggest you run a second same game thought. But now, shift from the days of the Scientific Revolution to the days of the Middle Ages. Good luck trying to understand the investor’s mind in times of constant war, famine and plague. Good luck attempting to observe the mindset of an investor in a world where if you had asked someone in the street why water flows from point A to point B, they would answer that it is God’s will.

Books about behavioral finance

Though I presented a skeptical view of the future of behavioral finance because it lies on a shaky assumption that human behavior stems from rules and principles, the study of behavioral finance has some concepts that should interest both serious investors and curious readers. And if you were to ask me which books on behavioral finance you should bring to a remote island, my answer would be as follows:

Start your reading with Daniel Kahneman. In Thinking, Fast and Slow, he gives an overview of behavioral finance key insights. I would then recommend a lighter read (Kahneman’s book can be a bit overwhelming). In Dan Ariely’s Predicatly Irrational, you learn about how behavioral finance concepts are tested in real life and what you can do about it. I also thought the CFA Institute did a great job in presenting the topic of behavioral finance and you can purchase the 2011 level 3 edition for $4.18 on Amazon. For the numbers-minded readers, by one of my all-time favorite economists, Robert Shiller’s Irrational Exuberance is a classic about irrational markets. And finally, while not on the concepts of behavioral finance per se, my boss recently handed me Michael Lewis’ The Undoing Project a Friendship that Changed our Mind, which I found to be an interesting read as well.