“Our brains are wired for narrative, not statistical uncertainty. And so we tell ourselves simple stories to explain complex thing we don’t — and, most importantly, can’t — know. The truth is that we have no idea why stock markets go up or down on any given day, and whatever reason we give is sure to be grossly simplified, if not flat out wrong.”
Cognitive biases aside, are there real, physical limits to what we'll ever know? How do we go about finding out about them? And where do we start?
We start from 1686 in France with Gottfried Leibniz, who stated that "a theory has to be simpler than the data it explains; otherwise it does not explain anything." This makes perfect sense because one can always construct an arbitrarily complex mathematical law to describe random, pattern-less data. Conversely, if the only law that describes some data is arbitrarily complicated, then the data is actually random and contains no patterns. Along the same vein, William of Occam illustrated a very good point with his metaphorical razor.
Gregory Chaitin has published a delightful book in 2005, called "Meta Math!: The Quest for Omega", that is "based on measuring information and showing that some mathematical facts cannot be compressed into a theory because they are too complicated." Now, that is a piece of news worth knowing, before we embark on our otherwise quixotic quest for data, statistics, and ultimately, knowledge. And what is knowledge for us but a neat summary of why and how things work (or not work!) in the market. Nobody wants to be running a fool's errand!
Perhaps that piece of news shouldn't have come as a surprise for us in the first place. After all, we are already familiar with Kurt Gödel’s incompleteness proof of mathematical logic. Alan Turing, too, had famously confronted us with his Halting Problem in every class of computation theory that was taught. Apparently, we have no shortage of things that man or machines can't do, in both mathematics and computer science. It's a sobering thought in this modern era of Big Data before Skynet.
As Chaitin pointed out, comprehension is compression; so a useful theory is a compression of the data. The implication for quantitative finance is clear: a useful theory of the market is a statistical compression of the market data. The simpler the statistics, the better we understand the market. This is a comforting thought for us. Complexities could be overwhelming, always putting us at a loss for words to explain them. Simplicity is good.
We always assume things work a certain way for a reason. That "know why" is called knowledge. The ancient Greeks called it epistêmê. Furthermore, if something is true, it must be true for a reason. This is the principle of sufficient reason that Leibniz, and the ancient Greeks, believed in. Mathematicians believe it, too. They demand proofs for everything. Mere instances would not do, not even after seeing millions and millions of concrete data points. Irrefutable proofs are required for absolute conviction.
Are the methods of quantitative finance more like physics, math, or computer science? (Image Credit: Sci. Am.).
So it came as a big surprise to everyone when it was revealed that there are mathematical facts that are true for no reasons. In fact, Chaitin showed that there are infinitely many mathematical facts that have no theories to explain them. He called these facts irreducible, both computationally and logically. In the context of markets, that means there are infinitely many market truths that cannot be neatly summarized in a statistic. These market truths could be anomalies, stylized facts, or regularities, depending on who you ask. That's good news for us. The opportunity frontier is boundless! And they are very complicated. But how could this be?
Turns out that there is a very neat construct called the omega number, i.e., a very long string of zeroes and ones, that Chaitin invented to help us understand this ultimate limit on logic and reason. We won't go into the details here, except to note that omega is a specific, well-defined number that cannot be calculated by any computer programs. Interested readers are referred to Chaitin's highly readable article, "The Limits of Reason". The implication for mathematics is immediately clear: there can never be a "theory of everything" for all of mathematics. Similarly, trying to find a reason behind every market behavior may be a fool's errand. In many cases, we should be happy to simply note their axiomatic existence, beyond logic, reason or proof. Problem is: we can't really tell when they are irreducible and when they are not! And what if we are wrong?
Incompleteness tells us that extensive statistical analysis can be extremely convincing, even if there is no underlying reason that we can find to explain the observed market phenomenon. So, what does this mean for our Zeroth Rule of Trading: "We do not trade what we don't understand"? Do we still insist on understanding or do we give up on opportunities that we can’t explain? How do we decide? And what do we do the next time when the cartographer pressed her new treasure map into our hands, and whispered knowingly, "Trust me, it's all good." Do we still strive to first understand how it works before sending MVP, risking life and limb, on a perilous journey across the seven seas?
It seems we have arrived at an impasse. We cannot abandon reason; and yet reason eludes us when we try to understand every market truth that we encounter. In the next post, we shall take a closer look at the definition of market imperfections and market inefficiencies specifically, and see if we can find some clues there to resolve this tricky situation.
References:
- Taleb, Nassim Nicholas (2010). The Black Swan: The Impact of the Highly Improbable (2nd Edition). Random House.
- Chaitin, Gregory (2005). Meta Math! The Quest for Omega. Pantheon Books, New York.
- Chaitin, Gregory (2006, March). The Limits of Reason. Scientific American, pp. 74-81. Retrieved from: http://www.umcs.maine.edu/~chaitin/sciamer3.pdf
- Yanofsky, Noson S. (2013, August). The Outer Limits of Reason: What Science, Mathematics, and Logic Cannot Tell Us. MIT Press.
- Dvorsky, Gregory (2013, April 26). How Skynet might Emerge from Simple Physics. Io9: We Come from the Future. Retrieved from: http://io9.com/how-skynet-might-emerge-from-simple-physics-482402911