Jacob Bernoulli, the famous Swiss mathematician, is best known for his introduction of the theorem known as the law of large numbers. Wikipedia defines the law of large numbers as “a theorem in probability that describes the long-term stability of the mean of a random variable. Given a random variable with a finite expected value, if its values are repeatedly sampled, as the number of these observations increases, the sample mean will tend to approach and stay close to the expected value (the average for the population).”

In other words, since the probability of flipping a coin and seeing a head (or tail) is 50%, the more we flip the coin, the more we will see the frequency of heads or tails approach 50%. Also, the expected value (50%) becomes more reliable the more we flip.

In his book “Against the Gods”, Peter L. Bernstein points out where Bernoulli performed various studies using marbles to conclude that a sample of 25,500 trials was necessary to achieve what he called “moral certainty.” Moral certainty he defines as a 2% chance of error.

He understood that the Law of Large Numbers is very important because it stabilizes long term outcomes for random events. In Las Vegas a player may beat the table in a given day, but eventually, whatever edge is built into the game will dominate, the more the game is played.

Is this true of trading systems though? I don’t think so.

First, there are big differences in reliability when testing such things as trading concepts and coin tosses. In the “real world” there are few longer term trading systems that can even produce 25,000+ trades as a sample size. Even if we could, Bernoulli had the advantage of working with a closed sample (5,000 marbles). Traders take in new data every day as new trades surface. This is the equivalent of throwing more marbles in the jar, perhaps even different colored marbles. Although a back test with 5,000 trades is likely to be more reliable than one with 200, it still has limitations that should be viewed with skepticism.

Tweet This Post

The quality of information determines the significance in the numbers. Probabilities are wonderful, but they are only as good as the quality of information that forms them. Information, such as the logic behind the method is critical. It is impossible to trust the interpretation of past data where logic is absent.

In his book “Fooled by Randomness” Nassim Nicholas Taleb illustrates this problem by saying:

This is not logical, even though there is a large sample of observations. The statistics should shed light on the logical intuitions of the developer.  In other words, it must make sense.

If a particular intuition is being tested (for example 20 day breakouts) it has a much higher degree of significance than throwing the computer at data and scanning for relationships. I’m sure with enough data we can find the relationship between Federal Reserve tightening and Federal Express deliveries, but it won’t have any significance.

Tweet This Post

Ed Seykota in the book, “Market Wizards,” writes, “Systems trading is ultimately discretionary.  The manager still has to decide how much risk to accept, which markets to play, and how aggressively to increase the trading base as a function of equity change.  These decisions are quite important, often more important than trade timing.”

It seems most sophisticated traders are aware of the fact that a system needs to be properly quantified and tested before trading. The sample size of the trades needs to be large. These traders are familiar with the terms of curve fitting and optimization. I wonder, however, how many traders continue to study the model as they trade their equity. How many understand the logic behind the entries, stops, exits, and money management techniques. How many are adjusting position size to meet expanding and contracting volatility and changes in market correlation.

Seykota also points out in the book, “Market Wizards,” that at times, “The markets shift from trending to non-directional price action.  These systems become unprofitable and under-capitalized. The trader without experience will be shaken out. Longevity is the key to success.”

It is this discretionary knowledge that examines the risk and helps build one’s confidence. Without this understanding, doubt will come into play and longevity compromised. A draw-down will inevitably force a decision based on emotion and not sound trading principles.

It is in our human nature to prefer pleasure, and avoid pain.  In spite of our natural instincts, good trading sometimes requires us to act against our natural inclinations. We must do what is necessary, not what is comfortable.

Tweet This Post