Garbage In – Garbage Out
Friday, August 14th, 2009
Like any other effort in science, the back test can vary from shallow and useless to very useful. It is a matter of the quality of information that forms the test. A shallow back test, and the discipline to stick with it, is a recipe for disaster.
The quality of information determines the significance in the numbers. Probabilities are wonderful, but they are only as good as the quality of information that forms them. Information, such as the logic behind the method is critical. It is impossible to trust the interpretation of past data where logic is absent.
In his book “Fooled by Randomness” Nassim Nicholas Taleb illustrates this problem by saying:
“I have just completed a thorough statistical examination of the life of George Bush. For 55 years, close to 16,000 observations he did not die once. I can hence pronounce him as immortal, with a high degree of statistical significance”.This is not logical, even though there is a large sample of observations. The statistics should shed light on the logical intuitions of the developer. In other words, it must make sense.
If a particular intuition is being tested (for example 20 day breakouts) it has a much higher degree of significance than throwing the computer at data and scanning for relationships. I’m sure with enough data we can find the relationship between Federal Reserve tightening and Federal Express deliveries, but it won’t have any significance.




on June 19th, 2010 at 8:00 PM Said:
[...] numbers until you get a nice painting, rarely work in the real world. As I have stated in previous posts, a system should be grounded in logic that stems from market [...]
on July 7th, 2010 at 5:44 AM Said:
[...] numbers until you get a nice painting, rarely work in the real world. As I have stated in previous posts, a system should be grounded in logic that stems from market [...]
on July 16th, 2010 at 2:38 AM Said:
[...] numbers until you get a nice painting, rarely work in the real world. As I have stated in previous posts, a system should be grounded in logic that stems from market [...]