#059: GENETIC OR “CLASSICAL” OPTIMIZATION?
For some reason, many beginning traders have a fear of using genetic algorithms (GA) to optimize their systems. However, it is obvious that classical (exhaustive), as well as modern (genetic) optimizations play their part in strategy development. Today I will try to explain what differences both types of optimization can offer and help you see that you don’t have to be afraid of “genetics”.
The biggest advantage of genetic optimization is, without a doubt, time. Especially if we have more parameters to optimize, we can literally shorten the whole process from days to only a few hours – which is an acceleration worth thinking about.
Most traders that don’t have any experience with genetic optimization have a feeling that thanks to this shortcut the whole process will be somehow like “cheating”. After all, how is it possible that GA can find solutions in 5-10% of regular time, when the classical optimization process takes “full” time?
First of all, in a surprisingly high amount of cases, genetic algorithms can find completely similar combinations which classical optimizations would find as well. Secondly, it can find combinations that can differ. This can be helpful, as I will explain shortly.
Let’s start with a simple comparison that shows rather realistic solutions found by GA - how close or how distant they are to solutions that classical optimization would find.
In this article I have chosen as an example a system of mine with 5 optimization parameters in total and on which I have applied Walk Forward Optimization (WFO). It means that I have divided used data into 7 segments and I have carried out 7 optimizations (one on each segment of data) to have more samples to compare. The comparison turned out as follows:
First of all, I have marked in yellow the parameters that GA found exactly the same as the classical method of optimization. From the picture it is obvious that it was in more than half of the cases – 54%. That isn’t bad at all, especially if we take into account the reduction of the original time needed for optimization is of 90-95%. Afterwards, I have marked in red the parameters GA found as very close to those the classical optimization technique found – the outcome shows there isn’t any noticeable difference. Yellow and red fields add up to 69%, which is a pretty decent result. In almost 3/4 of the cases, the results from GA were the same or very close to those where the classical optimization was used. However, GA found solutions in 5-10% of the time than what was originally needed.
The remaining 31% of results differ fundamentally, which can be a bit concerning for some people. But as I will explain later there is no need for that.
Usually the reason for a difference between results is that classical optimization is searching for the so-called local optimum and genetic algorithms rather tend to find a global optimum.
What exactly are these?
Imagine you have two parameters – N1 and N2 – and you would like to find a combination with the best profit. You carry-out an optimization which you can afterwards display in a 3D chart:
As you can see from the picture, the best combination shows profit at around the level of 130,000 USD. The problem is that this optimum is local, and not global. The local optimum doesn’t show similar results in the immediate vicinity. In other words, there is only one combination that provides great results, but any surrounding combinations will not. Local optimum is with a high probability over optimized combination and a classical optimization technique tends to search for local optimum.
Unlike genetic algorithms which are going for global optimum. I have marked such area in the picture with an arrow. As you can see, in this area the profit is around 110,000 USD, but optimum has many functional “neighbours”. Therefore, global optimum is more robust than local optimum (which earns more, but the combination of parameters is from the robustness point of view disputable).
Naturally, genetic algorithms don’t always hit global optimum and occasionally they also include a local optimum in their solutions. At the same time, classical optimization doesn’t always produce only local optimum – oftentimes, the best combination is part of a global optimum (and these are the moments when in all probable solutions found by GA and classical optimization will correspond or will be very close). Unfortunately, genetic algorithms also use the element of chance, so the final result is substantially less controllable – well, this is the price to pay for the acceleration of optimization.
Personally, I use both types of optimization and I think that both have their places in the development of ATS. Genetic algorithms may seem scary at the beginning, but once you carry out a few tests and you get used to them, you will realize that they are an irreplaceable help in many cases, and with their help you will do much more work. Of course, there are tons of academic studies discussing aspects of both optimizations. We could polemicize about anything in regards to both methods – but I personally am not an academic; I am a pure practitioner. From the practical point of view GA, can be really priceless, especially when used in a sensible and smart way. Therefore, there is no need to be afraid of them but it is important to use them with sense and reason.
Happy Trading!
Tomas
