HOW MANY OPTIMIZATION INPUTS IS STILL ACCEPTABLE?
Download FREE resources to boost your trading
SOTR images-05-Emini min.png
SOTR images-06-STU min.png
SOTR images-08-TMI.png
SOTR images-07-SFB min.png
SOTR images-02 v4.png
SOTR images-09-v3.png
SOTR images-03-MSA min.png
SOTR images-01-v3.png
SOTR images-10-Checklist.png
SOTR images-04-Adaptrade builder min.png
Automated trading strategies that work!

© 2015-2018 SystemsOnTheRoad.com. All Rights Reserved  | Contact | About Me | Privacy Policy

  • LinkedIn Social Icon
  • Twitter Social Icon
  • Instagram Social Icon

DISCLAIMER:  Futures trading systems and commodity trading bear a high degree of risk. People can and do lose money.
Hypothetical results have many inherent limitations. Past performance does not guarantee future results. 

 

ACTUAL RESULTS SHOULD BE VIEWED WITH CAUTION, BECAUSE PAST PERFORMANCE IS NOT NECESSARILY INDICATIVE OF FUTURE RESULTS. HYPOTHETICAL PERFORMANCE RESULTS HAVE MANY INHERENT LIMITATIONS, SOME OF WHICH ARE DESCRIBED BELOW. NO REPRESENTATION IS BEING MADE THAT ANY ACCOUNT WILL OR IS LIKELY TO ACHIEVE PROFITS OR LOSSES SIMILAR TO THOSE SHOWN. IN FACT, THERE ARE FREQUENTLY SHARP DIFFERENCES BETWEEN HYPOTHETICAL PERFORMANCE RESULTS AND THE ACTUAL RESULTS SUBSEQUENTLY ACHIEVED BY ANY PARTICULAR TRADING PROGRAM. ONE OF THE LIMITATIONS OF HYPOTHETICAL PERFORMANCE RESULTS IS THAT THEY ARE GENERALLY PREPARED WITH THE BENEFIT OF HINDSIGHT. IN ADDITION, HYPOTHETICAL TRADING DOES NOT INVOLVE FINANCIAL RISK, AND NO HYPOTHETICAL TRADING RECORD CAN COMPLETELY ACCOUNT FOR THE IMPACT OF FINANCIAL RISK IN ACTUAL TRADING. FOR EXAMPLE, THE ABILITY TO WITHSTAND LOSSES OR ADHERE TO A PARTICULAR TRADING PROGRAM IN SPITE OF TRADING LOSSES ARE MATERIAL POINTS WHICH CAN ALSO ADVERSELY AFFECT ACTUAL TRADING RESULTS. THERE ARE NUMEROUS OTHER FACTORS RELATED TO THE MARKETS IN GENERAL OR TO THE IMPLEMENTATION OF ANY SPECIFIC TRADING PROGRAM WHICH CANNOT BE FULLY ACCOUNTED FOR IN THE PREPARATION OF HYPOTHETICAL PERFORMANCE RESULTS AND ALL OF WHICH CAN ADVERSELY AFFECT ACTUAL TRADING RESULTS.

OUR NEW PLACE

I'm a paragraph. Click here to add your own text and edit me. Let your users get to know you.

#051: HOW MANY OPTIMIZATION INPUTS IS STILL ACCEPTABLE?

In today’s article, I would like to share with you one interesting study, which will answer to you one important question – how many optimization inputs are acceptable for a strategy and what are the limits we should not cross? Let’s take a look at it.

 

The study is based on an excellent work of my colleague in our hedge fund, who takes care of an automated strategy development process, the database and performing of analytical tasks in this database. Let’s first take a look at the methodology how was this study performed.

 

In our hedge fund, there is currently over 700 strategies, most of them on futures markets – intraday and swing ones. All of these strategies have already passed our tests, so they all meet our minimum quality requirements and are usable for live trading.

 

 

This is where all the work doesn’t end, but where it all begins. All our strategies are continuously monitored and updated, so we receive new important information about all strategies and their performance. So we have available not only the 3-month period of additional out of sample, but even the real out of sample – i.e. data that haven’t even existed when the strategy was developed. This gives us unique possibility to monitor the real OOS performance and compare it to the previous performance.

 

Further on, as part of our workflow we have created an index, that monitors the real OOS performance to all previous data we had available (thanks to fantastic work of my colleague) and this index (which is unfortunately private and I won’t share any more information about it) helps us to analyze what factors impact the real OOS performance. Today we analyze really big quantity of different aspects and components (plus we use Python Jupiter).

 

One of these studies was performed in order to show us what is the relationship between the OOS performance and the number of optimization inputs.

 

And this is the study I would like to share with you today.

 

The results are really simple for interpretation – simply, the higher avgDhidx value, the better is the performance of strategies with given number of parameters. We have compared 1-7 input parameters (none of our strategies have more than that) and here are the results:

 

 

Even though the study is not perfect as yet, as we don’t have enough samples for certain variants (we are developing new strategies every day, so the sample size is continuously growing), it is possible to state some general results:

 

  • It seems that having more than 6 optimization inputs is dangerous. 

  • It is the only level in our test that has negative index value.

  • The range from 2 to 6 seems reasonable (number 4 is a certain anomaly, which we need to investigate further later on).

  • Having just one optimization seems not sufficient, which is understandable when looking at the complexity of markets in these days.

  • If you want to keep the number of optimization inputs low (for example due to the number of iterations), it is better to choose 2-3 optimization inputs.

 

For me personally, the most important is to see and to confirm, whether a strategy with fewer optimization inputs, is more robust. Our study hasn’t confirmed that. You can have even 5-6 optimization inputs and as long as the strategy passes all our robustness tests, it can be as robust as the strategy which has 2-3 optimization inputs.

 

Happy Trading!

 

Tomas

 

 

 

Please reload

Author: Tom Nesnidal (more about me
Join 15 000+
LinkedIn Followers
Join 5 000+
Twitter Followers
Get Social With Tom
  • Black LinkedIn Icon
  • Black Twitter Icon
  • Black Instagram Icon
  • Black RSS Icon
What others are saying

"Tomas is one of the most creative traders I know. He is able to generate unique trading ideas and elegant solutions to system development challenges."

Andrew Swanscott, BetterSystemTrader.com

"Tomas has been a professional trader for more than a decade and I have had the privilege of monitoring his accounts in action since 2006. His systems are performing as some of the best I have ever monitored and executed. I have been in the trading industry since 2002 and worked with many developers from all over the world. Tomas is definitely the one to consider."

Martin Lembak,

Systems Trading Expert,

MFRM, CAIA 

"Tomas is a professional trader, who for the last 10 years has specialized in developing trading systems. We have been tracking his trading systems for about 5 years and they generally show very robust, stable and above average performance. Striker is pleased to work with someone like him - a real professional with enthusiasm and deep knowledge of trading."

William Galwas, President Of Striker Securities, Inc 

"Personal consultation with Tom helped me to re-evaluate the complexity of my robustness testing and optimization processes. Plus, it has helped me with a specific plan on how to take things forward. It has given me some great ideas on how to avoid overfitting and make my testing more robust and provided tips on low hanging fruit in terms of the best markets to trade for intraday/short-term breakouts."

Craig Peters,

semi-advanced trader,

United Kingdom 

"Tom's approach to Automated Trading Strategies design, tests of robustness and portfolio diversification is really unique. He has been a professional trader for many years and the depth of his understanding of Breakout strategies and Market Internals is hard to find elsewhere."

Antonin Fisher,

Hedge Fund Manager,

Czech Republic 

"Tom´s systems and trading approach do really work and can bring good, stable and reasonable returns. I can highly recommend him as a teacher."

David Hruby,

Trader, Czech Republic 

"Tomas is truly an inspiring mentor and great trader with a positive life attitude. He has got very deep understanding of markets, breakout strategies, robustness testing procedures, and Market Internals."

Jan Lesanovsky,

Client, United Kingdom 

"As an experienced trader, I used Tomas´s consultation to get more clarity on best-practise robustness testing procedures, risk management, and portfolio management. He was quickly able to help me refine my current trading and system development approaches, and pointed me to really practical measures that I could implement immediately. I found Tomas to be not only extremely knowledgeable but very helpful and a great teacher. This has given me more confidence in developing robust strategies that work, and I would recommend his mentoring as extremely valuable for any systems trader."

Trevor West,

advanced trader, Australia 

The author of this trading blog has been featured on: