My progress as a full-time investor & trader | Page 3 | Metals Mine

Options

Go to
Permalinks
Print Thread

Similar Threads

Are you a full Time Trader Or Part Time? 32 replies

Full-time student, Part time trader 7 replies

from retail trader to full-time trader 17 replies

Not enough time to be full time trader, so i do this. Good idea or not? 14 replies

Attachments: My progress as a full-time investor & trader
Exit Attachments

My progress as a full-time investor & trader

  • #41
  • Sep 26, 2013 12:20am Sep 26, 2013 12:20am
Quoting erik1
Nice work compared to the rest of the "herd" guys... I've been working on automatic system generation based on genetic programming for several large institutions for the past decades (being independent and successful using automatic generation approaches for the past 10-15 years). Just some warning: Manual system generation approaches are old-school. You will be predated and front-run by other traders if you do this. Using genetic programming you can mine systems with extremely good historical statistics and very high chances of success in live...
Hi erik1,

Thanks for your comments, they certainly provide some good food for thought.

The field of Neural Network and Genetic Algorithm based trading systems is really fascinating and being a developer at heart it's something I'd love to get into more seriously. I have spent some time reading up on this field and reading a little about frameworks available, but in truth I haven't pursued it in any detail simply because I haven't had time as yet.

There are a lot of things I want to study and become proficient at and one of the key challenges for me is not to move from one to another without having achieved some significant/quantifiable milestone. I also like to focus on what appears to provide the best bang for buck in regards to potential. In the field of forex I spent a significant amount (relative to other systems) with little to show apart from these EAs - but I needed to move on for now and circle back a little later. I also actively manage a reasonably significant quantitative based strategy in stocks (which is where I make the bulk of my income) and my current focus is improving my result in trading a live intra-day discretionary strategy on the eMini s&P which seems to have some excellent potential.

In any case, NN and GA are definitely on my list to study in the future and I would love to have a chat with you when I am more knowledgeable in the area, hopefully you can give me some further pointers.

Take Care,

mim2005
  • #42
  • Edited 1:42pm Sep 26, 2013 3:48am | Edited 1:42pm
Quoting erik1
From your work I would expect system failure within the next 1-3 years, with actual live trading statistics giving you a profit of about only 30% what you expect and drawdowns will be at least 2.5 times as they've been in your back-tests. Your approach is too weak to lead to significantly higher profitability but you're clearly headed down a better path than the rest. Best of luck guys.
Either EU will 'trend' far enough, often enough, to make the system viable, or it won't. Hence, regarding future performance, I have no expectations. We'll just have to wait and see.
  • #43
  • Sep 26, 2013 9:03am Sep 26, 2013 9:03am
Quoting erik1
Nice work compared to the rest of the "herd" guys... I've been working on automatic system generation based on genetic programming for several large institutions for the past decades (being independent and successful using automatic generation approaches for the past 10-15 years). Just some warning: Manual system generation approaches are old-school. You will be predated and front-run by other traders if you do this. Using genetic programming you can mine systems with extremely good historical statistics and very high chances of success in live...

I also put my bet against this 'robust' system in 3 years time.Its just the wise thing to do.You should too,bet conservatively and set your risk of ruin pretty low.
wo-yoy! wo-yoy! wo-yoy! wo-yoi! wo-yoy-yoy-yoy!
  • #44
  • Sep 27, 2013 10:22am Sep 27, 2013 10:22am
CONCEPTS, FRAMEWORKS AND ANALOGIES - MIND STUFF


In the nine odd months I spent studying systems and looking for edges, I noticed that the way I think about designing trading systems has changed.


So I've decided to journal a little about how I think about certain ideas and concepts. No doubt many others have their own way of thinking about things and much of what I will describe is well trodden ground ... but let's give this a try.


Warning - the concepts and analogies I will discussed are provided with no mathematical or statistical basis, merely my intuition formed over a short period. I would say it's simply the way my mind has worked out how to understand what I have seen in my trial of different strategies and playing with many parameters. It provides a little insight into how I visualise systems working. I tend to try to create mental frameworks like this to help me explore concepts and essentially perform thought experiments. In this post I will stick to covering the way I think about the systems being used by the three EAs discussed earlier in this post.


I offer this simply as it may fast track other peoples understanding of their own systems or at least provide an alternative point of view to consider.




HOW I THINK ABOUT WIN RATE, RISK REWARD, FREQUENCY, PSYCHOLOGY AND STATISTICAL VALIDITY


In the game of developing a consistently profitable strategy I found that I was essentially competing against myself i.e. how much time, study, discipline and psychological toughness could I muster.


From a strategy perspective I focused on finding edges that could defeat the spread or commission significantly. If you have edges that can consistently and significantly beat the spread in theory you have the essence of what can be a successful system (I know theres other important stuff ... but without an edge to start with theres little hope).


But the spread has a significant toll.


In higher time frames with larger stops the spread represents less of an obstacle to your edge but unfortunately edges obtained at higher timeframes usually have a much lower frequency and this reduced frequency/opportunity may result in less overall system profitability.


I primarily consider three things when looking at an edge. Win Rate, Risk vs. Return and Frequency. Increasing any one of these can improve your system. Unfortunately they are interelated and unique to your edge and as most will know increasing one can be easy but often detrimental to another, so some work is required to find a good balance of the three.

Most traders know the inverse relationship between Win Rate and Risk Reward. In my view i never think of these two things separately - ultimately I look for the balance which proves most profitable overall (which is different for each edge). I don't worry too much if the win rate is only 30% or 70% because I know overall the there is a profitably optimal point that can be worked out (historically but obviously it can change). An obstacle that is more important for me is to consider how I will handle extended consecutive losses psychologically.

In testing EAs I have found that sometime the best profitable result is from having a system that is skewed to a very low win rate. They experience many regular losing streaks but occasionally the winning trade is extremely good. The problem with this is that psychologically I find it difficult to trade systems that experience long draw downs e.g. 30 losers in a row and more importantly you run into a problem that although your winners are very large, statistically they happen much less and hence your frequency of wins is too low to really be statistically valid (you end up curve fitting for a few big winners).

So there is a balance between optimal results, psychologically being able to trade a system and statistical significance.

The other factor which is often much less considered is frequency. In general term if you have two identical edges with the exception that one occurs twice as frequently as the other then the more frequent edge will have twice the profitability of the first.

Wow, so the answer is simply to use strategies with edges in much smaller time frames, not so fast, because as we mentioned before the spread now becomes a much more significant obstacle.

So you need to look for edges that are as frequent as possible but can still overcome the spread. I highly suggest that you either empirically or mathematically determine the effect of the spread against you, as this is what your trying to significantly be able to beat before you even have a chance.


EXITS AND ENTRIES

When I design systems I consider both the entry and exit edge. Some people will argue one is more important than the other, I think mathematically they are equally important.

Don't get caught up in thinking you need to utilise the same entry edge to with your exits ... you don't, you can use an entry edge and a completely different exit edge and be profitable.

Some people utilise expectancy to determine where they should exit and this is perfectly fine also. Although personally I prefer finding a good different exit edge because in this game edges can add up (especially if they are completely unrelated) and the more edges you have in a system the better. (People may disagree with this but in the experiments I've done it's what I've found to be the case)

In the EAs I have shown the entries use specific small intra day edges and the exists use an edge that tries to exploit higher time frame trends if available.

Two different edges.

Trading one system that utilises two different edges (one for entry and another for exit) is mathematically equivalent to trading two systems one of which uses edge A for entry and exit, and the other which uses edge B for entries and exits. This concept was significant to me and changed the way I thought about systems.

In a nutshell the EAs will perform very well when the market trends (picking up substantial parts of the trend multiple times) but they will perform poorly when markets ranges, often drawing down consistently. As can be seen perfectly in the first 10 days of trading on the trade explorer.

On a positive note - since doing a fair bit of discretionary trading I am fairly certain I know how I can change the EAs so that the can change their trading style to suit ranging conditions - but I won't be revisiting them for some time to put this in.


SHOOT YOUR ARROW WELL, MAKE SURE YOU HAVE A PARACHUTE AND PRAY FOR WIND

Here is an analogy which may help you imagine how a combination of different entry and exist edges work together, it is really not a very good analogy but it's what I used when thinking about edges I had found ... and it's all I've got

When i was a child I was given a little plastic toy solider to play with. What was special about the toy soldier is that he had a parachute that would engage when the soldier was thrown in the air.

So here goes ...

Imagine you have arrows with inbuilt parachutes (like my toy soldier) which engage when the arrows start to fall to the ground.

You are an archer shooting these arrow straight up into the air but unfortunately a forcefield exist one meters above you. The forcefield represent the spread. If your strength in shooting the arrows (your entry edge) is not enough to be able to completely penetrate the forcefield consistently then you don't have a chance of a profitable system. You need to find an entry edge that can consistently over time beat the forcefield. This is the job of my entry edge, the better the edge the higher the arrows go into the air.

Note: As soon as I mentioned the "forcefield" I started to realise just how bad this analogy is, but alas I can't find a better one, let me know if you do.

When you are strong enough to shoot arrows that consistently beat the forcefield then your entry edge is doing it's job, it's effectively getting you in the market, beating the spread and hopeful reducing the chance of short term noise stopping you out.

When you have an arrow in the air (the higher the better) your parachute (exit edge) should kick in to improve your chances of letting the wind carry your arrow far away. In our case the edge in our exit via trailing stop loss is what tries to keep the arrow in the air as long as possible. Now in many cases there is no wind (i.e. no trend), no matter how good our exit edge is it will likely not be able to keep our arrow in the air very long. But on the odd occasion where a good gust comes along then our parachute does a great job of catching the wind (our exit edge does a great job of keeping our trade alive) and the further the wind can carry the arrow away from our position then the more profit we make.

The job of the entry is to shoot as many arrow in the air as often as possible with as much strength as possible - in an effort to get arrow above the forcefield. Once in the air our exit edge tries to catch a good few gusts of wind and we pray


Wishing you lots of wind

mim2005
  • #45
  • Sep 27, 2013 4:18pm Sep 27, 2013 4:18pm
Quoting FXEZ
{quote} I agree that it is not the quantity of backtests that are run that gives them meaning. Proper backtesting procedures aren't followed by the vast majority of traders. The problem is compounded by using deceptive tools such as the MT4 backtester. And thus those traders get poor results going forward. Overfitting is the most common culprit. It is very easy to do if you are optimizing a parameter that doesn't lead to consistency of results. Optimizing consistency generally solves this problem. There are ways to prevent or mitigate overfitting...

Hi FXEZ, from your posts i can see you have given systematic trading a lot of thougths. I work in this field and totally agree with all the pitfalls you have pointed here.

However there is another form of bias comes in that is very hard to avoid even by someone who is trying to be totally honest about their backtests and not cherry picking parameters/setups etc. This is due to the fact that after testing say 50 different strategies, naturally some will come up good on unseen data. Now as a systematic trader you will reject the ones that didnt work on unseen data and think the ones that worked well on unseen data must have some genuine edge. So this selection bias at strategy level itself can lead to data mining.
Think : Trade : Live Life
  • #46
  • Edited 10:11pm Sep 27, 2013 6:02pm | Edited 10:11pm
Quoting stt
{quote} Hi FXEZ, from your posts i can see you have given systematic trading a lot of thougths. I work in this field and totally agree with all the pitfalls you have pointed here. However there is another form of bias comes in that is very hard to avoid even by someone who is trying to be totally honest about their backtests and not cherry picking parameters/setups etc. This is due to the fact that after testing say 50 different strategies, naturally some will come up good on unseen data. Now as a systematic trader you will reject the ones that...
Hi stt,

I agree with your points regarding selection bias. This is one reason why system development is no trivial matter. Manual selection methods, I think, are especially susceptible to the introduction of selection bias. This is the reason I prefer automating the selection process. I think if you can design a framework for testing that provides good results on unseen data consistently, then some of the bias can be eliminated. It is a balancing act between having an appropriate fit of your strategy to the market, vs. identifying anomalous variables / systems that are masquerading as highly profitable systems because you turned your out of sample period into in-sample through trial, error and selection.
  • #47
  • Sep 28, 2013 3:52am Sep 28, 2013 3:52am
Quoting FXEZ
{quote} I agree that it is not the quantity of backtests that are run that gives them meaning. Proper backtesting procedures aren't followed by the vast majority of traders. The problem is compounded by using deceptive tools such as the MT4 backtester. And thus those traders get poor results going forward. Overfitting is the most common culprit. It is very easy to do if you are optimizing a parameter that doesn't lead to consistency of results. Optimizing consistency generally solves this problem. There are ways to prevent or mitigate overfitting...
Hi FXEZ,

There's not much to say except that I pretty much agree with everything you've said in your post. It sounds like you've spent some time with back testing strategies.

Thanks for the contribution. Look forwards to hearing more from you.

Take Care,

mim2005
  • #48
  • Sep 28, 2013 3:56am Sep 28, 2013 3:56am
good stats, keep moving
I have applied for myfxbook signal providing list!!! wish me luck
  • #49
  • Sep 28, 2013 10:02am Sep 28, 2013 10:02am
Excellent brewing of ideas here

@ mim2005 - Pretty much agree with everything you said on your last post. The idea of using different edges for entries and exits is a shiny pearl. That's something some systems I trade use, but didn't really gave much thought about until you've pointed it up.

How much one can rely on the predictive value of a backtest is the 1 million dollar question which I haven't found a definite answer for yet. Developing a profitable EA from nothing and getting proof that there's a 'real' edge to it is a time consuming mammoth task. Consequently my usual approach tends to be less original and more conservative/practical. I take a strategy which I trust has edge, because it has proven itself live for some time not too long ago. Then find the best parameters for the last 1, 2 years considering the number of trades generated. Then backtest using 1, 2 years prior. Assuming a satisfactory match between optimization and backtest, I'm ready to go live with small lotsize. Finally, assuming backtest and live test also match satisfactorily, I'm ready to increase lotsize.
"To hold, you must first open your hand. Let go." - Lao Tzu
  • #50
  • Sep 28, 2013 10:06am Sep 28, 2013 10:06am
Hi mim2005, and other fellows.

mim2005, that you so much for the further "wise" insight. I have two comments-questions if you would like going on sharing your views:

1. In a previous post you told us abot your strategies going on adding entries (even in the opposite direction) as new signals come in. As far as I understood there is no link between the exit mechanism and the entry, so it is well possible that new trades are added while the existing had not yet reached a riskless situation. May I ask you whether there is some mechanism to keep risk under a certain level? Or otherwise which is the range of exposure you have recorded in your backtests?

2. The idea of including an exit edge is also very useful. But, while you can esaily measure and calculate entry edge (a form of mathematical expectancy derived from MFE/MAE ratios, as a function of time after entry) how do you measure exit edge? This could be very interesting as a methodological approach, beyond your scripts.

To FXEZ and stt, I would just remind that there are out there sound statistical tests for calculating the probability of overfitting of backtested strategies thorugh numerical methods. So, just an impression, it might also be possible to make it without the need of automatic systems selection over cycles of IS/OS.

Bye

BL
πάντα δοκιμάζετε τὸ καλὸν κατέχετε
  • #51
  • Edited Sep 30, 2013 12:12am Sep 29, 2013 8:18pm | Edited Sep 30, 2013 12:12am
Hi all ... I've noticed this morning there are a couple of good post over the weekend and some very good question ... I will revisit them and give my 2 cents in the next day or so ... but for now ....

SO ... WHAT ARE THE PROBLEMS WITH THE ROBOT EXPLORER STRATEGIES?

It's worthwhile reflecting and being honest about some of the pitfalls that the robots (EAs shown earlier) have. Acknowledging the problems represent the first step in taking action to fix them.

So here goes ...

1) Disproportional reliance on one edge: Across the three EAs we have 4 completely distinct edges being applied - 3 intra-day entry edges and one common exit edge designed for trend following. This means holistically the 4th edge accounts for 3/6 or 50% of the systems profitability compared to ~17% for each of the other edges. In other words the system is excessively susceptible to poor performance in ranging conditions.

This could be diversified by using different exit edges which focus more on short term price movement rather than catching long trends: Definitely doable.

2) Reliance on Trends: They do not make use of possible edges in adapting to ranging markets. Price often spend considerable time in ranging markets and the systems do not cater for this.

There are trend/range detection methods that could be introduced to alleviate this somewhat - something I have already worked on but yet haven't had time to introduce.

3) Single currency risk: The EAs are all based on EURUSD and hence their is a high degree of correlation in their results. The high correlation means that the maximum drawdown experienced by each could be significantly additive and hence the drawdown will be significantly worse than the individual EAs - we're talking greater than 40% which is not psychologically sustainable for me.

Two of the three edges can be applied across multiple currencies and this sort of diversification (provided the pairs are not too correlated) should definitely improve results and gain the additional benefit of increased frequency. Alas another thing on my list for later.

4) The edges may simply not last: I haven't written about my thoughts on edges but it's clear to most that different types of edges are more robust than others. The search for edges is a little like searching for gold... sometimes you find real gold but often you find fools gold. There's no doubt that edges in these EAs face the risk of being fools gold. This is a whole topic in itself.

As mentioned by some of the other contributors there are thoughts and approaches that attempt to quantify the robustness of an edge. It's out of the scope of what I'll probably write about, I'll leave it at that.

5) Edges may not be real edges - they may be the results of curve fitting and selection bias etc. (similar to above point): This has been raised by many and I agree it's a significant risk. There are some approaches and diligence that can be used to minimize this risk (and I have used some) but overall it remains a high risk.

6) I'm clearly not giving these EAs the room that the stats tell me they will need to move: Although I know that these EAs could easily experience a 40% drawdown or more, you'll note that I've only given them a 20% breathing room (i.e. I've placed a failsafe account balance at $8k), there is no reason or logic here, in theory I should either halve the percentage risk and hence halve the drawdown % and profit % or I should increase the failsafe.

My bad. This is an example of my emotion clouding my logic or possibly the gambler in me.

7) Defects in the EAs: As with all development there is a good chance of the EAs having defects. I haven't had the EA code peer reviewed as yet and this risk is not substantially mitigated. In fact I have found one, possible two defect in the last week of observing (limit orders not being expired till much later than required). Unfortunately with limited time this is a risk as I won't have the time to debug for a week or two. It's a risk I am taking for now.

I'm sure there are a lot more problems I haven't listed.

Take Care,

mim2005
  • #52
  • Sep 29, 2013 11:55pm Sep 29, 2013 11:55pm
Quoting vox dei
Excellent brewing of ideas here @ mim2005 - Pretty much agree with everything you said on your last post. The idea of using different edges for entries and exits is a shiny pearl. That's something some systems I trade use, but didn't really gave much thought about until you've pointed it up. How much one can rely on the predictive value of a backtest is the 1 million dollar question which I haven't found a definite answer for yet. Developing a profitable EA from nothing and getting proof that there's a 'real' edge to it is a time consuming...
Hi Vox dei,

Thanks for your post. Yes I agree with you in regards to the 'million dollar' question, and I have some thoughts regarding the type/quality of edges that I'm keen to discuss, but it's probably worth a dedicated post, maybe a little later in the week.

I can relate to the approach you have to finding edges, I've used a similar approach on a few occasions. There's a couple of other approaches I've also tried that I'll share a little later.

Cheers,

mim2005
  • #53
  • Edited 2:54am Sep 30, 2013 1:34am | Edited 2:54am
Quoting betterlife
Hi mim2005, and other fellows. mim2005, that you so much for the further "wise" insight. I have two comments-questions if you would like going on sharing your views: 1. In a previous post you told us abot your strategies going on adding entries (even in the opposite direction) as new signals come in. As far as I understood there is no link between the exit mechanism and the entry, so it is well possible that new trades are added while the existing had not yet reached a riskless situation. May I ask you whether there is some mechanism to keep risk...
Hi Betterlife,

When I initially started using the approaches in these EAs I was worried about how many positions I had open at any one time and what the risk involved would be, in fact I put limits within the EAs to not open more positions once a certain number had been reached (although I took them out later). Then after watching the EAs working real time in a demo account for a few weeks I noticed that the positions would tend to build up a lot more over ranging conditions but the positions would build up in both directions more or less evenly - hence the net effect was that the net position was not particularly high. When the market starts moving a lot in one direction then a lot of the positions are closed out for a loss, but at the same time the winning positions often move towards covering some of their position.

Having said all of this I agree with you that there is a risk here, essentially after a long ranging period and subsequent break out you will be taking a bigger hit in SLs on the move and on the other hand your position in the break out direction will also be a bigger bigger size than normal. I guess if I was very worried I could probably simply close two hedged positions provided they are on similar lot sizes, or have the EA do this itself. Good point though, it's definitely a risk.

In regards to measuring the exit edge, alas I have used a fairly primitive approach. I effectively work out how much edge the exits have by deduction. When developing each entry edge I utilised some very simple edxits without edge e.g. Exit at 1:1 RR, exit 2:1 RR, exit 3:1 RR. This provided me with a good idea on what the ideal exit would be ... I'm glad to say that all of the systems are profitable without using any exit edge (although much less so).

I then worked on more sophisticated exit edges which I was able to determine improved the profit factor quite significantly on all three different entry edges. Hence by deduction I assumed the exit had an edge. I'd like to give you something a little more elegant than this but I don't have anything. I guess if I simply wanted to isolate the exit edge I would simply create a neutral entry (e.g. always long, or always short, or random) and measure the profitability of the exists.

Great questions!

Take care

mim2005
  • #54
  • Sep 30, 2013 10:09pm Sep 30, 2013 10:09pm
Quoting mim2005
{quote} Hi Vox dei, Thanks for your post. Yes I agree with you in regards to the 'million dollar' question, and I have some thoughts regarding the type/quality of edges that I'm keen to discuss, but it's probably worth a dedicated post, maybe a little later in the week. I can relate to the approach you have to finding edges, I've used a similar approach on a few occasions. There's a couple of other approaches I've also tried that I'll share a little later. Cheers, mim2005
Hi mim2005,

I'm very interested in hearing your views on detecting/measuring edges.

Concerning your thrice used exit edge (trailing profit), I think it is the simplest and possibly the perfect exit solution in a trending market as setting a take profit in such market would likely end up capping the profits. Like you've said all 3 EAs results are optimal assuming ongoing trend. However, considering that roughly two thirds of the time the market is ranging, using a volatility based take profit as exit edge might be beneficial. My suggestion then would be to use these 2 exit strategies in a complementary way as the resulting equity curve should be smoother (and still ascending ).

Regards,
d
"To hold, you must first open your hand. Let go." - Lao Tzu
  • #55
  • Sep 30, 2013 10:14pm Sep 30, 2013 10:14pm
Quoting betterlife
To FXEZ and stt, I would just remind that there are out there sound statistical tests for calculating the probability of overfitting of backtested strategies thorugh numerical methods. So, just an impression, it might also be possible to make it without the need of automatic systems selection over cycles of IS/OS. Bye BL
Thanks for that reminder, betterlife. I think you might be referring to the tests referenced in Evidence Based Technical Analysis by Aronson. Please correct my citation if you are referring to another source.

From the source cited above I've tested in some detail the White's reality check (WRC) bootstrapping procedure. I did not try the referenced Monte Carlo method. I found that WRC was weak: eliminating some "good" strategies, and allowing through many "bad" strategies that hurt future performance. I think this quote by Lord Ernest Rutherford sums it up:

Quote
If you need to use statistics, then you should design a better experiment.
After reading this quote I started designing better experiments and found a great deal of improvement in my testing results, particularly how well strategies generalize onto unseen data.
  • #56
  • Edited 3:41pm Oct 1, 2013 11:48am | Edited 3:41pm
Quoting mim2005
Hi all ... I've noticed this morning there are a couple of good post over the weekend and some very good question ... I will revisit them and give my 2 cents in the next day or so ... but for now .... SO ... WHAT ARE THE PROBLEMS WITH THE ROBOT EXPLORER STRATEGIES?
1) Disproportional reliance on one edge: Across the three EAs we have 4 completely distinct edges being applied - 3 intra-day entry edges and one common exit edge designed for trend following. This means holistically the 4th edge accounts for 3/6 or 50% of the systems profitability compared to ~17% for each of the other edges. In other words the system is excessively susceptible to poor performance in ranging conditions.


This could be diversified by using different exit edges which focus more on short term price movement rather than catching long trends: Definitely doable.

2) Reliance on Trends: They do not make use of possible edges in adapting to ranging markets. Price often spend considerable time in ranging markets and the systems do not cater for this.

There are trend/range detection methods that could be introduced to alleviate this somewhat - something I have already worked on but yet haven't had time to introduce.

3) Single currency risk: The EAs are all based on EURUSD and hence their is a high degree of correlation in their results. The high correlation means that the maximum drawdown experienced by each could be significantly additive and hence the drawdown will be significantly worse than the individual EAs - we're talking greater than 40% which is not psychologically sustainable for me.

Two of the three edges can be applied across multiple currencies and this sort of diversification (provided the pairs are not too correlated) should definitely improve results and gain the additional benefit of increased frequency. Alas another thing on my list for later.

Mim2005, (and anybody who is following this thread)

In regard to your above points:

The EAs have held their ground fairly well, considering EU's marked sideways movement over the last few days (see first attached screenshot) — which, if we examine EU historically, is about as sideways as it gets. My observation (from analyzing the trades in the Trade Explorer) is that the entries have been particularly good, allowing many of the trades to attain profit (>1RR in some cases), before they are stopped out for a 1R loss, as the sideways movement returns price to the original SL.

Hence your idea of detecting whether the market is trending or ranging, and (if the latter) possibly advancing the SL a little earlier, to make the losses smaller, might well be worth a try.

The 'tapering' meeting apparently caused the two profitable trades which helped to kickstart the account (candle 'C'). It's also interesting how candle D's upper wick was a high enough push to let the three most recent trades exit with ~ 1RR profit. This suggests that any moves along the lines of candle A thru B, or B thru C, should result in significant profit, which is very promising. But of course we'll have to wait and see.

In the second screenshot, I note the intense positive correlation of EUR (pink curve) and USD (white curve) on the D1 plot, over the last few days (dotted line enclosure). Hence the sideways movement on EU. I'm going to perform a daily correlation check, and try running the EAs (or at least the the two whose entries allow them to be run on pairs other than EU) by manually switching entries on, on the charts of the most negatively correlated pairs (currently GBP and CAD — resulting in their recent daily 'trend' in the third screenshot) on a demo account, and I'll also consider posting a Trade Explorer. Of course this meddling will result in a process that's not fully automated, but over time it may lead to the formulation of mechanical rules that could be incorporated in the EAs. Just as with the EU trend durations, the key will be how long the negative correlations hold good, and how quickly the strength plots are able to identify changes in correlation.

Just some random thoughts, at this early stage.

David
Attached Image(s) (click to enlarge)
Click to Enlarge Name: Image 110.png Size: 19 KB
Click to Enlarge Name: Image 111.png Size: 18 KB
Click to Enlarge Name: Image 112.png Size: 23 KB
  • #57
  • Oct 1, 2013 11:53pm Oct 1, 2013 11:53pm
Quoting FXEZ
{quote} Thanks for that reminder, betterlife. I think you might be referring to the tests referenced in Evidence Based Technical Analysis by Aronson. Please correct my citation if you are referring to another source. From the source cited above I've tested in some detail the White's reality check (WRC) bootstrapping procedure. I did not try the referenced Monte Carlo method. I found that WRC was weak: eliminating some "good" strategies, and allowing through many "bad" strategies that hurt future performance. I think this quote by Lord Ernest...
Hallo FXEZ,
sorry for the delay and the very limited time now.

No, I refer to the very recent research of Lopez de Prado on combinatorially symmetric cross-validation.

Best regards

BL
πάντα δοκιμάζετε τὸ καλὸν κατέχετε
  • #58
  • Oct 2, 2013 12:12am Oct 2, 2013 12:12am
Quoting mim2005
Great questions! Take care mim2005
Hallo mim,

Great reply! Thanks so much. A lot of food for thinking!

Bye

BL
πάντα δοκιμάζετε τὸ καλὸν κατέχετε
  • #59
  • Oct 2, 2013 1:53am Oct 2, 2013 1:53am
Quoting hanover
In the second screenshot, I note the intense positive correlation of EUR (pink curve) and USD (white curve) on the D1 plot, over the last few days (dotted line enclosure). Hence the sideways movement on EU. I'm going to perform a daily correlation check, and try running the EAs (or at least the the two whose entries allow them to be run on pairs other than EU) by manually switching entries on, on the charts of the most negatively correlated pairs...

Hi David and others,

not trying to hijack any thread or anything, just trying to talk/discuss about how to include indexes into decision making, plus, how one could distinguish potential range and potential daily trend formation in a making. My index is a bit derived from your excellent experiance done with a strenght meter. Cheers for that btw.


"Filtering currency pairs and entries"

From your words one could put a filter on the currency pair. Im doing statistical work on the 21 pairs I trade and also EURCHF pair which by each currency is highly positively correlated (picture1) is showing some good results lately. (thats because Im not targeting 500/1000 pippers). Still, this pair should be turned off if one wants to play a trend scenarios, mainly to limit the unnecassary risk on the table and to pick only high probability trades.


But how to do this? Positively correlated pairs are flattened in their index values. To determine how to intervene on particular pair, one needs values where it would stop trading that pair. How one would pick that? (a difference between them is less than xx in value= no trade) I what time series? We pick average?
What could be the difference between two indexes that one could say that we are establishing new trend.. So we allow the calculation of entries on particular pair.


What could determine a range after a trend in the index flattening?

I guess one again need a series of index values and try to give a statistical "opinion" on that issue.


Agreed about GBPCAD pair (picture2). It started on 4th of August and then we started to see some strenght in this pair. What was your date? The system recognized that both currencies were negatively correlated and we also got a follow thru reaction which resolved in this daily trend.

As picture shows Indexes are pretty much way apart and we only have some 4h and 1h consolidation phases. (my indexes have reverse value-so negative is strong).
This one seems it wont stop yet and we could continue while GBP is trying to do another leg up.



"..and how quickly the strength plots are able to identify changes in correlation"

To plot a change and potential consolidation when we are already inside established trend one could use a higher speed indexes. Meaning lower time series values which could give some kind of detection of a pause.
If this happens one could use this data to add on a strong trend OR to limit the risk and take partial profits and update SL levels. Again, its the problem how to determine when to add and when one would decide that a strong consolidation and strong retracement is gonna occur.


Any opinion and directions appreciated.
Attached Image(s) (click to enlarge)
Click to Enlarge Name: eurchfdaily.png Size: 81 KB
Click to Enlarge Name: gbpcaddaily.png Size: 84 KB
  • #60
  • Oct 2, 2013 2:54am Oct 2, 2013 2:54am
CONCEPTS THAT MADE A SIGNIFICANT DIFFERENCE FOR ME IN THE DEVELOPMENT OF EAs AND STRATEGIES

You can probably see from the notes in my earlier post that when I started working on forex I pretty much back tested all the traditional technical analysis concepts I could find. After a significant amount of work I was able to develop a few back test profitable strategies but the profit factors were relatively low and I didn't have confidence that the systems where truly profitable and not just curve fitted.

It's at this point that I pretty much decided to discard most of the traditional technical analysis teachings I had spent so much time learning. I am now fairly firmly of the belief that most of the typical stuff (such as candle patterns, looking for confirmation, MAs, Stochastics or other O/O, don't pick bottoms and tops etc.) offer little if no edge. I know others may disagree, but thats what I found in my own studies.

So ... A) I don't use traditional TA in any of my systems.


After reading several dozens of posts on forexfactory and testing various strategies I started noticing that some of the more experienced and presumably profitable traders paid a lot of attention to specific times within certain sessions. Now this isn't anything new, but it introduced me to the concept of looking for certain edges at certain times of day.

Unfortunately due to daylight savings it was very difficult for me to properly test specific strategies over several (5+) years which incorporated session knowledge. So I decided to bite the bullet and develop custom libraries that were able to determine the exact session time windows using timezone and daylight saving historical data over the last 10 years. I will not discuss or share how this was done but I know anyone serious about it can do it if they put in the effort. This makes it possible for me to back test strategies that could take into account sessions over several years of historical data across multiple sessions. On reflection this is probably the one most important concept I was able to use to develop profitable strategies.


So ... B) I always consider session times in everything I do & specifically when looking for edges. I only back test using accurate session time data (DST etc.) as a backdrop.


Finally, the other significant change for me was when I started to really analyse price movement within different sessions and imagine the types of things traders working for institutions were trying to achieve. Traders working for large institutions have objectives to achieve and also being people working typical hours they also tend to do things at certain times of day. So you need to pay attention to movements leading up to a session, what tends to happen in the mornings, what types of activity happen at lunch, where are the important levels for the day and when do these have a chance for being hit etc. This made me look at price action like a story as opposed to what candle formations are showing. I spent a massive amount of time looking at session charts and getting a feel for what happens in a session and why. This is not something you'll find a lot of information on the internet about.

So ... C) Understand the traders working for the institutions and their objectives during the session.


It's only when I started to absorb the points above that I was able to build much more profitable EAs, and in fact all of the edges (except one) used by the EAs were discover in a two week period after having an "aha" moment in regards to what to look for. Previously I was looking for edges in all the wrong places. Whilst now I feel I have a much better understanding as to the type of things to look for when it comes to the searching for edges in the future.

In regards to the edges exploited by the EAs, I feel I have a reasonably firm understanding as to why each edge works i.e. I can explain them in terms of institutional traders, session times and underlying orders. Note: As with all things forex I may be deluding myself here ... but my instinct tells me theres a good chance I'm not.

There are a few other points that also made a difference but not as much as the above, so I'll leave them out for now except to mention that you need to have a good feel for where the underlying orders are in the market.


I can honestly say that if I had understood the above points much sooner in my journey I would have saved myself a lot of time, so I hope it help others that are at a similar point in their journey.




Take Care,

mim2005
0 traders viewing now