Menu

Options

Bookmark Thread

First Page First Unread Last Page Last Post

Print Thread

Similar Threads

Technical Analysis Fallacy 63,555 replies

The Ultimate Fallacy in Technical Analysis 99 replies

The fallacy of Metatrader EA trading contests 19 replies

Statistical analysis fallacy 33 replies

Attachments: Retail Sentiment Fallacy?
Exit Attachments

Retail Sentiment Fallacy?

  • Post #1
  • Quote
  • First Post: Edited 9:44am Feb 13, 2020 8:51am | Edited 9:44am
I've recently been interested in using retail sentiment as a verification before I take trades because as most people know, a large majority of traders (90-95%?) lose money. Most sites/videos obviously then say to take a contrarian view of retail sentiment because of that stat, however when I observe the relationship between price and percent of retail traders net long/short as shown in this DailyFX article/chart: https://www.dailyfx.com/forex/techni...002040923.html , it seems the relationship is direct. When the buys are at extremes, 75%+, price goes up and vice versa for selling. Been staring at this chart trying to figure out if I'm just stupid and missing something glaringly obvious. There is a clear relationship between retail sentiment and price but it does not seem to be contrarian as expected. Also to be noted that this isn't being used as the sole determinant of a trading strategy, but more to confirm price action at key levels.

My only theory is that, for example, when retail has reached an extreme amount of buying that it is likely NOT position BUILDING, but position CLOSING from retail; they have been shaken out from shorting any more or there has been massive profit taking and this imbalance between a lack of selling leads to price going up. Of course one could say retail has a very insignificant impact on the overall movement of price but the relationship as shown below and many other examples can't be ignored.
Attached Image (click to enlarge)
Click to Enlarge Name: chart-USDCAD-202002040923__en.png Size: 37 KB
I spent a good year with retail sentiment (and wrote a thread about it years back), using hourly data feeds so I know my way around the whys and wherefores.

you need stats, rather than eyeballs on something this big. *Generally* speaking, it IS a contrarian indicator but only at the extremes. It all boils down to a particular way of trading that all amateurs undertake that gives you the curve. Typically, retailers will range trade; this is borne out by stats of success that show asian session trading is way more successful for retailers than us/uk. But ranges of course break, and when they do, they'll hold on to their losers, cut their winners short, and fight the trend, which gives you the extremes of positioning. So yes, you're theory is part right, but only in part.

You can, with a fuck ton of effort, use extreme retail positioning as a fairly reliable turning point; however it's the exceptions that prove the rule, when a megatrend comes along, and the extremes can last for days, rather than hours so any system you put in place needs to use losing trades as a marker of a potential megatrend, and use time as a way of filtering so you don't stand in front of the steam train that comes along all too regularly.

Your chart is based on dailies; most retailers are not swing trading for days, so it's not all that useful. You need as frequent as possible - FF have their own, Oanda used to provide hourly, FXCM have a realtime feed I believe.

If you add in session data (thinking of where dealers will be long/short according to time of day) you can construct a system that'll outperform all the FF system threads at the very least, so it's not time wasted.
  • Post #3
  • Quote
  • Edited 11:12am Feb 13, 2020 10:57am | Edited 11:12am
I use a combination of several services to get a sum. Here is my example for friday. For example, in USDCAD 16,8% of retail traders have been long (on friday, 7th of february).

Would be nice to get this automated, at the moment I use it to confirm / exclude my bot from trading signals.

Maybe anyone is interested in helping me to automatically grab that data?
Attached Image (click to enlarge)
Click to Enlarge Name: sentiment.png Size: 29 KB
  • Post #4
  • Quote
  • Edited 8:01pm Feb 13, 2020 3:11pm | Edited 8:01pm
Quoting triphop
I spent a good year with retail sentiment (and wrote a thread about it years back), using hourly data feeds so I know my way around the whys and wherefores. you need stats, rather than eyeballs on something this big. *Generally* speaking, it IS a contrarian indicator but only at the extremes. It all boils down to a particular way of trading that all amateurs undertake that gives you the curve. Typically, retailers will range trade; this is borne out by stats of success that show asian session trading is way more successful for retailers than us/uk....
Appreciate the insight. My original idea/strategy was to use sentiment as a contrarian indicator as you said, waiting for the extremes. Hence why I posed the original question, because that theory seemed to be false when you look at higher time frames. You are right however that sentiment and how we understand it is a matter of the time period for which we are aggregating, i.e., session aggregation, overall open positions, etc.

My goals and strategy for trading has over the years shifted from trying to predict movement on an hourly or intraday basis to trying to predict major flows based on liquidity/volume which ultimately determine price movement on the smaller time frames. I.e., movement during the day is just noise based on the decisions based on higher time frames, nothing conceptually new, but the major reason why I focus on sentiment in conjunction with higher time frames. The major loading and unloading of shares, currency, etc is what moves price and as we know, banks and big players need sufficient liquidity (hence counter trading retail because they become the liquidity) to do so. You point out that while extreme sentiment can be a good indicator for trading reversals that it isn't perfect in the case of a 'megatrend,' which I would agree.

My solution to that 'exception to the rule' would be to look for extremes in price/sentiment that are approaching a support/resistance that coincides with an absence or gap in liquidity, which can be represented by a ledge on a volume profile. For a short set up, the logic/theory would be: Price is approaching a well known resistance, sentiment is high, lots of buy orders are open. The obvious thing to do would be to place a sell at that resistance but how would you know that a bank/large player wouldn't keep driving the price higher to get more buy orders open to fill their big sell position? Well if that resistance level also coincides with a sudden drop off in liquidity it doesn't make sense for them to do so. Without sufficient liquidity above the resistance, 'pushing price up' , 'shake outs,' or 'manipulating the market,' becomes insanely expensive because there isn't adequate volume to do so and any effort to keep pushing the market to extremes leads to large losses via slippage. So now we have lots and lots of retail buyers at a point where there is a lack of liquidity, large institutions/banks looking to get their sell orders filled who can't push price any higher also because of lack of liquidity therefore those institutional orders should get filled, making for a good short set up. Of course this is all theoretical and I can only presume the operations of banks/institutions based on my knowledge of market dynamics and a culmination of reading/watching how bankers fill their orders. I'm sure you've studied sentiment more than I have so if there's an achilles heal (as their most likely is) I'd appreciate the insight or flaws in my ideas.

Quoting thj79
I use a combination of several services to get a sum. Here is my example for friday. For example, in USDCAD 16,8% of retail traders have been long (on friday, 7th of february). Would be nice to get this automated, at the moment I use it to confirm / exclude my bot from trading signals. Maybe anyone is interested in helping me to automatically grab that data? {image}
This might help you, you can also order the broker by weight (assuming volume?).

https://fxssi.com/tools/current-ratio?filter=USDCHF
Thank you. I know fxssi, but got the feeling it is not accurate all the time. But still, it might just work. I don't get the weight though, myfxbook is rated very strong.

But maybe it is a good central source for grabbing the data, I will play around with PHP and try to extract it.
Quoting Nghtmre
a large majority of traders (90-95%?) lose money.
My thinking is most traders don't lose over the long-term because they are positioned "incorrectly" in the market, but because of mindset i.e. overtrading, revenge trading, lack of confidence in their strategy during drawdowns and/or poor money management.
Quoting Nghtmre
{quote} Appreciate the insight. My original idea/strategy was to use sentiment as a contrarian indicator as you said, waiting for the extremes. Hence why I posed the original question, because that theory seemed to be false when you look at higher time frames. You are right however that sentiment and how we understand it is a matter of the time period for which we are aggregating, i.e., session aggregation, overall open positions, etc. My goals and strategy for trading has over the years shifted from trying to predict movement on an hourly or intraday...
OK, so everything that you say *sounds* sensible, but does the data really hold up to scrutiny? That chart you posted is actually more or less what I found; price is highly inversely correlated to retail positioning on higher TFs, but not predictive in any way. So correlation <> causation as they say. It's more or less an inverse moving average except on lower TFs where it has some value from my research.

If you can find value in longer term retail positioning and have absolute confidence, the system should stand up by itself.

Adding in a volume is whole other game. If you're using tick volume as a proxy for real volume as per threads on here, it's again tightly correlated but not predictive. you'll be better off studying the passage of time; getting absolute metrics on the speed of movement. And tying those into sessions. if you're using volume from CME, well as they're not market movers so it's akin to using retail positioning. If you really want to get your hands dirty, you need proper order flow - option data from DTCC is your way forward. as order flow is only as good as the data you're using. The threads discussing order flow in FF are generally educated guesswork based on price action, which is well... price action and nothing to do with actual order flow.

get on it!
Quoting ChilledT
{quote} My thinking is most traders don't lose over the long-term because they are positioned "incorrectly" in the market, but because of mindset i.e. overtrading, revenge trading, lack of confidence in their strategy during drawdowns and/or poor money management.
The mindset just determines how fast they lose. If they didn't bet large, they just lose by the speed of the spread. the idea that all these retailers are using sensible strategies hampered only by their own psychology is broker and trading guru bullshit spread around by those looking to rinse the newcomers. not a popular opinion I know.
Quoting triphop
{quote} OK, so everything that you say *sounds* sensible, but does the data really hold up to scrutiny? That chart you posted is actually more or less what I found; price is highly inversely correlated to retail positioning on higher TFs, but not predictive in any way. So correlation <> causation as they say. It's more or less an inverse moving average except on lower TFs where it has some value from my research. If you can find value in longer term retail positioning and have absolute confidence, the system should stand up by itself. Adding in...
Speed would be an interesting metric to look at because speed of a movement is related to liquidity of that movement. I.e, if price moves up extremely quickly it means that there wasn't many sell orders below to slow down the move, which should also make a reversal easier or less subject to downwards resistance because it also means that the buy orders to make that previous move happen were also not as large. This is often why if you look at a chart the fastest reversals come after the fastest push up or down. But yes, that's probably an easier metric to use than getting true volume for forex pairs. Thanks again for the insight.
  • Post #10
  • Quote
  • Edited Feb 15, 2020 10:35am Feb 14, 2020 3:25pm | Edited Feb 15, 2020 10:35am
Quoting ChilledT
{quote} My thinking is most traders don't lose over the long-term because they are positioned "incorrectly" in the market, but because of mindset i.e. overtrading, revenge trading, lack of confidence in their strategy during drawdowns and/or poor money management.
I think that's true that most retail traders can pick overall direction or trends fairly well, however, like you said, money management and psychology is most likely the determinant of the loss. But bad money management by retail traders is what provides liquidity to big traders. Most retail traders are sold systems that involve trading some kind of retrace, or getting back into an overall trend and to have tight stops to keep their risk : reward high. These things aren't necessarily bad practices however none of these strategies talk about market dynamics at all. Why does price have to turn at this fib level, or support level? How do we know price will turn at this support level and not that one? In which cases does is the level not valid and price bursts through, and I think that's where retail sentiment and thinking about volume / liquidity comes in place. Those things are the key determinants for price movement. I think quite frankly that some strategies such as support/resistance work but only when liquidity/volume verify them as I tried to explain before. I mean liquidity is THE THING; if a bank can't fill their order they don't enter, so they wait (or manipulate) until they can force a market to provide them that liquidity. It is impossible for 90+% of a retail market to be selling at a resistance and also have a bank sell at that position unless they want huge slippage or for another bank to see their stupidity and profit off it, so they force people to go long either by stopping them out or getting break out traders to enter. I mean big players have so much more access to information/data feeds, speed, interbank volumes, connections at other banks/institutions, it would be a difficult task for them to fool each other but a very easy one to fool retail traders.
I've studied sentiment at various times throughout my trading career and haven't managed to find anything reliably predictive. Its unfortunate because the theory behind it makes a lot of sense, I just haven't seen it useful in practice.

This is an area I'd like to reexamine so I'll be following along in this thread with interest.
Self-sufficiency is the greatest of all wealth. - Epicurus
I'm sure some of you guys have seen this guy's interviews on YouTube. Further emphasizing the point: "I want to put my entry where the masses put their stops."

Inserted Video
emotion .hmm
unknown.
because i have a system i just need to follow ..
so the rules are given.
  • Post #14
  • Quote
  • Edited 12:26pm Feb 15, 2020 12:11pm | Edited 12:26pm
This screenshot is of a level I was looking at 2 weeks ago which made for a really nice short, and has led to the Euro falling all week. I went back and applied FXSSI's sentiment indicator. I am currently just using the demo version so the data is only for a month. Note that these sentiment indicators provide data for OPEN positions, that is critical for how we interpret sentiment. A sell that CLOSES a buy position DOES NOT count as an open sell position. I noticed something that confirms both ideas discussed. Sentiment / open positions mean different things at different times: At an entry point vs during a trend.

1 - At Entry Point: Before a big move happens (swing trade) there is a HUGE swing in the % of buy/sell open positions. As seen there was a rapid fall in relative open buy orders and a rapid influx of sell orders. Sentiment changed very quickly. This is the case where sentiment is NOT a contrarian indicator but rather a confirmation of a level. This also follows what the first chart I posted represents, there is a large amount of open sell orders before price goes down.

2. The Trend: This confirms our original idea that retail traders are wrong because as price kept moving downwards, the number of open long orders went UP. This is most likely because for price to move down there needs to be traders that attempt to catch a falling knife. If there weren't naive traders to think price would go up then price couldn't go down, basic market dynamics, for every seller there needs to be a buyer.

My conclusion: The extreme swing in sentiment / open orders is a result of a huge influx of sell orders, a large player (whale) or a large amount of traders entered at this point to sell (confirming that price movement is a result of high supply or demand), however after that entry point the sentiment goes back to being long and price continues to fall (confirming the concept that for price to move someone needs to be taking the losing side). So the way I take this is that I should look for spikes in sentiment/open positioning in the direction I think it will go before entering.

So just to reiterate how we would use this: For the EURUSD trade, retail was majority long (contrarian as price was moving down), but then at my point of ENTRY there was a huge spike and swing to majority of orders being open sells (sentiment confirming entry) then after the entry, sentiment went back to being contrarian as price fell and sentiment/open orders went back to being majority buy.
Attached Image (click to enlarge)
Click to Enlarge Name: 2651bf0e26fd573d35d9add66b1bc2ac.png Size: 35 KB
Quoting Nghtmre
Sentiment changed very quickly.
That sounds interesting, please, elaborate. How would quick be defined, is it a minute or hourly change thing? And what percentage we are talking about?

I was looking at EURAUD last weeks. Overall, long sentiment was about 20-something percent. So majority was short - but EURAUD was really dropping, so majority was right that time. What a bummer ;-)

Maybe volume of positions could filter such false signals out?

I also agree to your large player statement. But how to profit from that?
Quoting thj79
{quote} That sounds interesting, please, elaborate. How would quick be defined, is it a minute or hourly change thing? And what percentage we are talking about? I was looking at EURAUD last weeks. Overall, long sentiment was about 20-something percent. So majority was short - but EURAUD was really dropping, so majority was right that time. What a bummer ;-) Maybe volume of positions could filter such false signals out? I also agree to your large player statement. But how to profit from that?

The great thing about sentiment is that it is not time dependent. Sentiment just measures the amount of open buy and sell orders a broker has, measured in percentage. So 80% would read that 80% of a broker's book on that pair are open buy orders, 20% means that 80% are short. You will get the same reading on any time frame, it is independent of time. Sentiment cannot be used alone and in fact will probably blow your account if that's the sole metric you are using to trade. What I'm trying to do is examine the relationship between sentiment, the change in sentiment, and future price at certain levels. For example, support and resistance can be seen to 'work' however the filter for which support and resistance work is subjective and sometimes random.

I'm going to test the theory further but the way I make sense of it for now is: Sentiment is contrarian during a trend, and during a re-entry or reversal sentiment will spike up in the opposite direction to fill an order, then resume back to being contrarian. The easiest way I can explain how this works is to again use the example above.

- EURUSD is trending downwards. Retail sentiment is 80% long therefore sentiment is acting as a contrarian indicator as expected.
- If a big bank wanted to re-enter this trend at a support/resistance/retrace level, it would be a good time to enter because so many people are buying they can fulfill their short position.
- So if we are looking to re-enter a trade where big players are, or where there is a large influx of sell orders/supply, we should wait for the sentiment to spike down indicating a large influx of sell orders, sentiment is now acting as confirmation.
- After that big order is filled, the original sentiment will continue and hopefully the trend as well with majority of retail long as before, sentiment is again acting as contrarian.

That is what the chart I posted is outlining.
Found out that even if sentiment is 100% correct, it is still far from profitability unless one has a good AUM and just BUY/SELL the market OR trade LOW LEVERAGE and give trades ROOM to develop... But if one is almost obsessed with Risk:Reward 1:1+ and/or high-leverage.. sentiment might not much.. I have proven this last summer Gave people 5 months of 100% correct sentiment but I doubt anyone turned it into profit, at least that info alone, even very important one...
Can you afford to take that chance?
Quoting auricforecas
Found out that even if sentiment is 100% correct, it is still far from profitability unless one has a good AUM and just BUY/SELL the market OR trade LOW LEVERAGE and give trades ROOM to develop... But if one is almost obsessed with Risk:Reward 1:1+ and/or high-leverage.. sentiment might not much.. I have proven this last summer Gave people 5 months of 100% correct sentiment but I doubt anyone turned it into profit, at least that info alone, even very important one...
Using sentiment alone would make no sense because as you said, it doesn't favor having a good risk : reward, or at least a consistent one. That isn't being suggested here at least from my perspective. Again, the goal is to use sentiment in conjunction with whatever strategy we have, to fine tune entry points and decision making. For example, just this week I avoided entering a trade because the sentiment did not prove my direction, and it ended up being a trade to avoid.

The point I'm trying to make is that, how sentiment or how the 'book'/open orders changes for a broker provides more information that validates our entry. Your entry can be based on support and resistance, Fibonacci's, oscillators, histograms, whatever, but those things as we know are rarely profitable on their own, and require a filter for good signals. I mean go look at any strategy on this forum. There is an original idea with one or two major points, but then it ends up requiring the traders to look at other indicators, more filters, then to incorporate other ideas, looking at more time frames, etc. I think examining sentiment and plotting the sentiment of large brokers can be a great filter, instead of adding 10 other filters to a strategy. Every indicator is just a derivative of orders/volume and the resulting price. Why not examine the source?
  • Post #19
  • Quote
  • Edited Feb 16, 2020 12:32am Feb 15, 2020 8:26pm | Edited Feb 16, 2020 12:32am
For anyone that thinks there is no value in retail sentiment data from forex brokers with 10s of billions in volume, take a look at this article. Forex is constantly marketed as the largest market in the world, but that statement has a lot of asterisks. Only 3.5% of that roughly '6 trillion daily turnover' that is constantly quoted, goes to retail flow and only half of that flows to spot FX (the other half going to FX swaps), meaning the real volume of the forex market that us retail traders trade is around $75 billion TOTAL.

https://www.financemagnates.com/fore...-forex-market/
When retail sentiment goes into imbalance buy-sell it means that there is a strong trend in progress and that it has been going on for some time.
Sort of like an MA indicator.
Not because they are wrong in their prediction, but because they do not want to close losing positions, ie without SL
0 traders viewing now