The Finance Book Club

Yes. You know a lot about losses. That is the base. You can do it within the next two years. The coming two will be your hardest...

Also they only take the top candidates from elite schools who are already fairly well versed in the basics. 7 years sounds about right. Of course it's good to get paid while learning on the job.

The first 6.5 years you have to work your arse off and brown nose like your life depended on it and then some big shot trader quits or retires. Then you are up to replace him only to find out that the firm went and found a new graduate who knows nothing about trading and they have you train them for the position. That's usually how it works unless you are the children of the ones in charge in which case you can usurp yourself over anyone at any time.

"Here in the City we practise 'intelligent nepotism'. That means each managing director has a number of nephews, one of whom is intelligent and he is going to be the next managing director. If you came from the same college he did you would have a chance of getting a job at the firm. But you're not even from the same country!" - Lazard Frères to a young George Soros

Evidence-Based Technical Analysis
by David Aronson

'Apply the Scientific Method and Statistical Inference to Trading Signals'
This book is a heavyweight contender for the best trading book ever written though it won't likely win the title because it's both somewhat academic and it has a dispiriting, disillusioning conclusion based on what might be, ironically, insufficient evidence. However don't let that fool you - the info in this book is vital to understanding what's real and what's voodoo.

I note another irony - the opening blurb recommendation quote is by Victor Niederhoffer, who himself is a bit of a cautionary tale. However it's also blurbed by Jack Schwager, Perry Kaufman, Sandor Straus (Merfin), John Bollinger (yes of the bands fame) and Fred Gehm.

Introduction

1. As a research approach, technical analysis (TA from now on) has suffered because it is a "discipline" practiced without discipline
2. It must evolve into a rigorous science.
3. Over 2 decades some journals have approached it in an honest manner
4. This book establishes the method, philosophy and statistical foundations for Evidence-based TA (EBTA)
5. It then tests that approach on 25 years of historical data with 6400 binary buy/sell rules on the S&P 500 A solution to a question that only serves to open up a whole can of worms once we are done with the book.
6. Data mining is useful for finding rules and signals, however new statistical tests are needed to determine the validity of those rules, since historical performance is upwardly biased. At least in US equities.
7. The subjective , interpretive methods traditionally associated with TA is not useful
8. David Aronson is/was an adjunct professor at Baruch College where he teaches a graduate level course in TA. He is also a CMT, and was a prop trader and analyst for Spear Leeds & Kellogg. Founder of Raden Research Group. Founder of AdvoCom, a firm that evaluated commodity money managers performance.

The algo used to test the rules is promised over at www.evidencebasedta.com but I didn't find it, only PDF links to the test results.

According to Wicki: In a discussion at the Los Angeles World Affairs Council in 2006, Alvin Shuster, former foreign editor of the Los Angeles Times, asked Soros, "How does one go from an immigrant to a financier? ... When did you realize that you knew how to make money?" Soros replied, "Well, I had a variety of jobs and I ended up selling fancy goods on the seaside, souvenir shops, and I thought, that's really not what I was cut out to do. So, I wrote to every managing director in every merchant bank in London, got just one or two replies, and eventually that's how I got a job in a merchant bank."\



So, he got a job out of one of the 2 interviews he got. Not bad considering he had no previous experience. He got lucky from the get go.

And the job he got was with a company whose director was Hungarian like him. Yes, luck and national chauvinism have a huge role in life's opportunities even if we hate to admit it.

Preface

1. EBTA charts a course between the magical thinking of gullible true believers and the relentless doubt of random walkers
2. Science frequently conflicts with intuition - for example to us it looks like the sun circles the earth
3. Informal, intuitive approaches to knowledge is likely to result in erroneous beliefs when phenomena are highly complex or highly random, features of market behaviour
4. Scientific methods are no guarantee of success, but the opposite is almost sure to produce fool’s gold
5. Much of the wisdom of popular TA is not legitimate knowledge

Key Definitions: Propositions, claims, belief and knowledge

1. Knowledge:
   1. Fundamental building block of knowledge is a declarative statement aka claim, proposition
      1. Exclamations
      2. Questions
      3. Commands

      
      
      

   2. Declarative statements have truth value
      1. They can be characterized as either true/false or probably true/false

      
      
      

   3. “Oranges are on sale for five cents/dozen at the supermarket.” is declarative
      1. Makes a claim
      2. May be true/false but not both

      
      
      

   4. “Holy Cow, what a deal!”
      1. Not declarative
      2. Not falsifiable

      
      
      

   5. “Trading rule X has predictive power.”
      1. Declarative
      2. Worthy of further investigation

      
      
      

   
   
   

2. Belief:
   1. For any statement to be considered as a candidate for belief it must set up an expectation
   2. For example the sale on oranges statement would set us up to expect we can get 12 oranges for five cents
   3. Such statements have cognitive content - they convey something that can be known
   4. Statements lacking cognitive content give us nothing to believe in
   5. Not all declarative statements have cognitive content
      1. Some of these are obvious nonsense - “The square root of Tuesday is a prime number.”
      2. Others are not obvious. They are ‘pseudo-declarative statements’
         1. They are meaningless claims
         2. Empty propositions
         3. Sophistry
         4. This doesn’t stop many people from believing in them
         5. Astrology

         
         
         

      
      
      

   
   
   

3. Cognitive Content
   1. One way to test if a statement has it is the discernible difference test
   2. Described by John H. Hall
   3. “Utterances with cognitive content make claims, either true or false, and whether they are T/F makes a difference that can be discerned. That’s why these utterances offer something to believe and there's no point trying to believe an utterance that makes no such offer.”
   4. Predictions can be tested using discernible difference
      1. If a prediction has cognitive content it will be clearly discernible whether the outcome came true or not
      2. Many if not most forecasts don’t meet this standard
         1. The predictions are too vague to say if they’re wrong

         
         
         

      3. Oranges are on sale for 5c/doz will make a discernible difference to me if I go to the market looking for oranges.
      4. This difference can be tested

      
      
      

   5. Hall condemned Freudian analysis for its lack of cognitive content
   6. The Intelligent Design theory has the same problem, says Aronson

   
   
   

4. Knowledge part 2
   1. Knowledge is justified true belief
   2. Declarative statements can be considered knowledge if it meets the following conditions:
      1. It has cognitive content (is a candidate for belief)
      2. It must be true (or probably true)
      3. It must be justified by sound inferences from solid evidence

      
      
      

   3. Beliefs, even when correct, that are based on no (or false) evidence don’t constitute knowledge
      1. Consider a caveman who suspects the sun moves across the sky because the earth is rotating
      2. His belief is true but he has no knowledge because he has no evidence
      3. His belief is therefore not justified

      
      
      

   4. Erroneous beliefs fail these conditions
   5. False beliefs are unavoidable when we attempt to know things about the world based on observed evidence
      1. Knowledge based on the scientific method is inherently uncertain, and provisional, though it’s better than ‘knowledge’ acquired by other means
      2. Scientific knowledge improves over time, a work-in-progress

      
      
      

   
   
   

Diagrams because everyone loves diagrams!
The cost of undisciplined analysis

1. Objective TA is
   1. Well-defined
   2. Repeatable
   3. Issues unambiguous signals
   4. Can be implemented as computerized algorithms
   5. Can be backtested on historical data
   6. Results can be rigorously evaluated in a quantitative manner

   
   
   

2. Subjective TA is
   1. The opposite, usually
   2. Needs private interpretation by an analyst
   3. This thwarts, computerization, back testing, objective evaluation
   4. Insulated from challenges by evidence
   5. Producer of meaningless claims, conveying an illusion of cognitive content
   6. Subject to different conclusions from different analysts based on the same data
   7. Based on faith, like religion
   8. Included in:
      1. Classical chart patterns
      2. Hand-drawn trendlines
      3. Elliott Wave principles
      4. Gann patterns
      5. Magic Ts
      6. And much, much more!

      
      
      

   
   
   

3. Objective TA can also spawn erroneous beliefs but differently
   1. From faulty inferences

   
   
   

4. Objective methods that are profitable in a backtest are not sufficient to conclude the rules have merit
   1. Historical success is necessary but not sufficient by itself to prove predictive power
   2. Can be influenced by luck or upward bias

   
   
   

5. Statistical inference can answer whether a method is good or just lucky

How EBTA is different

1. Restricted to meaningful claims
2. Uses advanced statistical inference to determine if profitable backtests are indicators of an effective method
3. Determines which objective methods are worthy of use
4. Rejects all subjective TA
5. Ethically superior as recommendations have a reasonable basis and do not make unwarranted claims

Results from Academia

1. For those who want exploitable ‘tips’ to use in the markets this is about as good as it gets from this book without doing research on your own
2. Expert chartists cannot distinguish actual price charts of stocks from charts produced by a random process
3. There is empirical evidence of trends in commodities, FX, that can be exploited by trend indicators
   1. Such trends may be explainable as the economic service provided to hedgers for transference of price risk from hedgers to speculators

   
   
   

4. Simple technical rules, used individually and in combinations can yield statistically significant profits when applied to averages of relatively young companies
5. Neural networks have been able to combined buy/sell rules into nonlinear models that show good predictive performance
6. Trends in industry groups and sectors persist long enough after detection by simple momentum indicators to earn excess returns
7. Stocks that have displayed relative strength and relative weakness continue to do so over horizons of 3-12 months
8. US stocks selling near their 52 week highs, outperform other stocks.
9. An indicator defined as the differential between a stock’s current price and its 52 week high is a useful predictor of future relative performance.
10. The head and shoulders chart pattern has limited forecasting power (random)
    1. Better results come from simple filter rules

    
    
    

11. Trading volume stats for stocks contain useful predictive info and improve profitability of signals based on large price changes following a public announcement
12. Neural nets, genetic algos and statistical learning and AI methods have found profitable patterns in technical indicators.

Basis for criticizing traditional TA

1. Mediocre performance using TA trading for Spear, Leeds & Kellogg
2. How We Know What Isn’t So - Gilovich
3. Why People Believe Weird Things - Shermer
4. Instead of art vs science it should be framed as science vs superstition

It’s unlikely that TA will discover rules that predict with the precision of physics, because of the inherent complexity and randomness of markets and the impossibility of controlled experimentation

Google Doc with my footnotes

Evidence-Based TA Chapter 1 : Objective Rules and Their Evaluation (part 1 of 2)
The Great Divide: Objective versus Subjective TA

1. Objective TA - clearly defined - simulatable, measurable
2. Subjective TA - not precisely defined - open to interpretation
3. Programmability criterion - a method is objective if and only if it can be implemented as a computer program that produces unambiguous market positions (long, short or neutral). Methods that cannot be reduced to a program are subjective by default.

TA Rules

1. Objective TA == mechanical trading rules or trading systems == rules
2. Rule function -> input (information/time series) -> output (market position/signed number like +1/-1)
3. Positive numbers = long; Negative numbers = short
4. Rules generate signals when the output value changes, for ex. When output goes from -1 to +1 it means close a short position, open a long. Volume isn’t restricted to one unit.
5. Binary rules = rules with binary output, for example only +1 or -1.
6. All the rules considered in this book are binary, long/short
7. The time series is assumed to contain a mixture of signal and noise
8. Thresholds assign a critical level when the rule will engage, it is a form of filter
9. Logical operators such as inequalities detect critical events that cross the threshold
10. Example: Moving average cross rule:
    1. Variable threshold is used to generate signals on a time series that displays trends
    2. Signals when the time series crosses from one side of the MA to the other
    3. “If the time series is above its MA then the rule output value = +1, otherwise, rule output value = -1”
    4. Signals are mutually exclusive. Either long or short but not both at the same time.

Attached Image

1. Fixed value threshold rules are suitable for market time series with no trends - said to be ‘stationary’, not the strict mathematical definition, but meaning relatively stable average value over time, with fluctuations that are roughly horizontal in range
2. TA practitioners refer to these series as oscillators
3. Time series with trends can be detrended - in other words, transformed into a stationary series
4. Once detrended it will be seen to fluctuate within a well-defined horizontal range around a stable mean value
5. Once stationary, fixed threshold rules can be employed
6. Multiple thresholds create more than two possible conditions
7. This requires a different operator, known as a flip-flop, referring to the output value that flips one way after one event then flops the other way upon the occurrence of a second event
8. An example using flip-flop logic and multiple variable thresholds is the MA band rule.
9. MA is surrounded by upper and lower band
10. +1 on upward piercing of upper threshold, -1 on piercing of lower threshold

Traditional Rules and Inverse Rules

1. Given that traditional TA may be questionable it is desirable to test rules that are contrary to the traditional interpretation
2. Patterns that are supposed to predict rising prices might actually predict falling prices
3. This can be tested with an additional set of rules whose output is opposite traditional TA rules - inverse rules

Benchmarks in Rule Evaluation

1. Performance figures are only informative when they are compared to a relevant benchmark
2. If a rule earns 10 percent that might sound good but if other rules earn 30 percent on the same data that’s actually inferior performance
3. “This book” defines rule performance against a benchmark performance of a rule with no predictive power, for example a randomly generated signal
4. Compare medicines compared against placebos
5. Rules must not just beat the benchmark, but beat them by a wide enough margin that their outperformance is not attributable to luck

Conjoint (combined) Effect of Position Bias and Market Trend on Back-Test Performance

1. A rule’s back-tested performance has two independent components
   1. Good: the rule’s predictive power, if any
   2. Bad:
      1. the rule’s long/short position bias
      2. The market’s net trend during the back-test period

   3. These can dramatically affect back-tests and make rule evaluation difficult

2. Position bias
   1. If one output state (+1/-1) dominated during the backtest then the rule is said to have position bias

3. Market’s net trend
   1. Daily average price change during the backtest, if more or less than zero, will impact the rule’s performance, either adding to or subtracting from the performance component that is due to the rule’s predictive power

4. Roulette wheel example
   1. Imagine a roulette wheel where 75 of the 100 slots are +1 and 25 are -1. Even though the wheel itself generates random outcomes, a strategy that went long using the wheel’s advice would generate positive expected return if the market was in a net uptrend.
   2. This is a TA rule that has a long position bias but has no predictive power
   3. Just as it is possible for a rule with no predictive power to produce a positive rate of return, it is also possible for a rule with predictive power to produce a negative rate of return, especially if the rule has a position bias that is contrary to the market’s trend
   4. Market trend + position bias can offset any positive return

5. Position bias and market trend must therefore be eliminated to develop a valid performance benchmark
6. The way a rule’s long/short conditions are defined can create position bias, which can easily be mistaken for predictive power.
   1. If a rule’s long condition is more easily satisfied than its short, for example it would get a performance boost during an upward trending market

7. S&P500 / Dow Jones Transportation Avg example
   1. Assume a MA with bands set at +3% and -3% applied to DJTA
   2. Short the S&P500 while the DJTA is below the lower band
   3. Long at all other times
   4. Such a rule would benefit if S&P is in an uptrend

1. Expected rule return is based on:
   1. Proportion of time rule spent in long positions
   2. Proportion of time rule spent in short positions
   3. Market’s avg daily price change during the test period

2. Attached Image

   

We can adjust the back-tested (observed) performance by the expected return of a rule with no predictive power that has an equivalent position bias to remove the deceptive component

Google Doc

Evidence-Based Technical Analysis Chapter 1 part 2 (of 2)

A Simpler Solution to Benchmarking - Detrending Market Data

1. The above procedure can be ‘burdensome’ with many rules being tested
2. Instead we can just detrend the market data
3. Detrended data is used to calculate daily rule returns - not to generate signals
   1. Signals would be generated from actual market data

4. Detrending
   1. Simple transformation
   2. Results in a new market data series whose average daily price change = zero
   3. Expected returns from a rule with no predictive power on detrended data should be close to zero, and conversely a rule with predictive power should be > 0.
   4. This positive return reflects the rule’s intelligent positioning, as opposed to randomness
   5. Method
      1. Calculate the average daily price change of the market during the test period
      2. Subtract that value from each day’s price change

   6. Maths
      1. ER = [p (long) x av. daily return] - [p (short) x avg. daily return]

   7. Example
      1. If position biases were 60% long and 40% short the expected return is zero
      2. 0 = [0.6x0] - [0.4 x 0]

Using Logs of Daily Price Ratio Instead of Percentages

1. Returns have been computed as percentages but there are problems
2. Problems can be solved using logs of daily price ratios
3. Log {current day’s price / prior day’s price}

Look-Ahead Bias and Trading Costs

1. “It is said the devil lives in the details.”
2. Look-ahead bias, aka ‘leakage of future information’ occurs when info that was not truly available at a given time is (incorrectly, usually unknowingly) assumed during testing
3. A signal generated with future knowledge is obviously, invalid
4. Often a subtle problem
5. Example
   1. Suppose a rule uses market closing prices
   2. No actual trading can be placed after market close
   3. So it’s not legitimate to assume we could enter at the market’s closing price
   4. Acting on this assumption would be look-ahead biased
   5. Earliest opportunity to enter would next day’s opening price, so “this book” assume execution at the opening price on the following day

6. Look-ahead bias can infect back-test results when rules use data with a lag or that is subject to revision - ex. Mutual fund cash statistics, released on a 2 week delay, must take this lag into account by offsetting the signals by 2 weeks to reflect the data’s actual availability
7. “This book” doesn’t use lagged data or revised data

Trading Costs

1. Should trading costs be taken into account in rule back-tests?
   If the intent is to use rules on a stand-alone basis for trading then yes.
2. If the purpose of rule-testing is to discover signals that contain predictive info - then rule costs obscure the value of a rule that reverses frequently. So this book doesn’t impose trading costs in its rules.

Lolz - this has to be the most academic thing I’ve ever read. Is your intention to make trading rules to make money? It is??! Why then yes you must take trading costs into account. If not, like me, I have no expectations of earning anything but a tenured salary, why would I bother? (puffs on pipe and rubs elbow patches)

Chapter 2 : The Illusory Validity of Subjective Technical Analysis (part 1)
The difference between a crank and a charlatan is the charlatan knows he is dealing in snake oil, the crank does not. - Martin Gardner
If you only read one chapter summary, for most retail traders it should be this one.

1. Chapter goal1 : encourage an attitude of skepticism toward subjective TA
2. Chapter goal2: underscore the need for rigor, objectivity in knowledge acquisition in the face of human tendency to form beliefs in the absence of evidence or contradictory evidence
3. We believe we acquire knowledge purposefully, but actually we make all kinds of automatic assumptions, cognitive errors, and hold illusions to be real
4. This is a serious liability

Senses Can Deceive Us
If you’ve read anything by Kahneman/Tversky or have some background in cognitive biases, then you have a grip on everything in this first section. People suck at thinking. There are better books to read to learn about this if you don’t know it already. I’m going to skip a few things but show you this one optical illusion because it really is amazing.

Being a skeptic I took the original image and cut it out in Photoshop and reassembled it to compare side by side.

Then I overlapped the two tables to compare:

You can see there is just a sliver of the second table showing behind the first so it TECHNICALLY isn’t true that both tops have the same size and SHAPE, Prof. Liar!
Still, we take the point, people (and eyes) are easily fooled.

Personal anecdotes

1. Aaronson learned TA as a teenager
2. TA seemed ‘magical but plausible’
3. “Like most TA true believers, my initial convictions came secondhand, reading the revealed wisdom of the field’s self-proclaimed authorities.”
4. Authorities often don’t base their claims on firsthand research but on regurgitated claims
5. Artemus Ward “It’s not so much the things we don’t know that get us into trouble as the things we know that just ain’t so.”
6. How I Made Two Million Dollars in the Stock Market - Nicholas Darvas; used a charting method called box theory
7. This is really just support/resistance levels
8. Chartcraft’s point and figure method
9. Technical Analysis of Stock Trends - Edwards and Magee “the bible”
10. Aaronson started giving stock tips but after some initial luck his reputation faded
11. “It was always possible to explain away my failures.”
12. Philosophy of Science taught that science is more than a set of facts.
    1. It is a method for distinguishing real from illusory knowledge
    2. Took Aaronson 40 years to see the connection between TA and epistemology

    
    
    

13. Trading for a prop firm his skepticism grew
    1. Made profits for the first 3.5 years between Oct 1996 - Feb. 2000
    2. Gave back all his gains in March 2000
    3. 5.5 years later, results were ‘lackluster’
    4. Tried to make a ‘systematic trading program’ but these plans never came to fruition - blames limited time and capital (???)
    5. Kept a detailed journal
    6. Noted trade rationales; managers insisted on an objectively falsifiable prediction
    7. This practice made it hard to rationalize failures - “there they were staring me in the face”
    8. Considered that maybe he was incorrectly implementing TA dictates
    9. Colleagues dismissed his skepticism - TA was obviously valid
    10. “Trends and patterns in historical charts were simply too evident to be illusory”
    11. The same patterns appear in random data
    12. “Obvious validity” is an inadequate standard for judging validity

    
    
    

14. The mind is a natural pattern-finder
    1. Human tendency to find patterns is based on evolutionary adaptations
    2. Perceiving order was crucial to hunter/gatherer survival
    3. We are more prone to adapt falsehoods than to overlook truths
    4. The complexity of modern life has changed this picture
    5. Decisions are more complex
    6. The costs of learned falsehoods are greater
    7. Global warming - real or false?
    8. If the disbelievers are wrong then future generations will pay dearly
    9. The costs of treating it as a serious threat when it isn’t are more modest
    10. We are no less superstitious than our ancestral cave-dwellers
    11. ‘Epidemic of weird beliefs’ - astrology, ESP, spirits, psychics, etc.

    
    
    

Cognitive Psychology : Heuristics, Biases, and Illusions

1. Concerned with how we process info - sensory input -> transformed, reduced, elaborated, stored, recovered
2. Common sense, intuition is mostly correct
3. Human intelligence is poorly adjusted to making choices in uncertainty (inadequate info)
4. Kahneman, Slovic, Tversky - showed that illusory knowledge originates
   1. With cognitive biases and illusions that distort experience
   2. To deal with limited processing power our minds create shortcuts called heuristics (rules of thumb) - automatic, basis of intuitive judgments
   3. Generally successful but in some types of situations catastrophically bad at making correct choices
   4. Erroneous info is problematic because it is resilient

   
   
   

1. Herbert Simon - humans are ‘boundedly rational’
2. We attend to only a small fraction of the info the world presents us
3. Configural thinking
   1. Even more limited is our mind’s ability to handle problems requiring a multitude of factors (variables)
   2. We can handle at most three factors when they must be evaluated in a configural fashion
   3. Configural thinking is more demanding than linear or sequential, where variables can be analyzed independently

   
   
   

4. Human experts are less effective than human-designed models because they fail to combine the information in the consistent manner of a formal mathematical model
5. Configural problems contain webs of relationships that cannot be evaluated in isolation

Overconfidence Bias

1. In general people are overconfident in their own abilities, and arrogant about their actual level of knowledge
2. Overconfidence may be inborn, since confidence is a feature of successful mating strategies, we are likely the offspring of swaggering parents
3. As a species this may be advantageous - after enough overconfident individuals fail, eventually one will succeed and there are benefits for the group
4. Overconfidence is reinforced by other biases
5. Overconfidence is most extreme for difficult or impossible tasks - like trading!
6. Confidence increases with new information though prediction accuracy does not
7. Indicator correlation increases confidence but unless those indicators are independent (uncorrelated) the increased confidence is unwarranted
8. Quarterly earnings forecasts average error is 44 percent
9. Two professions that have managed to avoid overconfidence:
   1. Meteorologists
   2. Horse-race handicappers
   3. Why?
      1. Face similar problems every day
      2. Make explicit falsifiable probabilistic predictions
      3. Obtain swift and precise feedback on outcomes
      4. When these conditions are not satisfied we should expect overconfidence for both experts and non-experts

      
      
      

   
   
   

10. Falsifiable forecasts are rarely given in financial markets - overconfidence persists

Optimism Bias

1. Tendency to remain overconfident even in the face of mediocre results
2. Newsletter writers and buyers

Self-Attribution Bias - Rationalizing Failure

1. People tend to attribute successes to their own abilities and to attribute failures to external circumstances
2. Common sense tends to overlook luck - that’s for statisticians to detect

The Knowledge Illusion

1. False confidence in what we know
2. Based on the false premise that the more info we have the more knowledge we have
3. Based on overconfidence in the mind’s ability to perform configural reasoning

The Illusion of Control

1. Unwarranted belief in our ability to control outcomes
2. Control makes people happy and relaxed
3. Nofsinger - activities that are most likely to induce the illusion of control have:
   1. High level of personal involvement
   2. Large range of choice
   3. A great amount of info to consider
   4. High level of familiarity with the task
   5. Early success in the activity
   6. The first four clearly apply to subjective TA - making charts, reading indicators, choosing markets, instruments, regular use increases familiarity

   
   
   

Hindsight Bias

1. Viewing a completed event’s outcome as being more certain than it really was
2. We tend to forget how uncertain we were prior to knowing an outcome
3. Subjective TA lacks clearly defined rules for pattern identification, forecast generation, so it is susceptible to this bias

 

1. Whatever the outcome, it is always possible to selectively notice chart features that seemingly predicted correctly, among what is an ambiguous mix of bull and bear cues
2. Historians fall prey to this when they say something like ‘the rise of the Third Reich was quite predictable’ but in reality history could have unfolded many different ways
3. Studies show that even having been warned about the bias, and told to avoid it, people still fall prey to it
4. Only objective TA methods offer the opportunity for avoiding hindsight bias because only info known at a given time is used to generate signals and those signals are evaluated objectively - they are, in other words, falsifiable

Second-Hand Info Bias : The Power of a Good Story

1. Stories are the most popular way of acquiring knowledge - for millennia
2. Good stories are therefore more powerful persuaders than objective facts
3. Bertrand Russell - when we learn things informally (nonscientifically) we are impacted by the “emotional interest of the instances, not by their number” - for this reason scientists train themselves to ignore dramatic stories and pay attention to facts and numbers
4. Conflict exists between desire for knowledge and the desire that it be delivered as a good story
5. H.L. Mencken - “what ails the truth is that it is mainly uncomfortable and often dull. The human mind seeks something more amusing and more caressing.”
6. Storytellers know info delivered with too many qualifiers is unappealing. Inconsistencies and ambiguities are minimized, while cohesive aspects are amplified, robbing it of its essential truth.
7. Especially persuasive are accounts that link events with the chain of cause and effect
8. Perception of causal relationships is a natural cognitive capacity and causal chains satisfy our need to explain events
9. Juries - best causal narratives win in courtrooms - tie evidence into a coherent and credible sequence
10. False cause-effect explanations are hard to detect when they are plausible and appeal to a ‘sense of the ironic’ - especially true with urban myths

Elliott’s Tale

1. A good story may explain the enduring appeal of the Elliott Wave Principle (EWP)
   1. EWP holds that price waves express a universal order and form that is found not only in financial markets but in the natural world (sea shells, galaxies, sunflowers)
   2. EWP : market trends are fractal - a nested hierarchy of waves, with the same form but ranging in magnitude
   3. 8 segment configuration of rising and falling price movements
   4. Claims to describe just about anything that goes through cycles of growth and change
   5. Ties in with Fibonacci and the golden ratio - phi
   6. “That which purports to explain everything, explains nothing.”
   7. Not really a theory - just an eloquent story by Robert Prechter
   8. Made possible by loosely defined rules, large number of nested waves of varying magnitude
   9. This allows the Elliott analyst the same freedom and flexibility as pre-Copernican astronomers to explain all planetary movements even though their earth-centred theories were wrong
   10. EWP waves - > medieval epicycles
   11. Unlimited ability to fit past observations but no ability to make testable future predictions
   12. A story too good to let die

   
   
   

Confirmation Bias : How Existing Beliefs Filter Experience and Survive Contradicting Evidence

1. There is some rational basis for confirmation bias - without it, new information would too easily supplant old, and beliefs would become unstable
2. Consider the myth of cold fusion - rejected because it conflicted with prior knowledge
3. People are not usually aware of their unjustified beliefs
4. Beliefs shape expectations, which shape perceptions, which shape conclusions
   1. Thoreau - “We hear and apprehend only what we already half know.”
   2. “I’ll see when I believe it”
   3. The fake alcohol experiment - subjects who thought they were drinking booze acted with less social anxiety
   4. We fail to perceive new info that conflicts with past beliefs
   5. We are reluctant to draw conclusions at odds with current beliefs
   6. Confirmation bias predicts that erroneous beliefs will strengthen with time and familiarity
   7. A vicious cycle of growing self-deception, as erroneous results fail to overturn past lucky false positives

   
   
   

5. Motivational Factors
   1. TA practitioners are emotionally invested in their favored method
   2. We are motivated to maintain consistency within our system of beliefs
   3. Confirmation bias can affect how questions are framed, biasing search for new evidence

   
   
   

6. Confirmation bias can occur in objective TA
   1. Analyst searching for profitable rules with back testing
   2. After the first rule provides unsatisfactory results the search continues
   3. Rules are altered, sequences, parameters, logic, indicators, changed until a good one is found
   4. This is data mining, because the scope of the search is unbounded
   5. This guarantees a rule with good past performance will be found, and the researcher’s initial belief that a good rule will be found is confirmed

   
   
   

However past performance of a rule discovered with data mining will overstate its future performance (data mining bias)

Google Doc (footnotes)

EBTA Chapter 2 (part 2)

How Vague Evaluation Criteria Contribute to Confirmation Bias

1. Subjective TA evidence for success is ambiguous
2. Allows the analyst to point to successes and prevents identification of prediction errors
3. Prediction errors can be obscured by:
   1. Pattern renaming - changing a failed upside breakout to a bull trap
   2. No clear endpoint / infinite horizon
   3. Use of vague time horizons - “intermediate long-term”
   4. “Expect the market to advance 5% before it declines 5%” is objective
   5. Vaguely defined patterns
   6. Having two standards of evidence acceptance
      1. Evidence that supports the belief need only be plausible
      2. Evidence that would contradict the belief must be convincing beyond a doubt

4. Evidence that contradicts a belief can actually strengthen that (preexisting) belief
   1. Losing gamblers in an experiment evaluated their losses and wins differently

5. True Believers
   1. Studies show even after subjects are told they were tricked they hold onto erroneous beliefs
   2. Thought experiment - if Gann and Elliott were resurrected and announced that their methods were intentional scams, how many of their practitioners would continue to believe?

6. Subjective methods that are most likely to suffer from confirmation bias
   1. Have elaborate causal explanations, compelling stories
   2. High retrofit power - ability to explain past market behaviour accurately
   3. No ability to generate falsifiable predictions
      1. Elliott Waves
      2. Hurst cycle theory
      3. Astrology-based forecasting
      4. Gann analysis

1. More intelligent people are more likely to suffer confirmation bias!
   1. Higher IQ people are better able to construct articulate rationales for their beliefs
   2. Because of this strong ability to rationalize, intelligent people are more apt to form beliefs that are resistant to change
   3. Snelson - ideological immunity
   4. The most intelligent and experienced subjective TA practitioners will therefore find it hardest to abandon a method in which they previously believed

Illusory Correlations

1. Erroneous beliefs can be about ourselves and our abilities
2. They can also be about two observed phenomena that we believe have a causal link but are actually just coincidence
3. The entire business of TA can be viewed as a search for these correlations
4. Illusory correlation is perception of a correlation between two variables that doesn’t exist
5. Causes of illusory detection are related to info processing
   1. We pay too much attention to confirming instances, not enough to contradicting ones
   2. Most believers found their beliefs on having seen instances in which a pattern has been successful (confirmatory examples)
   3. However confirmatory instances constitute only one type out of four possible types of outcomes that can occur between a pair of binary variables
      1. Pattern/signal occurs and expected outcome occurs
      2. Pattern/signal occurs but the market trend predicted by the pattern doesn’t occur
      3. Pattern/signal doesn’t occur but the market outcome happens anyway
      4. Neither the pattern/signal nor the outcome occurs

   4. Type one and four are ‘correct’ predictions, two and three are errors. Type 2 is a false positive, type three is a failure to signal or a false negative.
   5. Most focus too much on the first type of outcome - type i above

1. We have faulty intuitive notions of what constitutes sufficient evidence to conclude that correlation exists.
2. Initial conjectures make confirmatory cases more salient (noticeable)
3. Necessary versus sufficient evidence
   1. Confirmatory evidence by itself is usually thought to be sufficient - WRONG
   2. Confirmatory instances are necessary but they form an incomplete picture
   3. The other cell outcomes in the 4 cell table above need to be taken into account

4. Studies show that informal analysis of two variables tends to make 2 common errors
   1. Failure to detect correlations that are valid (invisible correlations)
   2. False perception of correlations (illusory correlations)
   3. Both errors related to observer’s expectations based on pre-existing beliefs
   4. Most people need correlations above 0.7 (on a 0-to-1.0 scale) to detect a correlation - this is stronger than anything likely to be encountered within the domain of TA
   5. Smedslund showed people have a tendency to find correlations/patterns where none exist also

5. The statistically valid way to detect a correlation between two binary variables is the Chi-square test
   1. Determines whether the number of instances falling into each cell differs significantly from the number that would be expected in each cell if there were no correlations between variables
   2. When cell counts depart significantly from a random pattern, the Chi-square stat takes on a large value - legitimate evidence of a correlation.

EBTA Chapter 2 : part 3 (of 3)

1. Asymmetric binary variables in illusory correlation
   1. Illusory correlations are more likely to emerge when the variables are binary variables
   2. Symmetric binary variables - red/blue; Republican/Democrat
   3. Asymmetric binary variables - attribute is present/attribute is absent; event occurred/event didn’t occur
   4. Asymmetric variables are more likely to show illusory correlations since the presence of something is way more salient than the absence of something

   
   

2. Hidden or missing data compounds the problem of illusory correlations
3. Random reinforcement (rewards) produce the most resilient habits/resistance to disbelief - this is an effect seen in gambling, video games, drug addiction, etc.
4. Random reinforcement may also be the basis for superstitions

Misplaced Faith in Chart Analysis

1. Humans have a need to find order and meaning in experience
2. An informal search for order is sure to find it
3. What is ‘obvious’ is not necessarily valid One criticism I can make of this book is there is some repetition of themes early on. After a while, any reader who has made it this far has accepted Aronson’s proposition or they haven’t, and further ‘argumentation’ is not going to convince them.
4. Harry Roberts - “technical analysts fall victim to the illusion of patterns and trends"
   1. Because the usual method of graphing prices gives a picture of successive levels, creating the appearance of a pattern or trend
   2. Chance behaviour itself produces patterns, that are spurious
   3. Roberts showed the same patterns appear in random walks

   
   

5. The same illusory trends appear in sports stats

Intuitive Judgment and the role of Heuristics

1. Intuition is automatic
2. Perceptive, rapid, effortless
3. Kahneman - type 1 thinking
4. Controlled thinking is effortful
5. Slow
6. Critical, analytic
7. We use heuristics (shortcuts) and they often fail
8. Availability heuristic (bias)
   1. Used to estimate the likelihood of future events
   2. Events that are most easily brought to mind are more likely to be falsely considered likely events

   
   

9. Representativeness heuristic (bias)
   1. Used to make intuitive classification judgments
   2. Dogs > poodles
   3. Chart analysis is inherently flawed because of a bias toward the perception of order and pattern even in data that is truly random
   4. This bias is due to a misapplication of the representativeness heuristic
   5. Example of the man on an airplane who matches stereotypes of a terrorist
   6. Bayes theorem

   
   

10. Base rate fallacy
    1. The representativeness heuristic uses a base rate to make assumptions
    2. Most people have inaccurate base rates - e.g. thinking terrorism is common

    
    

11. Conjunction fallacy
    1. A failure to realize a basic law of logic, which is that a proper subset must contain a smaller number of objects than the set to which it belongs.
       1. Horses - are in the group animals, not the other way around.
       2. People make this mistake when thinking about stereotypes
       3. The stereotype must be a smaller subset than the group of people it contains
       4. Example of Linda - outspoken, bright, philosophy major, concerned with justice, discrimination, antinuclear
       5. Is she a bank teller or a bank teller and a feminist?
       6. It’s more likely she is just a bank teller since as soon as you add more qualifiers to her category it shrinks the likelihood she belongs to the subset
       7. 85 percent of subjects got this wrong

    
    

12. Aronson suggests we make a chart of 300 coin flips

1. Randomness, true randomness, doesn’t ‘look’ random to us.

1. Belief in visual chart analysis as a valid method of detecting trends is based on a flawed chain of thinking
   1. On the basis of the rep. Heuristic, it is thought that data generated by a random process should ‘look’ random, formless, with no hint of organized trends or shapes
   2. It is erroneously assumed to manifest in all samples of random data, regardless of the number of samples
   3. A small data sample is examined and appears to NOT match the stereotype of randomness generated by our intuitions
   4. The data is thus deemed to be a product of a nonrandom process
   5. Nonrandom processes are amenable to prediction, hence it is worthwhile using the data to make predictions

   
   

1. Law of large numbers and the crime of sample size neglect
   1. Law of large numbers tells us that sample size is of paramount importance
   2. Because it determines confidence we can have in inferences based on data
   3. Aka law of averages

   
   

1. Only samples of large numbers of observations reliably reflect the total population of data
2. Small samples can therefore be misleading
3. Coin flip experiment examples (yawn)
4. The erroneous notion that random processes should alternate more and streak less is responsible for two false perceptions
   1. Clustering illusion
   2. Gambler’s fallacy
   3. I’ve talked about these things before so I’m a skip ‘em ok?

   
   
   
   
   

The Antidote to Illusory Knowledge : The Scientific Method

1. In ch. 3. That’s right, we’ve plowed through 2 whole chapters of this behemoth. Congrats if you made it this far!

Evidence-Based Technical Analysis Chapter 3 (part 1)
In this chapter we take a detailed look at the scientific method. How can this possibly help you become a better trader? Well, it might not do so by itself but if you currently subscribe to a method that is overall random or possibly even worse, without making allowances for randomness, then you are likely falling prey to a logical or evidentiary fallacy. Recognizing what makes good science will benefit you when evaluating the claims of others, and when choosing a strategy. Heck, it could even benefit you in other ways, like knowing when something you read in the news is probably not based on facts (BS).


Chapter 3 : The Scientific Method and Technical Analysis

1. TA’s central problem is erroneous knowledge
2. Mostly dogma and myth
3. The scientific method is a set of methods designed to describe, interpret observed or inferred phenomena, aimed at building a body of knowledge

The most Important Knowledge of All : A Method to Get More

1. The scientific method is the “most valuable knowledge given to the rest of the world” by the West
2. The method’s rigor protects us from ‘frailties of the mind’
3. At this point Aronson launches into a ‘brief history of science’ which is interesting, but not relevant to our concerns - skipping a lot
4. Principle of simplicity - a bit like Occam’s razor - simple is better
5. The random walk is a preferred explanation to Elliott Wave or Fibonacci sequences because its explanation is simpler
6. Objective observations can be shared and confirmed by others

The Nature of Scientific Knowledge

1. Scientific knowledge is objective
2. Empirical - based on observations
3. Quantitative - measured, based on maths principles - reducible to numbers
4. Purpose of Science - explanation and prediction
   1. Scientific laws aim to make predictions about new observations
   2. Why does B follow A
   3. The more general a theory, the more prized it is
   4. The more consistent the prediction, the more valuable
   5. Most important type of scientific law is the functional relationship
      1. Summarizes a set of observations in an equation
      2. Dependent variable - (Y) is a function of (dependent on) one or more other variables (independent, or predictors) denoted by (X)
      3. Y = f(X)
      4. Can be deduced from explanatory theories
      5. Estimated (induced) from historical data by function fitting (regression analysis)

The Role of Logic in Science

1. Scientific knowledge is respectable because its conclusions are based on logic
2. This avoids two common fallacies in informal reasoning:
   1. Appeals to authority - ‘proof’ from a respected source
   2. Appeals to tradition - think of the Dothraki women in Game of Thrones. “It is known.”
   3. Much of popular subjective TA is justified on these ground

3. 1st rule of logic - consistency
   1. Law of the excluded middle - something either has or doesn’t have a specific attribute; it can’t have AND not have it at the same time
   2. Law of Noncontradiction - A given statement is either TRUE or FALSE, not both
   3. Violating either of these principles makes a statement self-contradictory
   4. It may not be possible to prove that a rule has predictive power, but using the method of indirect proof it may be possible to prove that a rule doesn’t have it

4. Propositions and arguments
   1. Proposition is a declarative statement
      1. Either true or false
      2. “The head-and-shoulder pattern has more predictive power than a random signal” is a proposition
      3. Unlike questions, exclamations, commands, only propositions can be affirmed or denied

   2. Arguments
      1. A group of propositions
      2. One of which is the conclusion, which follows logically from the other propositions, called premises

5. Deductive Logic
   1. Categorical syllogisms - contain two premises and a conclusion
      1. Deals with logical relations between categories
      2. Premise states a general truth about a category
      3. “All humans are mortal”
      4. Ends with a conclusion that states a truth about a specific instance of that category - for ex. “Socrates is mortal”
      5. So:
         1. Premise 1: All humans are mortal
         2. Premise 2: Socrates is a human
         3. Therefore (ergo): Socrates is mortal

      6. General form
         1. 1: All members of A are members of B
         2. 2: X is a member of A
         3. Therefore X is a member of B

   2. Deductive logic’s appeal is its certainty - if two conditions are met then a conclusion arrived at by deduction is true with complete certainty
      1. If and only if the premises of the argument are true
      2. And if and only if the argument is in a valid form
      3. Otherwise the conclusion is false with complete certainty

   3. Truth and falsity are properties of individual propositions
   4. Validity is a property that pertains to the form of an argument
   5. Validity refers to the correctness of the logical inference linking premises to the conclusion
   6. Validity makes no claims to the factual truth of the argument’s premises
   7. Therefore an argument can be valid even if its propositions are false
   8. Here’s a valid argument with a false conclusion:
      1. All humans are immortal
      2. Socrates is human
      3. Therefore, Socrates is immortal

   9. We can use euler circles (diagrams) to demonstrate validity

1. Conditional syllogisms : the logic of scientific arguments
   1. Another form of deductive argument
   2. Logical basis for discovery of new knowledge
   3. Similar to categorical syllogism in that it has 3 propositions - 2 premises and a conclusion
   4. Conditional propositions combines two propositions with if-then
   5. The proposition after if is the antecedent clause
   6. The proposition following then is the consequent clause
   7. General form:
      1. If (antecedent), then (consequent)

   8. Example
      1. If it is a dog, then it has four legs
      2. ‘If it is a dog’ is the antecedent clause
      3. ‘It has four legs’ is the consequent clause

   9. TA example:
      1. If the TA rule has predictive power, then its back-tested rate of return will exceed zero
      2. The ultimate goal is to establish the truth of the antecedent clauses

   10. The second premise of the conditional syllogism either affirms or denies the truth of the antecedent or consequent of the first premise
   11. Second premise examples using our dog example:
       1. It is a dog - affirms truth of antecedent
       2. It is not a dog - denies the truth of the antecedent
       3. It has four legs - affirms the truth of the consequent
       4. It does not have four legs - denies the truth of the consequent

   12. Conclusion affirms or denies the truth of the remaining clause of the first premise - whichever clause wasn’t mentioned in the second premise
   13. Complete conditional syllogism example:
       1. If it is a dog, then it has four legs
       2. It is a dog.
       3. Therefore, it has four legs.

   14. If its form is valid (conclusions are logically compelled by the two premises) and its premises are factually true, then the conclusion must also be true

2. Valid forms of the conditional syllogism
   1. Affirming the antecedent - 2nd premise affirms truth of the antecedent clause of the 1st premise
   2. Denying the consequent - 2nd premise denies the truth of the consequent clause of the first premise
   3. Affirming the Antecedent example (all examples assume dogs have 4 legs)
      1. 1: If it is a dog, then it has four legs
      2. 2: It is a dog
      3. Therefore it has four legs

   4. General version of the above
      1. If A is true, then B is true
      2. A is true
      3. Therefore, B is true

   5. Denial of the Consequent example
      1. 1: if it is a dog then it has four legs
      2. 2: it does NOT have four legs
      3. Therefore it is not a dog.

   6. General version of the above
      1. If A is true, then B is true
      2. Be is not true
      3. Therefore A is not true

3. Science uses evidence to prove that an antecedent is false
   1. The antecedent in a scientific argument is called the hypothesis
   2. We hypothesize it is a dog
   3. The consequent predicts what will be observed if it is true

4. If we can prove that hypothesis A is false, we can indirectly prove that another hypothesis is true
5. People tend to commit two fallacies when reasoning with conditional propositions
   1. Affirming the consequent (invalid)
   2. Denying the antecedent (invaid)

6. Example of affirming the consequent
   1. If it is a dog then it has four legs
   2. It has four legs
   3. Therefore it is a dog.
   4. It could be a dog, but it could also be any number of other animals with four legs
   5. A very common error in poor reasoning, scientific and otherwise

7. Consider the following:
   1. If TA rule X has predictive power then it should produce a profitable backtest
   2. The backtest was profitable
   3. Therefore the TA rule has predictive power

8. Deductive reasoning has one Achilles heel. It cannot reveal new knowledge about the world.

1. Inductive logic
   1. The logic of discovery
   2. Aims to reveal new knowledge
   3. The price of this knowledge is uncertainty - they can only be true with some degree of probability
   4. Induction and probability are intimately connected
   5. Proceeds opposite way from Deduction
      1. Deduction -> premise expressing general truth ->applies to unlimited circumstances (Socrates is a man)
      2. Induction -> premised based on limited observations ->generalized conclusion applies to an unlimited number of similar unobserved instances (most observed healthy dogs have 4 legs, all healthy dogs will have four legs)
      3. In leaping beyond what has been directly experienced by observation induction incurs uncertainty

   6. All Xs are Ys (universal generalization)
   7. 100% of Xs have attribute Y (universal generalization)
   8. P percent of Xs are Ys (non-universal generalization)
   9. P percent of Xs have attribute Y (non-universal generalization)

2. Probability is inextricably tied to generalization
   1. Non-universal generalizations aka probabilistic generalizations
   2. Example: higher percentage of bulldogs are prone to violence vs poodles
      1. Doesn’t claim ALL bulldogs are violent
      2. However it claims bulldogs are more likely than poodles to be violent

   3. Aristotle failed to appreciate the value of nonuniversal generalizations
      1. He focused on deductive proofs, refusing that nature could be probabilistic

3. Induction by Enumeration
   1. Premise ->enumerates evidence in a set of observations ->draws general conclusion pertaining to all similar observations outside the set
   2. Example
      1. Over past 20 years, there have been 1,000 instances in which TA rule X gave a buy signal and in 700 of those instances, the market moved higher over the next 10 days
      2. Conclusion: In the future when rule X gives a buy signal, there is a 0.7 probability the market will be higher after 10 days.

   3. Like all induction, inherently uncertain
   4. Some inductive arguments are stronger, therefore more certain than others

4. More similar observations means more certainty
   1. Also depends on the quality of the observations (evidence)
   2. Also relevant is the degree to which evidence cited is representative of the kind of observations likely to be encountered in the future
      1. A rule that was tested in rising markets is unlikely to be as successful in falling markets

5. Hasty generalizations are the most common fallacy of induction
   1. Based on too little evidence or evidence of poor quality

(Doc with footnotes)

Evidence-Based Technical Analysis Chapter 3 (part 2)


The Philosophy of Science

1. Seeks to understand how and why science works
2. Scientific method
   1. (one of) Man’s greatest inventions
   2. Most effective method of acquiring objective knowledge about the natural world
   3. Monument to the ‘industrial west’
   4. Worked before people really understood why it worked
   5. Philosophical question - how can so much useful knowledge emerge from such an inherently uncertain (therefore flawed) method of inference?

   
   

3. Bacon’s enthusiasm
   1. Scientific revolution was in part a revolt against Aristotelian science (deduction)
   2. Ancient Greeks regarded the natural (physical) world as an unreliable source
   3. Plato
      1. The world is merely a flawed copy of the truth and perfection that exists in forms - a metaphysical world
      2. Forms are perfect archetypes of everything that exists - the perfect circle, the perfect dog, etc.

      
      

   4. Empiricism rejects this view. The natural world is worthy of study and even more, only knowledge that is gathered directly from observations is reliable.
   5. Francis Bacon was maybe the first philosopher of science and a pioneer of empiricism
      1. Science is a completely objective rational practice
      2. Observe without bias, then generalize based on those observations
      3. Drawbacks:
         1. Nature must be assumed to be uniform (universal)
         2. Many natural phenomena defy observation

   
   

4. Several pages of interesting but ultimately irrelevant history of science examining Descartes (skepticism), Hume (problem of induction), Whewell (role of induction in hypothesis-induction cannot produce truths on its own, it needs a creative aspect), and Popper (falsificationism, rejecting logical positivism - science is a baloney detector, not a truth detector)
5. Data cannot be used logically to deduce the truth of a hypothesis, attempts to do so commit the fallacy of affirming the consequent
6. Data can be used to deduce the falsehood of a hypothesis by denial of the consequent
7. This all runs counter to ‘common sense’ (intuition?) which is biased in favour of confirmatory evidence.
8. John Stuart Mill and the black swan - amply covered in Fooled by Randomness (Taleb) or his book of the same name.
9. All scientific knowledge is provisional (a target for challenge and replacement)
10. The net result of building upon successful ideas (whose predictions have been confirmed) and pruning away wrong ideas (whose predictions have been falsified) is a body of knowledge that continually improves.
11. Science must restrict itself to testable statements
12. Hypotheses must be able to make predictions about future observations

The Demarcation Problem - Distinguishing Science from Pseudoscience

1. Popper’s method solved a key problem in the philosophy of science - defining the boundary between science and non-science (nonsense?)
   1. Science is confined to propositions that make predictions that are open to refutation with empirical evidence
   2. Falsifiable
   3. Meaningful
   4. Unfalsifiable propositions might appear to assert something but they don’t
   5. Falsifiability is therefore related to information content
   6. Informative statements are falsifiable, otherwise they are not information
   7. Example - a weather forecast that predicts cloudy, sunny, wet or dry, windy or calm, cold or hot - allowing for all possible outcomes. Cannot be wrong. Yet is completely useless.

   
   

2. Limitations of Popper’s method
   1. Real science deals with complex hypotheses - nonuniversal, probabilistic
   2. A proposition that rests on auxiliary propositions might make a prediction that is later falsified but it wouldn’t be clear which proposition was at fault
   3. Example- Uranus’s orbit
      1. Orbit was weird and didn’t conform to Newton’s laws - was Newton wrong?
      2. No, there was another undiscovered planet that was affecting the orbit (Neptune)
      3. Was Newton completely correct? No his theory fails on some finer points in the 20th century.

      
      
      

   4. Probabilistic hypotheses with contradictory observations are not therefore certain evidence of falsity, as the aberrant data might be due to chance

   
   

Information Content of Hypotheses

1. Theories that make bold claims are more informative than timid hypotheses that are harder to prove wrong
   1. Timid hypotheses include the Elliott Wave principle
   2. It seems to be brave on the surface, proclaiming that all price moves in all markets over all time scales can be described by a unifying principle
   3. In reality it makes no falsifiable predictions of future price, and is therefore timid
   4. Another example is the - Efficient Market Hypothesis (EMH)
      1. In an efficient market relevant info is presumed to be reflected in price almost immediately
      2. Three flavors - strong, semistrong and weak
         1. Strong - financial markets are efficient with all info, even private inside info, so insider tips are useless.
            1. This is the most falsifiable and therefore most useful
            2. Any evidence of excess (abnormal) profits from any strategy would be enough to refute it

            
            

         2. Semistrong - less informative, narrower, harder to falsify
            1. Market is only efficient with respect to public info
            2. Can be falsified by evidence of market-beating returns produced by an investing strategy based on public fundamental data or technical data
            3. Denies the usefulness of technical or fundamental analysis

            
            

         3. Weak - least bold, least informative, hardest to falsify
            1. Market is only efficient with respect to past price, volume and other technical data
            2. Denies the usefulness of technical analysis
            3. Falsifying this version is the most difficult as you would need to show excess returns generated by an investment strategy based on TA

      
      

   5. The gain in knowledge that occurs upon falsification of an hypothesis is inversely related to its info content
   6. The most audacious claims have the most content
   7. The most timid claim possible has a special name - the null hypothesis
   8. The null hypothesis is the starting point for new claims, it is the default assumption - that a new discovery or test will not offer any new knowledge

   
   

How Scientists should respond to Falsification

1. 2 possible responses increase knowledge; scientists should respond with the one that maximizes knowledge (advances the frontier)
   1. Preserve the existing hypothesis by predicting new unknown facts
   2. Or throw out the old hypothesis and propose a new one that explains the old observations and the discordant facts in the new observations

   
   

2. Somewhat long-winded examples that have already been cited
   1. Uranus’ orbit - since Newton’s theory wasn’t invalid it must mean there is some data we don’t know about - the existence of a new planet - theory confirmed
   2. Efficient Market Hypothesis - when the facts didn’t fit the theory the theorists made up weaker versions of the original, reducing its falsifiability, but also removing its information content - Popper’s falsification immunization
   3. Later Einstein showed that Newton's laws were incomplete and his theory of general relativity modified Newtonian laws

   
   

3. Removing false ideas and replacing with more informative ones produces a body of knowledge that is self-correcting and continually improving

End result - the hypothetico-deductive method

1. Scientific method could be described as intelligent problem-solving
2. Five stages
   1. Observation - possible pattern or relationship
   2. Hypothesis - insight, past knowledge, inductive generalization
   3. Prediction - deduced from hypothesis - “if hypothesis is true then X should be observed if operation Y is performed”
   4. Verification - New observations are compared to the predictions
   5. Conclusion - inference about truth or falsity of the hypothesis based on degree to which observations conform to predictions

   
   

3. TA example
   1. Observation - S&P 500 above 200 day MA generally appreciates over several months (probabilistic generalization)
   2. Hypothesis - based on this observation, inductive generalization, prior findings of TA: upward penetrations of the 200 day MA by the DJIA/SP500 will on average produce profitable long positions over 3 months. Nickname this rule 200H
   3. Prediction - A backtest will be profitable. If 200H is true then the backtest will be profitable. However to avoid affirming the consequent we phrase it - ‘if the backtest is unprofitable, 200H must be false’.
      1. However our goal is to prove the truth of 200H so we formulate a null hypothesis at stage 2. ‘Upward penetrations of a 200 day MA does not generate profits over the following 3 months.’ Call it Null-200. From this we can formulate a conditional proposition.
      2. If Null-200 is true then a back-test will NOT be profitable.’
      3. Law of Excluded Middle - either Null-200 is true or 200H is true.
      4. Conditional syllogism:
         1. If Null200 is true, then the backtest will be unprofitable
         2. The backtest was NOT unprofitable
         3. Null 200 is false, therefore 200H is true

      
      

   4. Verification - back test and observe profitability
   5. Conclusion - determine the meaning of the results using statistical inference.

   
   
   

Evidence-Based Technical Analysis Chapter 3 (part 3 of 3)

At the end of this chapter we get one of the rewarding reveals that is actually just a punch to the gut for long-time technical analysts from the Alexander Elder/Magee & Edwards school. In the imagined words of Mr. T, "Time to change schools, fools!"

Rigorous and Critical Analysis of Observed Results

1. Evidence must be analyzed
2. Quantitative data is the evidence of choice
3. Statistical inference is the tool of choice for drawing conclusions from the data
4. Simpler explanations are preferable to complex ones (Occam’s razor)

If TA Were to Adopt the Scientific Method

1. Subjective TA would be eliminated
   1. By transforming into objective TA (non-trivial)
   2. By being abandoned

   
   

2. Elimination of meaningless forecasts
   1. Most market calls are meaningless - they don’t provide a timeframe, or the conditions under which the call could be considered wrong
   2. How much adverse movement would have to occur to admit the forecast is wrong?
   3. Ways to add objectivity:
      1. Define a future time to re-evaluate the forecast
      2. Define the maximum adverse movement allowed
      3. Predict the magnitude of favorable movement prior to an amount of unfavorable movement

   
   

3. Subjective forecasts are derived in an undefined way, so even with a profitable track record, it isn’t clear if it’s part of a consistent analysis procedure that can be reproduced in the future
   1. Studies of expert subjective judgment indicate that experts do no combine information in a consistent manner from one judgment to the next
   2. Intuitive judgments suffer from random inconsistencies due to
      1. Fatigue
      2. Boredom
      3. “Human factors”

   
   

4. Paradigm shift
   1. Aronson attacks the ‘intuitive analysis paradigm’ of pioneers like Dow, Gann, Shabacker, Elliott, Edwards, Magee
   2. And the CMT institute
   3. Many of the teachings of these practitioners will vanish like early Greek physics
   4. Some methods will survive the transformation to objectivity
   5. Aranson’s criticism should not be mistaken for a call to cut off all research into TA methods - for ex. Socionomics

   
   

Objectification of Subjective TA : An Example

1. Example derived from Chang & Osler’s head-and-shoulders pattern research
2. Pattern is usually displayed as a sequence of noise-free price swings
3. Subjective pattern definitions describe what patterns should look like but don’t explain how to distinguish pattern-like, but disqualifying pseudo-patterns
4. Definitions of subjective patterns are unclear
5. Solution - define objective rules

 

1. C&O defined the H&S pattern as five pivot or price reversal points that define the patterns three peaks and two troughs
2. Head must be higher than surround peaks (shoulders)
3. Other features were described inconsistently in subjective TA texts so they were excluded

 

1. Fractal scaling - an obstacle to defining peaks and troughs
2. C&O used an Alexander filter (zigzag indicator) to filter out noise from peaks/troughs identified using defined thresholds

 

1. The threshold amount causes a lag. For ex. If the price has to move 5% up or down for a peak/trough to be identified the time it takes to move that amount is lagged time
2. Different filter thresholds would identify different H&S patterns of varying sizes
3. C&O used 10 different zigzag filters using a range of threshold values to identify head-and-shoulders patterns on a variety of scales
4. Filter thresholds adjusted for volatility, defined as standard deviation of daily percentage price changes over the past 100 days
5. 10 thresholds were derived by multiplying V by 10 coefficients between 1.5 and 6.0
6. Practitioners were consulted to confirm that the zigzag filters did an acceptable job of identifying H&S patterns.
7. Next rules were defined to qualify candidate patterns
   1. Head must be higher than the shoulders
   2. Must be in an uptrend prior to formation of the H&S pattern
   3. Vertical and horizontal symmetry
   4. Time to completion
   5. Net result is a set of rules that qualify a pattern as H&S or NO

   
   

8. Vertical symmetry rules
   1. Exclude patterns with necklines that are too steeply sloped

   
   

 

1. Vertical symmetry rules continued:
   1. Price level of left shoulder peak, A, must exceed price level of point Y
   2. Price level of right shoulder peak, E, must exceed the price level of point X
   3. Price level of left trough, point B must be less than price level of Y
   4. Price level of right trough, D, must be less than price level of point X

   
   
   
   
   

2. Horizontal symmetry rules
   1. Head, at point C is roughly equidistant from the peaks representing the pattern’s two shoulders (A and E)
   2. Distance from one shoulder should be > 2.5 times the distance of the head to the other shoulder. 2.5 was an arbitrary choice.

   
   
   
   
   

1. Pattern completion rule: maximum time to neckline penetration
   1. Excludes patterns that take too long to penetrate the neckline once the right shoulder has formed
   2. Defined using internal proportions
   3. Max time allied from point E to neckline penetration = time distance between A&E. In other words the pattern must complete in as much time or less as it took for the shoulders and head to form

   
   
   
   
   

2. Look-ahead Bias
   1. It is possible for the prices to cross the neckline before the right shoulder is identified by the zigzag filter
   2. To combat this C&O required filter identification of the right shoulder pattern before entry

   
   
   
   
   

3. Many many more details later…
4. Does the H&S shoulder pattern carry predictive information?
   1. NO
   2. Worthless on stocks
   3. ‘Modestly’ profitable on currencies - in 2/6 tested
   4. Outperformed by a ‘much simpler’ objective signal based on zigzag filters
   5. H&S in conjunction with zigzag rule added no value
   6. On stocks H&S actually lost slightly more than trading random data
   7. C&O describe traders who use the H&S pattern as ‘noise traders’
   8. Lo et al used a different methodology to arrive at similar conclusions
   9. Bulkowski’s research differed but Aronson says his research ‘falls short’

   
   
   
   
   

Subsets of TA

1. Subjective TA - useless
2. Objective TA with unknown statistical significance - use methods in ch. 4,5,6
3. Objective TA that is NOT statistically significant - useless
4. Objective TA that is statistically significant - useful

https://lh3.googleusercontent.com/be...tMQq6vUeHrx0AX

EBTA Chapter 4 : Statistical Analysis (part 1)
If you thought things were a bit dense, academic and technical before, buckle up, buttercup. If your high school math days are long behind you and you weren’t really paying attention you can always brush up at sites like Khan Academy. But actually the math needed to understand this section is very basic.

1. Statistics is the science of data
2. H.G. Wells said an intelligent citizen in a 20th century free society would need to understand statistical methods
3. An intelligent practitioner of TA has the same need
4. Stats are not needed when a body of data conveys a clear message - for ex. Everyone drinking from the same well dying of cholera
5. Identifying which TA methods have genuine predictive power is “highly uncertain”
6. The essence of TA is statistical inference
7. Bertrand Russell’s famous quote - “Uncertainty in the presence of vivid hopes and fears, is painful, but must be endured if we wish to live without the support of comforting fairy tales.”
8. Jokes about statisticians - three go hunting, one shoots a foot to the left, the other shoots a foot to the right, the third exclaims, “we got it!”
9. Critics charge that stats are used to deceive - this is because powerful tools can be put to bad use (powerfully)
10. We seek the middle ground between blind distrust and blind gullibility - open-minded skepticism
11. This requires skillful data interpretation

A Preview of Statistical Reasoning

1. As discussed in ch. 3 start by assuming all TA rules have zero power
   1. Assume profitable back tests are due to luck
   2. The null hypothesis
   3. Refutable with empirical evidence
   4. If a TA rule has zero predictive power, its rate of return on detrended data will be zero
   5. However on small data samples the profitability of a rule can deviate considerably from zero as a result of luck
   6. Usually deviations in a useless rule are small but sometimes they are significant

2. Probability density function
   1. The best protection against being fooled by randomness
   2. Shows the degree to which chance can cause a useless rule to generate profits
   3. Density curve is centred at zero, reflecting the expected return of zero

1. If a rule’s profitability falls within the range of chance deviations the null hypothesis remains unchallenged and suggests the rule is useless

1. The farther the return rate lies at the extreme edge of the curve, the less likely the result is due to chance

1. When the observed performance is far enough above zero, the evidence is sufficient to refute the null hypothesis and to attribute predictive power to a rule
2. Important to distinguish that the evidence only speaks to the probability that the evidence could have occurred with the null hypothesis being true
3. In other words it speaks to the likelihood of the evidence, not the likelihood of the truth of the hypothesis
4. Recall from ch. 3 - evidence that a creature has four legs doesn’t prove it’s a dog.
   1. Four legs is consistent with the creature being a dog
   2. But insufficient to prove deductively that it is actually a dog
   3. Similarly, evidence of positive performance is consistent with a rule having predictive power
   4. But insufficient to prove that it actually does

5. Return to syllogisms
   1. If a rule’s expected return is equal to zero or less, then a back test should generate profits that are “reasonably” close to zero
   2. The back-tested performance was not reasonably close to zero; in fact it was significantly above zero
   3. Valid conclusion: Therefore the contention that the rule’s expected return will be zero or less is likely false

6. How unlikely or rare must positive performance be to reject a rule?
   1. No hard and fast rule
   2. By convention ‘most scientists’ would be ‘unwilling’ to reject (???) an hypothesis unless observed performance has a 0.05 probability or less of occurrence under the assumed null hypothesis. I think Aronson phrased this the wrong way around? Most scientists would be unwilling to accept any hypothesis with more than 5% chance of error.

7. Data mining
   1. So far we’ve only looked at single rules
   2. Computing power, back-testing software and plentiful historical data invite testing with many rules
   3. This is known as data mining
   4. Data mining is “an effective research method”
   5. However testing many rules increases the chance of discovering a lucky performance

1. The need for rigorous statistical analysis
   1. Informal data analysis is simply not up to the task of extracting valid knowledge from financial markets
   2. Data “blossoms with illusory patterns”
   3. Valid patterns are “veiled by noise and complexity”

An Example of Sampling and Statistical Inference

1. Statistical reasoning is abstract and runs counter to the ‘grain of common sense’ (intuition?)
   1. Good because it helps where ordinary thinking lets us down
   2. Bad because it’s hard to understand

2. Extrapolating from samples
   1. Sample of observations is studied
   2. Pattern discerned
   3. Pattern is expected to hold for observations outside the sample

3. Statistics - a new approach (Wallis & Roberts)
   1. Imagine a box filled with a mix of grey and white beads
   2. Total numbers of beads is unknown
   3. Figure the fraction of grey to white beads
   4. F-G (fraction-grey in box) is the actual fraction of grey to white beads in the ENTIRE box
   5. You are not allowed to open the box to look at all the beads (mirroring real-life examples where it’s impossible to see the whole population of observations)
   6. You can take samples of 20 beads at a time from the box
   7. Lowercase f-g stands for the fraction of grey beads in a sample (of 20 in this case)
   8. After sampling, the beads are put back in the box and it is shaken up giving each bead an equal chance of being selected - makes sure each sample is random
   9. Repeat the sampling process 50 times
   10. At the end of the procedure we have 50 different values for f-g, one for each sample

1. Since the beads are replaced each sampling, the F-G value remains constant
   1. This makes the problem stationary
   2. Financial markets ‘may indeed be nonstationary’
   3. Population means all the observations we are interested in.
   4. In this case the population is all the bead colours in the whole box
   5. Also keep the numbers straight - we took 50 samples of 20 beads each
   6. 50 = number of samples, 20 = the number of beads in each sample

Probability Experiments and Random Variables

1. Probability is the mathematics of chance
   1. A probability experiment is an observation of our environment that has an uncertain outcome
   2. Random variable is a quantity or quality observed in a probability experiment
   3. Random because it is affected by chance
   4. Individual observations of randomness are unpredictable, by definition a larger number of observations made on a random variable may have highly predictable features
   5. Example: coin toss
      1. A single toss could be heads or tails, you don’t know
      2. After 1000 tosses you can be reasonably sure heads will come up close to 500 times

2. Sampling: The most important probability experiment
   1. Extracts a subset of observations from a population
   2. Random variable in this case is a sample statistic
   3. f-g is an example of a sample statistic
   4. Sample should be chosen randomly
   5. Samples should be independent of each other

3. Back to the bead box experiment
   1. Imagine taking one bead with your eyes closed and then observing its colour
   2. This is a sample size of one
   3. The bead’s colour is the random variable
   4. Suppose the first sample had 13 grey beads out of 20
   5. The sample statistic f-g is 0.65 (13/20)
   6. If we stop now we have a very rough idea of F-G - too rough
   7. We do know something just from one sample though
      1. F-G is not 1.0 (all grey beads)
      2. F-G is not 0.0 (all white beads)
      3. We can’t rule out that F-G is something low like 0.1 but that is very unlikely, same with a high value like 0.95

   8. A sample is always a partial representation of the population, even if we take numerous samples there is always some uncertainty
   9. More samples = more knowledge about F-G
   10. After measuring 49 more samples we notice
       1. Every sample varies unpredictably, some will have higher values for f-g, some lower
       2. This is called sampling variability or sampling variation
       3. This is what’s responsible for the uncertainty in statistical conclusions

1. The value of f-g isn’t entirely determined by chance - it is ‘strongly influenced’ by the value of F-G
   1. Each observed value of f-g is the result of two influences
      1. The underlying composition of F-G
      2. Randomness in sampling
      3. F-G acts like a ‘center of gravity’ always keeping f-g within a certain range
      4. From sample to sample, the value of f-g oscillates randomly sound this center of gravity

   2. If the sample was larger -say 200 beads, the amount of uncertainty would drop
   3. The random variation in f-g would be smaller - in fact, about ⅓ of the 20 bead sample (not 1/10 as intuition would provide)
   4. This effect derives from the Law of Large Numbers - large samples reduce the role of chance
   5. One of the most important principles in statistics

2. Frequency distribution of the sample statistic f-g
   1. 21 possible values for the random variable f-g (0.0 to 1.0 in jumps of 0.05 because there are 20 in each sample)
   2. Frequency distribution plots the occurrences of f-g’s possible values

1. Equivalence of frequency and area
   1. Frequency distributions show the relationship between the frequency of a particular value and the area covered by the bars
   2. Summing the counts of each bar we see they add up to 50 - the total number of samples
   3. This works for any fraction of the distribution’s area
   4. A single bar’s area / Total area = fraction of the total observations represented by that bar
   5. Example
      1. Vertical bar with f-g value 0.6 shows a count (frequency) of 10
      2. 10/50 = 20% of the observed values
      3. This bar’s total area of the graph is also 20%
      4. See figure 4.7

1. This idea might seem trivial or obvious but it’s central to statistical reasoning
2. We will use the fractional area of a distribution to measure the probability that the back-test profits of a rule could occur by chance
   1. If the probability is small the rule likely has predictive power

3. Relative frequency distribution of f-g
   1. Relative frequency - a bar’s height = number of observations divided by the total number of observations
   2. Example
      1. F-g was 0.6 in 10/50 observations
      2. 0.6 has a relative frequency of 10/50 or 0.2
      3. The distributions bar would be drawn to a height of 0.2 on the vertical scale
      4. See 4.8

1. Frequency = area in relative frequency tables just as it does in ordinary frequency distributions
   1. Relative frequencies add up to 1.0 (100% of the observations)
   2. Relative frequency of any bar or group of bars is = to their portion of the distribution’s total area
   3. An f-g value of 0.65 or larger = 0.1 + 0.06 + 0.02 + 0.02 = 0.2
   4. The relative frequency of f-g values >= 0.65 is 0.20

1. What have we learned from sampling?
   1. We started with an f-g of 0.65
   2. After more samples the f-g value coalesced around a central value
   3. This central value will be close to F-G but remain uncertain due to variation
   4. It’s reasonable to conjecture F-G lies between 0.4-0.65

2. A second box of beads (F-G 2)

https://lh5.googleusercontent.com/PP...LMvGGUjeGYhhRc

1. The general shape of the distribution is similar to box 1
2. The central value is different - box 1 centered around 0.55, box 2 is around 0.15

https://lh3.googleusercontent.com/Ue...p_tyW5_WG2HK9G

1. From this we can conclude that box 2 has fewer grey beads than box 1
2. The degree of random variation between box 1 and 2 is different
   1. Box 2 has less variable results as defined by its narrower clustering about a central value
   2. Thus we can say we know F-G 2’s value with greater certainty than F-G’s

3. What we learned
   1. As Yogi Berra (maybe) said ‘you can observe a lot just by looking’.
   2. Even though samples are just a portion of the universe (population) it can teach us a lot about the composition of the whole
   3. But there are limits imposed by randomness on what we can know

Up next : statistical theory

(Google Doc (with footnotes - often brief but trenchant critiques)

Evidence-Based Technical Analysis by David Aronson
Chapter 4 (part 2)
The maths get a little denser now, but still manageable, and not vital to understanding. Segments of this chapter (and the rest of the book) get bogged down in defending the science of the methodology rather than defining and supporting the methodology itself, but (spoiler alert) since the conclusions are a bit of a letdown anyway, this is actually the book's 'golden age'. Try to enjoy the ride especially if, like me, you weren't a natural whiz at school.

Statistical Theory

1. The 6 elements of a statistical inference problem
   1. Population
      1. All the possible observations of a random variable
      2. Large (usually)
      3. Possibly infinite
      4. In the box of beads experiment the population was the set of beads filing the box
      5. The bead colour was the random variable
      6. For a TA Rule the population would be all the conceivable returns that could be earned by the signal over the immediate practical future (IPF)
         1. Since it’s unreasonable to expect financial markets are stationary it’s also unreasonable to expect that profitability of a rule will endure forever
         2. IPF - a finite future time period over which it would be reasonable to expect a rule’s usefulness to persist; this has to be assumed otherwise all TA is pointless
         3. IPF has to be long enough for a researcher to profit by it, to compensate them for the time taken

         
         

      
      

   
   

1. A sample consisting of a set of observations, randomly selected from the population
   1. A subset of the population
   2. In the box of beads - 50 independent samples were taken
   3. In TA we backtest a rule over a segment of market history

   
   

2. A population parameter
   1. An investigated fact or characteristic about the population, usually numerical
   2. In the case of a TA rule this is usually the rule’s expected performance over the IPF
   3. In some statistical problems it’s safe to assume this parameter doesn’t change - for ex. the box of beads experiment
   4. Imagine if a malicious demon was replacing or adding beads of different colours between samplings - then the parameter would be changing (unbeknownst to the experimenter)

   
   

3. A sample statistic
   1. A measurable attribute of a sample.
   2. Known through observation
   3. Usually the same measurable attribute as the population parameter
      1. In the box of beads the sample stat, f-g, is the fraction of grey beads in an any sample
      2. F-G is the pop. parameter

      
      

   4. Usually in financial matters it’s restricted to numerical values
   5. Examples:
      1. Proportions
      2. Percentages
      3. Standard deviation
      4. Average rate of return
      5. Trimmed average

      
      

   
   

4. An inference
   1. Inductive leap based on observed value of the sample stat (known for certainty but true only for a sample of data) to the population parameter (uncertain but theorized to hold true for wider number of unobserved cases)
   2. If a rule’s positive performance is too high to be reasonably attributed to luck it can be inferred that the rule has genuine predictive power

   
   

1. A statement about the inference’s reliability
   1. The science of statistics goes beyond admitting that its inferences are uncertain
   2. The uncertainty is quantified
   3. Inferences can be wrong in two ways
      1. Good luck shining on a worthless rule - concluding that a rule has power when it doesn’t
      2. This will translate into taking market risks that won’t be compensated
      3. Bad luck obscuring a good rule - concluding a rule has no value when it does
      4. Translates into missed opportunities

      
      

   
   

Descriptive Statistics

1. Statistics can be subdivided into two main areas
   1. Inferential - most important for TA - covered already
   2. Descriptive
      1. Descriptive statistics are useful to describe sample data
         1. Goal is data reduction
         2. Reduce a large set of numbers to a smaller more intelligible set

         
         

      2. Three descriptive tools are important for our work
         1. Frequency distributions
         2. Measures of central tendency
         3. Measures of variation

         
         

      
      

   
   

2. Frequency Distributions
   1. We saw one already with the box of beads experiment
   2. Reduced 50 observations of f-g into one plot diagram
   3. Often the first step in analyzing a data set
      1. Quick visual impressions of
         1. Central tendency (average values)
         2. Degree of dispersion about a central value
         3. These values also need to be quantified

         
         

      
      

   
   

3. Stats that measure central tendency
   1. 3 examples
      1. Average
         1. Aka arithmetic mean
         2. Common summary statistic
         3. Sum of observed values divided by number of observations
         4. Population mean and sample mean are not the same thing

         
         

      
      

   
   

1. 1. Median
   2. Mode

   
   

2. Statistical measures of variability (dispersion)
   1. Describes degree to which observations in a sample are dispersed about their central tendency
   2. Quantifies the width of a frequency distribution
   3. 3 examples
      1. Variance
      2. Mean absolute deviation
      3. Standard deviation
         1. Square root of the average squared deviation of each sample from the mean of the data

         
         

      
      

   
   

 

1. The point of figure 4.16 is that central tendency and variation are independent characteristics of a frequency distribution

Probability

1. Quantifies uncertainty
2. The chance of a conclusion reached via statistical inference is susceptible to error
3. The chance of an error is given in terms of probability
4. We routinely make rule-of-thumb calculations about probability in everyday life and we call it vague things like risk, luck, likelihood, chance, uncertainty
5. For our work we need a more scientific definition of probability based on relative frequency
6. Relative frequency
   1. The number of times an event happened divided by the total number of chances the event could have happened
   2. Given as a fraction between 0 and 1
   3. Example
      1. On any given day it could rain
      2. But in April it rained only 11 out of 30 days
      3. The relative frequency of rain in April is 0.366
      4. If it never rained the relative frequency would be zero
      5. If it rained every day it would be 1.0

      
      

   4. In the box of beads f-g was measured 50 times
      1. F-g = 0.65 happened five times
      2. So it had a relative frequency of 5/50 or 0.10

      
      

   5. Relative frequency is always a theoretical notion because we cannot observe an infinity of observations; however with a large number we can approach theoretical probability

   
   

7. Law of Large numbers
   1. Relative frequencies tend to converge to theoretical probabilities as the number of observations increases
   2. A lot of explanation using coin toss experiments, but dude, you can stop flexing, we believe you

   
   

8. Theoretical versus empirical probability
   1. Theoretical probability
      1. Determinable with good degrees of confidence purely on logical grounds
      2. Independent of prior experience
      3. Derived most often on “arguments of symmetry”
      4. Example
         1. Coin toss
            1. No need to toss it 10,000 times to know its long run probability of heads is 0.5
               1. Coin is fair
               2. It has two sides
               3. There’s no reason to think one side would appear more often than the other and there’s only two options
               4. Won’t land on its edge

               
               

            
            

         2. Chance of rolling six on a die
         3. Chance of winning the lottery
         4. Chance of getting a royal flush in poker

         
         

      
      

   2. Empirical probability
      1. Based on observed frequencies
      2. Examples
         1. Technical Analysis
         2. Chance of snowfall somewhere
         3. Chance of a stock market rise after a 2% drop in interest rates

         
         

      3. Must be characterized by the same set of conditions for each observation
         1. Impossible with Technical Analysis
            1. Past instances of a 2% drop in interest rates will be similar in that one respect (interest rates) but many other conditions will be different
            2. Even the interest rate before the drop might be different
            3. We could add more conditions to the comparison but this has the effect of reducing the number of comparable observations, and it’s impossible to deal with all conditions
            4. An unfortunate fact of life with nonexperimental/observational sciences

            
            

         
         

      
      

   
   

Probability distributions of random variables

1. Aka a probability density function
2. Shows how often we can expect relative frequencies
3. Shows how often we can expect different possible values of a random variable
4. Probability distribution of a random variable = relative frequency distribution built from infinite observations
5. If we take a sequence of relative frequency distributions, building on it with increasing observations, and progressively narrower bins or intervals, gradually the relative frequency distribution smooths out and becomes a probability distribution

1. It only makes sense to speak of the probability of a random variable within a range of values, since the odds of a specific single value of a continuous random variable occurring is zero (like a point on a plane occupying no space)

As Marty McFly said to Doc Brown, this is getting heavy.
Let’s recap:

1. The probability of an event = its relative frequency within an infinite number of chances that it could occur
2. A probability density function is a relative frequency distribution built from an infinite number of observations using intervals (graph bars) of zero width
3. The relative frequency of a random variable within a given internal is equal to the fractional area of the frequency distribution sitting on top of that interval (see fig. 4.24)
4. The probability that a continuous random variable will assume a value within a specified range is equal to the fraction of the probability density function encompassed by that range (see fig. 4.25)

When a probability distribution’s interval refers to the extreme end of a distribution we say it’s in the ‘tail’ of the distribution. See fig. 4.26
https://lh6.googleusercontent.com/X_...BQXLGwfJWlWD59
Why all this focus on probability distributions? In chapter 5 we’ll show that an hypothesis represented by a probability distribution of a random variable, using an observed value of the random variable, can test the truth of that hypothesis.
Next: sampling distributions, to which Aronson devotes many pages and for which he maybe didn't need to.

Evidence-based FX chapter 4 (part 3 of 3)

The Sampling Distribution

1. Shows the various possible values the sample statistic can assume with their associated probabilities
2. In rule back testing a sample statistic is a measure of performance observed in a back test. In ‘this book’ it will be the rule’s average rate of return
3. The pattern of random variation is well-behaved, which makes stat inference possible
4. Certainty is directly related to the dispersion of the sampling distribution
5. Example of a rule whose hypothetical expected return is equal to zero
   1. Rule’s back-tested return is greater than zero
   2. Is this sufficient to conclude the hypothesis is wrong?
   3. Depends on how far above zero the backtest is relative to the width of the sampling distribution
   4. If relative deviation is large, reject hypothesis

   
   

6. Sampling distribution of trading performance
   1. Confined to mean sampling distribution because ‘this book’ uses the average rate of return for all rules
   2. However there are other performance measures
      1. Sharpe Ratio
      2. Profit Factor
      3. Mean return over Ulcer Index
      4. Each of their sampling distributions would be different from the mean

      
      

   3. Methods used in ‘this book’ for generating sampling distributions might not work well for performance stats with ‘elongated right tails’. This includes ratios like the Sharpe, mean-return to Ulcer, and the profit factor. This problem can be ‘mitigated’ by taking the log of the ratio to shorten the right tail. (???) Really, he spends half the book explaining the importance of sampling distributions but then breezes right over this little issue of the long right (not left??) tails? What about skewness?

   
   

7. The 3 Distributions of Statistical Inference
   1. The Sampling Distribution -an infinite sized distribution of the sample statistic - a rule’s mean return
   2. Data distribution of the population - an infinite sized distribution composed of all possibly daily rule returns within the IPF
   3. Data distribution of the sample - a distribution of a finite number (N) of past daily rule returns

   
   

Imagine a population distribution of daily rule returns that is infinite in size. Take 50 independent samples from the population. Determine the mean return for each sample. Then plot the sampling distribution of the 50 mean averages. That’s what’s represented by the above diagram (4.29). You can see the 3 different types of distribution on each level of this diagram.

 

1. The real world - the problem of one sample
   1. In most real-world stat problems, there is only one sample of observations, and one value for the sample mean
   2. This gives us no notion of the sample statistic’s variability (shown in figure 4.3) Note that the sample mean doesn’t jive with the population mean

   
   

2. Solution
   1. 20th century math inventions gave us two powerful solutions to this problem
      1. One classical
      2. One computer Intensive (used for the rule tests in ‘this book’)

      
      

   2. This is what makes the field of statistical inference possible

   
   

Deriving the Sampling Distribution - Classical Method

1. Most often taught in courses
2. Attributed to Fisher & Neyman
3. Uses probability theory and integral calculus to derive the sampling distribution on the basis of a single observed sample
4. Provides estimates of the sample distribution’s dispersion, its mean, and its shape
5. Quantifies the reliability of an inference based on a single sample (!)
6. Sampling distribution of the sample mean
   1. The mean of a large sample is a good estimate of the population mean because of law of large numbers
   2. The dispersion of the sampling distribution depends on the size of the sample - the larger it is the less dispersion

   
   

1. 1. The dispersion also depends on the amount of variation within the data that makes up the parent population
   2. The shape of the sampling distribution tends toward normal (bell-shaped) because of the central limit theorem

   
   

2. Normal distribution
   1. Described completely by its mean and its standard deviation
   2. ~68% of all observations lie within one standard deviation
   3. ~95% of all observations lie within two standard deviations
   4. The tails of the distribution get very thin beyond two standard deviations, so values beyond three standard deviations are rare, values beyond four are extremely rare
   5. A fundamental feature of the natural world

   
   

1. Standard error of the mean
   1. Another name for the standard deviation of the sampling distribution
   2. Directly related to the population from which samples are taken
   3. Inversely correlated to the square root of the sample size
   4. Equal to the standard deviation of the population divided by the square root of the sample size
   5. Standard deviation of the population can be estimated from sampling
   6. The estimate is designated by ‘sigma-hat’ as shown in this equation

   
   

1. If we have the standard error of the mean and we are right that the sampling distribution is ‘normal’ then we can estimate the sampling distribution of the mean based on one sample

Since we might not be sure that the distribution is normal, there is another approach that uses computer simulation

Deriving the sampling distribution with the computer-intensive approach

1. By systematically resampling the single available sample many times, it is possible to approximate the shape of the sampling distribution
2. Two methods of doing this
   1. Bootstrap resampling
   2. Monte Carlo permutation

   
   

3. Will be used to determine the benchmark for evaluating rule performance

Next chapter: confidence intervals

Evidence-Based Technical Analysis Chapter 5 : Hypothesis Testing and Confidence Intervals (part 1)
You’ve got to have lots of confidence to be a successful trader, har-har. Just a little math dad-joke before we get pummelled by pedantic facts.

If you're feeling a bit overwhelmed (as I was on first reading since statistics/probability are not my background) take heart for we are nearly halfway through. Also don't feel that everything should make sense right away. Sometimes it takes multiple readings and some practical experience with the concepts to really 'grok' them. And lastly even if you don't master the mechanics, the ultimate conclusions are easy to understand; it's just important to have a factual basis behind your acceptance/rejection of the arguments.

Questions are always fine, and I'll do my best to answer them, if I can.

Two Types of Statistical inference

1. Hypothesis testing
   1. Concerned with the unknown value of a population parameter
   2. Determines if a data sample is consistent with an hypothesis
   3. Evaluates the truth
   4. Relies on the method of indirect proof - establishes truth by proving the falsity of something else

   
   
   
   

2. Parameter estimation
   1. Concerned with the unknown value of a population parameter
   2. Uses information from a sample to determine the value
   3. Establishes a range of plausible values

   
   
   
   

3. Both attempt to draw conclusions about populations based on samples

Hypothesis Tests vs Informal Inference

1. If a trading rule is profitable we have to decide if it was because of the rule or just luck. The hypo test does this
2. Confirmatory evidence
   1. Necessary
   2. Nice to have
   3. But as we saw in Ch. 2 informal inferences are biased in favor of it (as opposed to denying evidence)
   4. Confirmatory evidence is necessary but not sufficient (recall the 4 legged dog examples)

   
   
   
   

3. Statistical hypothesis
   1. A conjecture (educated guess) about the value of a population parameter
   2. If we’re talking about trading rules we default to an assumption that the value of the rule is zero (null hypothesis)
   3. If the value of the sample stat is close to the hypothesis we assume it’s correct, if it’s far off we assume that it’s inccorect BUT
   4. ‘Close’ and ‘far’ are ambiguous terms - instead we use a range between 0.0 and 1.0

   
   
   
   

4. Falsifying a hypothesis with improbable evidence
   1. We start by assuming the hypo is true (the rule is worthless)
   2. We deduce predictions about the likelihood of new observations (rule’s return will be zero or close to it)
   3. Compare predictions with new observations
   4. If they agree, null hypothesis is confirmed
   5. If they disagree (low probability outcomes are observed) the hypothesis is in question, falsified and a new hypothesis (alternative hypthesis) generated
   6. The trick is to decide if the variance between observed results and hypothesized results was big enough

   
   
   
   

1. Why do we start by assuming the null hypothesis is true?
   1. Scientific skepticism
      1. Science is skeptical towards all new claims of knowledge
      2. Justified because making claims is easy, proving them is hard
      3. The burden of proof is on those making new claims

      
      
      
      

   2. Principle of simplicity (parsimony)
      1. Claims that simpler theories are more likely to “capture true patterns” than complex ones. Recall Occam’s Razor.
      2. Simpler theories are less likely to fit data by chance
         1. Example of a complex function with several coefficients, versus a simple slope with 2 coefficients
         2. The simple function is more likely a description of the real relationship without the random fluctuations of the complex function

         
         
         
         

      
      
      
      

   
   
   
   

Hypothesis Testing : The Mechanics

1. 3 ingredients of an hypothesis test
   1. An hypothesis
   2. A test statistic (sample statistic used to test the hypothesis)
   3. A means of generating the sampling distribution of the test statistic

   
   
   
   

2. With regards to rule testing those three become:
   1. The hypothesis - H₀ - that the rule has an expected return of zero or worse
   2. The test statistic - the rule’s mean return obtained with backtesting on historical data
   3. The sampling distribution - the random variation in the rule’s mean return after many independent tests (samples). Centred at a mean return of zero reflecting the assumption of H₀.

   
   
   
   

3. Is the test statistic improbable?
   1. The basic idea of an hypothesis test is simple. If it’s true then it’s unlikely you’ll see an outcome that proves it false
   2. Suppose a TA rule called MA50 is back-tested
      1. MA50 rule says if S&P 500 close is greater than 50 day MA, hold a long position in the S&P500, otherwise hold short
      2. Alternative hypothesis Hₐ says the rule has predictive power and is expected to earn a return greater than zero on detrended data
      3. The null hypothesis, H₀ says otherwise

      
      
      
      

   Attached Image

   
   
   
   
4. MA50’s mean return is plotted on the same axis as the hypothesized population parameter (fig. 5.6)
5. There is positive deviation, but is it surprising enough to reject H₀?
6. The positive deviation could be due to sample error (luck) or because H₀ is wrong.
   1. To determine which it is, the observed average return, and its deviation from the hypothesized value is compared to the sampling distribution.
   2. Recall that the sampling distribution gives the probability for various deviation sizes between the observed value of the sample statistic and its expected value due to sampling error
   3. If the observed value’s deviation is > reasonable sampling error then H₀ is rejected and Hₐ is adopted.
   Attached Image

   
   Attached Image

   
   
   
   

These (confusing) diagrams show one result that fails to beat the null hypothesis (5.7) and one that does (5.8). In each case the width of the sampling distribution is crucial.

P-Value
The observed deviation is quantified as a probability, a conditional probability since it is dependent on some other fact being true.

 

1. In an hypothesis test this conditional probability is given a special name - the p-value.
   1. The smaller the p-value, the greater is our justification for questioning the validity of the null-hypothesis
   2. If the value is less than a predefined threshold, then Hₐ is adopted over H₀
   3. The p-value is also a measure of the chance that that H₀ will be erroneously rejected in favour of Hₐ.
   4. Equal to the fraction of the sampling distribution’s total area that lies at the values equal to or greater than the observed value of the test statistic

   
   
   
   

2. In the context of a rule test:
   1. If a rule’s return on backtest was +3.5%, we mark that on the horizontal axis of the sampling distribution
   2. Then determine the fraction of the sampling distribution’s area with values >= to 3.5%.
   3. Suppose that area = 0.1 of the total area, then the p-value = 0.1
   4. This is equivalent to saying if the rule’s true return was zero, there is a 0.1 probability that its backtested value would reach as high as 3.5%+ due to chance

   
   
   
   

P-value, Statistical Significance and rejecting the Null-Hypothesis

1. Another name for p-value is statistical significance (of the test)
2. The smaller the p-value, the more statistically significant the result
3. The more surprising (improbable) an observation is in relation to a given view (hypothesis) the more likely the established view is false
4. Common / standard values:
   1. P-value = 0.1 “possibly statistically significant”
   2. P-value = 0.05 <= “statistically significant”, the cutoff for rejecting H₀ usually
   3. P-value = 0.01 <= “very significant”
   4. P-value = 0.001 <= “highly significant”

   
   
   
   

5. Test Conclusions and Errors:
   1. Type 1 error : A low p-value leads us to reject H₀ when it is actually true. “Fooled by Randomness”
   2. Type 2 error : a high p-value causes us to retain H₀ (reject the alt hypothesis) when it is actually false, in other words the rule actually does work even though our test convinced us it doesn’t.

   
   
   
   

1. “Only God knows for certain” at the time of the test if there is an error and which type
2. Practical Consequences:
   1. Type I error : we invest in a rule that is actually worthless and lose money
   2. Type II error: we miss a profit-making opportunity
   3. As we know from past readings, managing risk, avoiding losses is more important than maximizing profits and exploiting opportunities, so Type I errors are far worse.

   
   
   
   

Next: methods for generating sampling distributions
Google Doc with footnotes