Nah, I'm using VS Code which can use .py or .ipynb files. I think it was just a dataframe issue where I reprocessed it too many times. I think I know what's going on here. My price data for USD/JPY comes from Yahoo Finance and my new feature's data ...

Diary of a Pro Quant

I added a new feature and the results improved significantly. Here are the metrics version 2 produced on the test set: image That appears to be a great system, but I'll admit I'm skeptical. As they say, if it seems to be too good to be true it ...

Diary of a Pro Quant

I moved the strategy from a python file to a Jupyter notebook and the results improved. image Of course, the results didn't really change as everything runs exactly the same. However, something could have been wrong with the dataframe I originally ...

Diary of a Pro Quant

I build a rough classification model to predict whether tomorrow will close higher or lower for USD/JPY. The results were exceptionally poor with an accuracy rate of 54%. The CAGR was -.50 and the Sharpe was just .20... Only up from here!

Diary of a Pro Quant

Thanks yoriz, I always appreciate your feedback! I'll definitely add performance metrics into an strategy I produce. ML classification actually lends itself very well to vectorized backtesting. So I'm about to start on a new script... I'm debating ...

Diary of a Pro Quant

A few closing thoughts for tonight: My favorite model right now is still XGBoost. It didn't have the best performance in the ISM example, but extreme gradient boosting should lead to better/quicker results. I read CatBoost (category, not the animal) ...

Diary of a Pro Quant

Added some more key metrics, sorted by Mean Squared Error this time: ARDRegression() Mean Squared Error: 2.421 Root Mean Squared Error: 1.556 Mean Absolute Error: 1.191 Median Absolute Error: 1.087 Max Error: 3.492 R2 Score: 0.916 HuberRegressor() ...

Diary of a Pro Quant

Reran it with a couple of new metrics, now sorted by median absolute error: GradientBoostingRegressor() Mean Absolute Error: 1.277 Median Absolute Error: 0.775 R2 Score: 0.877 XGBRegressor() Mean Absolute Error: 1.294 Median Absolute Error: 0.93 R2 ...

Diary of a Pro Quant

I revised my ISM script and basically added every Scikit Learn linear regression model to see how they all compare. Here were the results, with each respective model and its test Mean Absolute Error. LinearRegression - 1.179 Lars - 1.179 Ridge - ...

Diary of a Pro Quant

Good points yoriz! I did some more research and it sounds like Random Forest and XGBoost are different enough to where I should keep both. I'm going to bring in some new models too, at least temporarily. In python, introducing new models doesn't ...

Diary of a Pro Quant

Yes, and I think a drawdown in the 60% range is probably a mistake. Imagine the psychology of riding a sinking ship that far down before coming back up. A real practical downside of that kind of drawdown is its very unforgiving. If you have a ...

What is the best Drawdown?

It really depends on the trader and strategy. Professionally managed funds tend to shut down after about a 20% loss. That said, if I recall correctly the turtle traders could have >60% drawdowns and come back so there's no finite rule.

What is the best Drawdown?

The data came in at 0.2% and 3.0%. My M/M models all nailed it, with the exception of XGB. XGB is still probably the best model though so I'm not reading much into this. The Y/Y models all missed this one, but XGB came close along with Random ...

Diary of a Pro Quant

By the way, this is all for fun and practice at the moment. I don't expect any of these ML models to perform well yet. I've got all the scripts so it's very little work to add/remove features, rescale the data, tune parameters, etc. It may not be a ...

Diary of a Pro Quant

I put together some ML models tonight for CPI. They use the prior CPI, average hourly earnings, import prices, WTI, and gasoline as features. The FF calendar has 0.3% and 3.1% for the MM and YY forecasts. M/M, my models came up with an average ...

Diary of a Pro Quant

Thank you! I preprocess much of my data and sometimes normalize it. It hasn't made as much difference as I initially expected, but I'm still newer to ML. For hyperparameter tuning, I actually just ran a for loop over a few critical parameters and ...

Diary of a Pro Quant

So, from the 5 models I've used XGBoost really should be the best. I did some basic hyperparameter tuning and the model didn't actually improve much. For ISM, the MSE went from 1.29 to 1.25. Better, but I had hoped for more. I'm not sure if it's ...

Diary of a Pro Quant

Happy Sunday! What a great time to do some hyperparameter tuning. I was researching the tradeoffs of grid search vs random search and it sounds like random search is probably the better option. Apparently grid search is better in a sense that it ...

Diary of a Pro Quant

Thanks for chiming in yoriz! You're 100%, historical data of XYZ alone won't have much predictive value apart from a possible momentum or mean reversion factor. That's a part of it sure, but insufficient really so I always try to bring in any other ...

Diary of a Pro Quant

It came in at 209k, which was pretty far off the best model at 255k. That was expected, the question is how to improve it... My initial thoughts for new features: Sum of previous 4 weeks of Initial Jobless Claims Challenger Job Cuts PMI data, ...

Diary of a Pro Quant