Wish I had seen the thread so I could show him thisewok55 wrote:On January 9, 2018 I actually reported the same "experiment" in a message to 12squared:Aitrus wrote:So, I decided to do an experiment. I would pretend that I was in December 2004 and was planning my strategy for 2005, and all I had was 2004 data and the tspcalc website. I started with 2004 and pretend I had only that year’s data to work with for coming up with the strategy for 2005. Then use 2004 and 2005 for the 2006 strategy, use 2004 - 2006 for 2007, and so on.
I used my personal preferences and risk tolerance for investing as if I were to use the daily strategies on tspcalc. This means I set the StdDev to 10%, and 0% time in the I Fund. I would use whichever strategy had the highest mean, had less than 10% StdDev, and had no time in the I Fund.
This is what I came up with.
For 2005: #5523, earning 16.01%, vs C (4.96%) and S (10.45%)
For 2006: #27482, earning 25.73%, vs C (15.79%) and S (15.30%)
For 2007: #27482, earning 21.59%, vs C (5.55%) and S (5.49%)
For 2008: #27482, earning 24.20%, vs C (-36.99%) and S (-38.32%)
For 2009: #21024, earning 41.61%, vs C (26.67%) and S (34.85%)
For 2010: #19323, earning 37.14%, vs C (15.06%) and S (29.06%)
For 2011: #19323, earning 52.43%, vs C (2.11%) and S (-3.38%)
For 2012: #21024, earning 33.06%, vs C (16.07%) and S (18.57%)
For 2013: #21024, earning 31.15%, vs C (32.45%) and S (38.35%)
For 2014: #19323, earning 29.62%, vs C (13.78%) and S (7.8%)
For 2015: #19323, earning 25.23%, vs C (1.46%) and S (-2.92%)
For 2016: #19232, earning 27.2%, vs C (12.01%) and S (16.35%)
For 2017: #27117, earning 28.48%, vs C (21.82%) and S (18.22%)
For 2018: #27117
It was only afterwards, however, that I learned the strategies I used were made by people in 2017 with knowledge of the results (i.e., a posteriori, not a priori as I had thought). In other words, that strategy you used to predict 2005 based on 2004, was actually made by somebody in 2017 or 2018 trying to fit the data.Per our discussion about the TSP Calculator for daily seasonal strategies, I ran an a priori simulation where I set parameters at Mean >30 and Std <10 and used the top performing Model for the next year. For example, I used the top performing model in 2004 to predict 2005, the top performing model from 2004-2005 to predict 2006, the top performing model from 2004-2006 to predict 2007, and so on until I used 2004-2016 to predict 2017. Below are my results, but they're very promising. I'm toying with other similar methods to choose a model (i.e., using the previous 5 years to predict the next year). They all seem to give average returns around 29-30%. It seems too good to be true, but I guess we'll see after this year!
Thanks again for your reply to my previous post, and good luck!
Year Predicted, Plan Used, Return
2005, 22945, 24.84%
2006, 16753, 33.87%
2007, 19012, 8.91%
2008, 17836, 29.89%
2009, 19861, 43.73%
2010, 19861, 34.83%
2011, 17434, 39.20%
2012, 19855, 33.82%
2013, 17333, 39.49%
2014, 17333, 17.96%
2015, 19858, 24.36%
2016, 19858, 31.62%
2017, 17414, 26.45%
A better experiment, in my mind, was when KWG_5 used the best strategies from TSPCalc to predict performance from years not in the database (I believe he used 1988 to 2003; his post can be found at here). Results of that experiment found performance slightly below the S&P fund. Why? I think it's because the "best" models in TSPCalc were made without knowledge of what happened in 1988 to 2003, which could be simulating what is happening now in 2018 (and likely in future years).
Very excited, nonetheless, to see the models that Mjedlin66 is working on. This will go a huge way in being able to test models based solely on their previous year's performance, without the confounding variable of a strategy-creator's future knowledge.
https://www.portfoliovisualizer.com/backtest-portfolio