Do standard deviations even tell us anything on tspcalc.com?

[KWG_5]

First off, I think tspcalc.com is awesome. With my own TSP account I follow a variation of a daily strategy. However, I'm not sure it's valid to assume we can use standard deviations to make assumptions about future returns. Here's why, standard deviations only work when the data have a normal distribution (i.e. it looks like a bell curve).

My fear is that as we manipulate strategies to minimize standard deviations and maximize average returns we are not creating a normal distribution and therefore standard deviations can't tell us what percent of future data points will fall within x number of standard deviations.

I'm posting this in the form of a question because it's been a long time since I took statistics and I'm not a math major. I just don't think there's any way that strategies based on 14 years of returns have essentially a zero percent chance of having a negative return. I've read that the stock market has a normal distribution (and most things do), but I don't know if the strategies we make up do.

[michigande]

I've noticed a variation of this question asked a number of times and have given it some thought. You ask it more thoughtfully than I've seen before though.

Full disclosure, I too muddled my way through college statistics and quantitative analysis (holy crap was it hard), but do recall the fundamentals. Academically, as I understand it, a minimum sample size to start running these sort of standard deviation calculations and so on, is 30... for reasons I don't recall, but it's 30. And yet people are basing gambles, HUGE gambles on statistics that are literally only half baked (14 year of data to look at).

The way I see it is this, we're not on a college campus and don't have 30 years of sampling data to work with. We have what we have and we make informed decisions on imperfect data, which is far better than a decision on no data.

Speaking only for myself, if the "standard deviation" for the past 14 years, experiencing election cycles, jobs reports, bull markets, bear markets, occupy wall street and the realization that high fructose corn syrup is bad for you, has changed little, I believe it *probably* won't deviate too far from the norm this year either... probably. And that's the calculated risk, as I see it.

Smart question though. Not sure I answered it in any satisfactory way, but smart question. And as always, people more learned in these things please shoot holes in my logic!

[mjedlin66]

Under a normal distribution, 95% of the data points will fall within 2 SD.

These shortcomings of only 14 data points only mean that the confidence is not as high. So maybe we will experience 80% within 2 SD.

But more importantly, the standard deviation still works well as a comparison value. So when comparing one strategy to another, they both face the same data shortage. So the effect of reduced confidence is the same. Therefore, we can still make the conclusion that a 2% Standard Deviation is better than 3%, regardless of how many data points we have.

[evilanne]

This topic has pretty good discussion on standard deviations & risk http://tspcenter.com/forums/viewtopic.php?f=14&t=14940

[cswift01]

First off, I think tspcalc.com is awesome. With my own TSP account I follow a variation of a daily strategy. However, I'm not sure it's valid to assume we can use standard deviations to make assumptions about future returns. Here's why, standard deviations only work when the data have a normal distribution (i.e. it looks like a bell curve).

My fear is that as we manipulate strategies to minimize standard deviations and maximize average returns we are not creating a normal distribution and therefore standard deviations can't tell us what percent of future data points will fall within x number of standard deviations.

I'm posting this in the form of a question because it's been a long time since I took statistics and I'm not a math major. I just don't think there's any way that strategies based on 14 years of returns have essentially a zero percent chance of having a negative return. I've read that the stock market has a normal distribution (and most things do), but I don't know if the strategies we make up do.

I recently commented on this from the business cycle perspective. With the 14 years we have 1.5 business cycles as opposed to more. Given mjedlin's constraints, I still think his work is most impressive and so I do a mixture of the standard seasonal and daily strategies. I also throw in some research.

[KWG_5]

I've decided I'm going to back-test a couple strategies that just use G, S, and C. I'm going to use the indexes the S and C are based on and just use an average of the G fund. The I fund just doesn't track its underlying fund that well. I've already got something set up for 2013 to today so I don't have to start from scratch.

I want to test a strategy with a low standard deviation and see how the numbers fall.

On a related note, has anyone ever tried doing a Freedom of Information Act request to the TSP to try to get actual share prices/returns prior to 2013? They certainly exist in an electronic format somewhere.

[mjedlin66]

On a related note, has anyone ever tried doing a Freedom of Information Act request to the TSP to try to get actual share prices/returns prior to 2013? They certainly exist in an electronic format somewhere.

Not necessarily. Prior to mid 2003, there were no TSP share prices. In mid 2003 they set the share price of every fund to exactly $10 and then started publishing daily share prices.

Prior to that, the smallest increment of time that the TSP analyzed was 1 month. So every month they would calculate the change as a percentage and spit out a report.

[evilanne]

Aitrus has monthly earning from inception in 1988. A FOIA could work if you ask the right questions. Since they added the S & I funds in 2003, they probably had research and possibly return information (daily or monthly) for S & I equivalents that could be useful in recreating prior year data. You don't know what they have unless you ask. I would probably ask TSP directly (you can submit questions within you account) before messing with a FOIA.

[Amber9132]

You are correct from a general statistics perspective that the standard deviation is best applied to a distribution which reflects the normal distribution, however it can be applied elsewhere. I think the misunderstanding that some people have and you are wise to point out is that there is no guarantee that the next year or a following year's return will fall within one, two, or even three standard deviations. Using a normal distribution the likelihood of such an occurrence is typically 68, 95, and 99.7%. I don't think anyone is foolish enough to think there is a 99.7% chance of any type of return using a given strategy.

I use strategy 17680 with a mean of 31.08 and a SD of 3.78. By no means does that mean I virtually guarantee a return between 19.74 and 42.42 (3 SDs) even if it has never had a year beyond that range.

However, you can apply the standard deviation as a sort of mental heuristic as to the overarching variance (that's ultimately just what standard deviation is anyways) that you can expect. The stock market is not normally distributed, but at the end of the day a strategy with a higher SD will fluctuate more (on average) than a strategy with a lower one. Take it with a grain of salt. Next year 17680 could easily lose money, but I expect it is less likely to than many other strategies.

(Source: PhD level statistics classes)

[mindofmush]

I also questioned whether the statistical analysis applied to only 13 (now 14)years is valid when 12squared introduced us to it. I've since studied enough to realize that CLT (Central Limit Theory) generally needs population of 30 or more for a valid normal distribution. CLT allows smaller samples if data is continuous, as small as 4 years. Bessel's correction for smaller samples maintains validity but it's important to remember that the StdDev of the sample will be less than or equal to the StdDev of the population. There are even more correction factors available for small samples as well as other methods of dealing with small sample size that I barely understand.

Bottom line is that I believe the numbers are valid even with only 14 years of data. As for predicting the future of the market, it's better than flipping a coin.

[KWG_5]

I'm getting close to finishing my look at years 1988 through 2003 using some strategies that only use G, S, and C (or an approximation of G and the indexes S and C are based on). I want to double-check my work before I post my results. I was mainly curious to see how closely past returns fell in line with years 2004-2017. I'm comparing some low standard deviation strategies with some high standard deviation strategies.

My initial impression is that low standard deviations from 2004 to 2017 don't guarantee a low standard deviation from 1988 to 2003.

[KWG_5]

So here's what I've come up with. If someone's done something similar I'd be interested if they came up with the same results. I haven't seen anyone else do this, so hopefully I didn't reinvent the wheel.

Table.JPG compares four strategies against the C fund/S&P 500. The colors go from red to green each year.

Graphs.JPG shows a scatter plot for each of the strategies.

I'll let people draw their own conclusions, but I think this shows that strategies with really small standard deviations during the past 14 years weren't nearly as small the prior 16 years. Also, past results weren't very impressive. They were pretty similar to just staying in the C fund.

If anyone has questions please let me know. I can also give more details on how I came up with these results, but it's pretty boring and the short story is it took a lot of copying and pasting formulas in Excel.

Now that I have my spreadsheet completed I can test additional strategies in just a matter of minutes, so if someone wants to test another strategy I can. While looking through strategies that only used G,S, & C they all were very similar. I had a hard time finding some that weren't really close. I ended up making 27557 for this test.

[nweis]

Hi KWG_5,

Can you please test 20241? That is the strategy I am currently using and I am curious to see what it looks like.

Thank you!

-Nancy

[ewok55]

I'll let people draw their own conclusions, but I think this shows that strategies with really small standard deviations during the past 14 years weren't nearly as small the prior 16 years. Also, past results weren't very impressive. They were pretty similar to just staying in the C fund.

Thank you for doing this. I think it demonstrates exactly what myself (and a few others) were cautiously saying, namely that models based on past data might describe that past data well, but that doesn't equate to "predicting." It is easy to confuse the two, but in reality you're dealing with a posteriori hypotheses.

Find me an a priori model that can actually PREDICT (i.e., numbers used to create the model are independent of numbers it tries to predict), and then you've found the rainbow's pot of gold. Otherwise, you're just proving that "hindsight's 20/20."

[mjedlin66]

Find me an a priori model that can actually PREDICT (i.e., numbers used to create the model are independent of numbers it tries to predict), and then you've found the rainbow's pot of gold. Otherwise, you're just proving that "hindsight's 20/20."

Every strategy is based off of past data. Seasonals. Technical, sell in may go away, all of it.