Mining Your Data

I once was asked why I expend the effort to collect so much data about my trading. My answer then, as it is now, is that without data, there's nothing to analyze in order to improve. Anybody can collect data although few do. The trick is to make good decisions about what to collect, to collecte it, and then to do something with it.

There was a great comment over on The Chart Of the Day where the question was asked about time frames; whether the length of my trades has increased or decreased over the years and whether there was any correlation between that and my success or failure rate. The follow up question was whether risk was reduce/increased as a result as well. They were very thoughtful questions and reminded me that I indeed needed to revisit my data looking for answers.

Trading is Personal

Before we begin data mining the data that I have collected over the years, you first have to understand that trading is personal; it's a reflection of you. Although many people trade in a similar fashion, I suspect that no two people trade exactly alike. We are all individuals whoose make up is unique. The decisions we make about what to buy, to hold, to sell is very individual. There is no holy grail to trading successfully. It's about trial and error, analysis and improvement.

Data Mining

Before we begin to mine the collected data, we have to ask what we are looking for. The specific questions asked over on The Chart Of the Day was about length of trades and perceived risks. I would broaden that a bit to ask about ??


Time Frames - Winning and Losing

Over the past four plus years I have executed almost 7000 trades with the bulk of those trades coming in the past two years when I have been able to dedicate more of my time to the markets. As a result of the previous analysis I had done, I remember that the most important conclusion that came out of that reflection was the desire to not hold losers but instead to cut my losses sooner and to move on.

Towards the end of 2004, I had much more time to devote to trading and by 2005, I found myself trading in very short time frames; partly because I was experimenting with the idea of whether I trade successfully on a daily basis, partly because the markets dictated a shorter term focus (in a previous article I've talked about the affect of the general market on particular stocks).

Getting back to the data though. The tables you see here is a summary of that data examining:

• The percentage of trades that fall into day, 2 to 5 days, 6 to 10 days, 11 to 15 days, 16 to 30 days, and more than 30 days (for the 4 years of data points)
• The win percentage for each of these time frames
• The average return for each of these time frames
• The drawdowns for the three time frames where significant changes have ocurred over the 4 year period, namely the day trade, 3 weeks to a month, and greater than a month

From the data points collected over the past 4 years now, I clearly met the objective of cutting losses more quickly, having dramatically reduced the average loss on a stock from 38+% in 2002 to 0% in 2004 for trades that lasted more than 30 days. The improvement is clearly seen as well in trades between 16 to 30 days where I cut losses quicker moving from 27% to the low teens on a percentage basis.

At the same time, with my expermentation with very short term trades, I've seen my loss percentages rise on day trades reaching 23% but I improved the winning percentage of those trades once I began to concentrate on them from a toss up the first 3 years to almost 7 out of 10 in 2005. The devil is in the details though. If you dig deeper, the dollar figures show that the reason I increased the day trade win percentage is because I took profits early and allowed losers to run longer during the day. That's reflected in the percentage returns where you see 23.58% loss on a day trade basis compared to a 19.84% gain on the same time duration. Unless I can improve my absolute dollar gains on a day trading basis, it's a losing proposition.

Where Is the Sweet Spot?

The real gains in 2004 came from the trades that were > 15 days in duration but less than a month. Those trades accounted for 80% of the returns. When I look back at 2004, it was exactly the same story, accounting for some 90% of the gains for the year. This has been consistently true for me since the beginning. The bulk of my gains come in the 2 weeks to a month time frame. More analysis is needed to understand why this is so. In the next series, I'll take a look at what types of trades are the ones that make up the bulk of the sweet spot. Are they retrace trades, breakouts, etc. Is there a pattern that's repeated that we can look to exploit?

... to be continued ...