Last month in this space we talked about how not to pick mutual funds. We suggested that one should not pick funds merely on the basis of recent total returns (absolute or relative) or even on risk-adjusted performance or other quantitatively driven measures of past performance. Any such shortcuts can be financially damaging, as there are simply too many other variables, both qualitative and quantitative, that one needs to analyze and monitor to thoughtfully build and maintain an appropriate portfolio of mutual funds.

That said, quantitative measures can have an important role. At Kobren Insight Management, we have built several proprietary quantitative models that we find extremely effective in making our fund research process more successful. A quantitative "model" is simply a summation of a number of quantitative measures (variables) using varying weights for each to "predict" an outcome -- in our case, the likelihood that a mutual fund will deliver future relative outperformance.

Building A Quantitative Model of Fund Performance
In developing any quantitative model, there are three important tests that must be met. First, do the relationships between the variables used in the model and the outcome you are trying to predict make intuitive sense. People naturally want to find explanations for things and if you just go looking for "patterns" in the data, you can often "find" them. The problem, of course, is that if the relationship between the variables doesn't have a logical basis, the observed pattern is most likely spurious and will not hold up into the future.

Two examples from the recent Presidential election illustrate the point. Months before the election, the press reported on models that (accurately) predicted Bush would win, based on the performance of several key economic factors. While we might take exception to such a model as being too simplistic with economic issues being the only variables, the relationship between the economy and how people vote makes obvious sense.

More recently, the press made a (admittedly tongue-in-cheek) big deal about the "Washington Redskins Indicator." It had a perfect record, correctly predicting the outcome of every Presidential election since 1940 (the first year the Redskins played in Washington). When the Redskins won their game right before the election, the incumbent party also won. If the Redskins lost, the incumbent got dumped, as well. So when the Redskins lost to Green Bay on Sunday Oct 31 of this year, Kerry was a shoo-in. Oops. Because there was no rational relationship between the fortunes of the Redskins and the fortunes of the incumbent party, there was no rational reason to expect the indicator to be correct.

Second, are the "intuitive" relationships you believe to hold supported by academic research. We are fortunate that there is a large (and ever expanding) body of academic research on all important aspects of investing. For example, there is an intuitive notion that if a fund has good past performance, it is likely to do well in the future as well. However, there is ample academic evidence that the relationship between good returns in the past and future strong returns is virtually nonexistent.

Third, and most obvious, the model should be tested against a variety of scenarios using past data (back-testing) and should be tested on current data going forward for a sufficient length of time to "validate" the models effectiveness. Most important, after adoption, the model's effectiveness should continually be measured and the model modified if necessary.

Our primary fund rating model (which we apply to every mutual fund in the industry) meets all three of these tests, but before going into some detail on what comprises our model, I would like to discuss what we do with the model, as well as what we don't do with the model.

How We Use Our Fund Rating Model
The principal use for the model is to add structure to our research process. The model is a tool to sharpen our analysis. If we like a fund, but our quantitative model does not, where is it in the model that the fund is coming up short? Why might that be the case? Are we overlooking something? And conversely, if a mutual fund is rated highly by our model, but we have not been covering it with our research effort, then we need to examine the situation in fuller detail to see if we should add it to our coverage list. In short, both of these outcomes cause us to ask questions of ourselves and typically of the portfolio managers of the funds in question.

It is also important to understand what we don’t use these measures for. We don’t make any fund recommendations or trades based solely on our model's ratings. The models may raise questions, but they don’t supply the answers. After identifying a situation that may need further exploration, we need to make qualitative assessments to see what our models may be missing. In other words, these models are not "black boxes" that make our fund selections for us.

The Factors We Consider
What goes into our models? After dedicating several years and several analysts to the task, working with different variables, time frames, and weighting schemes, we identified several factors that were reasonably and consistently relevant to helping select mutual funds that may outperform funds in their respective peer groups over the next 12-18 months.

1. Expense Ratios
The first factor we have found to be important is expense ratios -- costs do matter. While there are certainly many cases of funds with higher than normal expense ratios outperforming their peers, all else being equal, the odds still favor the fund with a lower expense ratio. This has led us to a bias in favor of low expense funds. While we will on occasion use funds with higher expense ratios to meet a particular need, on average, the funds we invest in have expense ratios around 50 basis points (or one half of 1%) lower than their peers. (A natural question might be why then don’t we use index funds and other passive strategies which typically have the lowest expenses? Stay tuned, as we’ll address that issue in this space in a few months).

2. Selection Skills
One of the leading tenets of our investment philosophy is that, properly selected, active managers should be superior to passive managers (indexers). So in our fund selection, we look for those managers who demonstrate the ability to deliver "excess" returns over and above a benchmark index appropriate for the fund's investing universe. When evaluating an active managers excess return, we perform what is called attribution analysis, which is simply breaking down the factors that contributed to those returns. In the case of a fund manager's excess return, we break it down into what we call "Timing" skills and "Selection" skills.

Timing attempts to measure the portion of a manager’s return stemming from the positioning of their portfolio relative to their benchmark in terms of economic sector exposures or size (large-, mid- or small-cap) exposures or style (value or growth) exposures. For example, are they overweight growth stocks versus their benchmark, underweight energy, etc. Then, given those exposures, their "Selection score" measures the part of their return due to how well they did in picking individual stocks. In other words, within say their energy holdings, how did their specific energy stocks fare versus energy stocks in general. This breakdown is not just an academic exercise, as the distinction between Timing and Selection skills is critical. That is because we believe consistent success in Timing on the part of managers can be elusive, but that strong Selection skills do persist. As a result, while relative performance among managers of similar funds may fluctuate, sometimes considerably so, due to the "chance" nature of Timing, focusing on those managers with higher Selection skills will over time result in superior relative performance. We often state we are looking for good "stock-pickers" and this is why. (Note: the examples listed above are for equity funds; we use different criteria for measuring Selection when examining fixed income funds).

3. Past Performance
Lastly, we do indeed look at past performance. At this point I'm sure you are thinking, "Wait a minute, didn't you just say that academic studies have shown past performance was a poor way to pick funds?" We did. But those studies were based on the way people normally use past performance, which is that good past performance leads to good future performance. Without going into too much detail of what our proprietary model entails, one characteristic we look for in funds ispoor relative performance over the last three years, because our analysis has shown that there is actually a negative correlation between historical 3-year relative performance and future 1-year relative performance. (A negative correlation between two variables means they tend to move in opposite directions.) This is truly a unique feature of what we do and one that adds real value given that most investors wouldn't even consider investing in a fund that had dramatically underperformed for the last three years.

The Moral of the Story
It is often said that the biggest mistake most investors make is trading too often. There is some truth to that because of transaction costs and taxes, but it's an oversimplification. Money management firms parrot the "buy and hold" mantra to keep your business. The real problem for investors, isn't that they trade too much, but that they keep buying what has been hot and about to get cold, while selling what has been cold and about to get hot. Yet, money management firms keep touting their hottest products. How many mutual fund ads have you seen that say "Buy Fund X, it's up 50% over the past three years!. We're more likely to be selling.