Garbage in, garbage out

Why is it not sufficient to supply an AI system with all available data and instructions and expect correct results?

For all the progress in machine learning, the old principle still holds: garbage in, garbage out. A model is only as good as our understanding of the problem and the way we structure the data.

Here is a simple example from my index tracking research, where the task is to build cost-efficient portfolios that track indices such as the S&P 500 using only a subset of their constituents. I benchmarked several machine learning methods, among them linear regression, regression trees, and autoencoder-based selection.

Linear regression came out on top — but only once a non-negativity constraint on the portfolio weights was imposed. Without it, performance deteriorated noticeably.

For example:

• Constrained least squares regression: tracking error 1.12% p.a., turnover 23.31%, rHHI 108.72
• Unconstrained least squares regression: tracking error 1.48% p.a., turnover 54.93%, rHHI 81.49

Without the constraint, tracking error rises, turnover more than doubles, and the portfolio becomes more concentrated. The rHHI (reciprocal Herfindahl–Hirschman Index, \(1 / \sum_{i=1}^{N} w_i^2\) with \(\sum_i w_i = 1\)) can be read as the effective number of stocks, so the drop from 108.72 to 81.49 means fewer stocks carry the portfolio.

Why does the constraint help? The returns of an index and of a long-only portfolio are both non-negative linear combinations of their constituents' returns. Enforcing non-negative coefficients therefore mirrors the underlying data-generating process and stabilizes estimation, especially when the predictors are highly correlated.

In this example, a constraint derived from the structure of the problem mattered more than the choice of model class.

A detailed treatment of index tracking is given in our research paper.