Unlock Editor’s Digest Lock for Free
In December 2021, Bryan Kelly, head of machine learning at Quant House AQR Capital Management, named his name on an academic paper that caused quite a stir.
The virtue of complexity that Kelly co-authored by Semion Mallamdo and Kanji Zhou found that complex machine learning models are superior to simple models in predicting stock prices and building portfolios.
This finding was a big deal as it contradicts one of the guiding principles of machine learning: bias variance tradeoffs. Given too many parameters to play, bots tend to overequip the output to random noise in training data.
However, Kelly and his co-authors have concluded, surprisingly, that more variables always improve returns. The only limit is the available computing power. Below is a video explaining to Wharton School in 2023 that the same principles applied to the billions of parameter models apply to ChatGpt and Claude AI, as well as the accuracy of financial forecasts.
Many scholars disliked this paper. Jonathan Burke of Stanford Business School has said in his theoretical analysis that it is “very narrow and practically useless to financial economists.” According to some Oxford University researchers, performance relies on sanitized data that is not available in the real world. Daniel Bunsic of Stockholm Business School says that the larger models tested by Kelly et al. are only more outperforming, as they choose a less favorable measure of the smaller model.
This week, University of Chicago’s Stephen Nager joined Pyleon. His paper argues that the seemingly integrality in predicting rewards – is the “surprising” results presented by Kelly et al. . .
. . . In effect, the weighted average of past returns is highest in the period when the predictor vector is most similar to the current one.
Nagel challenges the central conclusion of the paper that a highly complex box can make good predictions based on stock performance data for just one year.
This discovery is rooted in an AI concept known as double descent. This means that deep learning algorithms make fewer mistakes if they have variable parameters than data points. Having a model with a huge number of parameters means it fits perfectly around the training data.
According to Kelly et al., this all-envelope blob approach to pattern matching allows you to select predictive signals for extremely loud data, such as a year of US stock trading.
Garbage, says Nagel:
In a short training window, similarity simply means modernity, so predictions decrease to a weighted average of recent returns – essentially a momentum strategy.
Importantly, the algorithm does not advise momentum strategies because it finds it beneficial. It’s just the latest bias.
The bot “averages the most recent several returns in the training window to correspond to the prediction vector that is most similar to the current prediction vector,” says Nagel. “We don’t learn from the training data whether momentum or inversion dynamics exist. We impose mechanical momentum-like structures regardless of the underlying return process.”
Outperformance presented in a 2021 study, “and therefore reflects the accidental historical success of volatility-timed momentum, rather than predictive information extracted from training data.”
Skip details. Readers who want to know about the mechanism of kernel scaling with random Fourier features will be better served by authors who know what they are talking about. Our main interest is the AQR, $136 billion management Quant, which boasts its academic roots.
Kelly will serve as the frontman of AQR for better investments through machine learning. His “Virtuality of Complexity” paper can be found on the AQR website, along with some careful commentary from Boscliff Asuness on the value of machine-generated signals.
The field aves of Kelly and others, including a professor at the University of Chicago, are both him and Asnes’ alma mater, and look no different. However, since simple momentum strategies have historically been one of AQR’s best-in-the-box scheduling, perhaps this split in academic AI hype is not a bad thing for investors.
