Also, the portfolio optimization problem looks a lot like unsupervised learning task or representation learning task: having a set of assets we need to group them into some “clusters” based on their profitability and after allocate more funds on the most predictive ones and less on the opposite side. What algorithms could we use for this purpose?
An illustration of different principal components of a portfolio: https://systematicedge.wordpress.com/2013/06/02/principal-component-analysis-in-portfolio-management/
One of the first unsupervised learning models you get familiar with at the machine learning class is a principal component analysis (PCA). It decomposes multidimensional data into the set of linearly uncorrelated variables, where the first such variable (also called principal component) is explaining the most variation in the data, and all next variables are sorted by having the maximal variance while being orthogonal to the previous variable. How we can use it for portfolio management? Many ideas are published, the most common one says that the first principal component serves as an approximation of the market, hence, choosing second and other components will give uncorrelated to the market strategies, which is what most of the investors want.
A visualization of the general idea behind autoencoder neural networks: https://sefiks.com/2018/03/21/autoencoder-neural-networks-for-unsupervised-learning/
PCA is a great tool, but it relies only on the linear dependence between the data axis. One of the alternatives, that allows non-linear dimensionality reduction, is based on neural networks — autoencoders. They can “squeeze” input data into some low-dimensional vector and after restore input from this representation. The idea of autoencoder can be exploited in many ways for portfolio selection, one of them is related to the evaluation of the risk carried by the particular asset: if some asset movement can’t be restored well (the predicted value differs a lot from the input in terms of, let’s say, mean squared error) from the low-dimensional representation — it’s associated with higher risk (and, possibly, higher profits). Please, see an example of this kind of portfolio in the experiments section.
Hierarchical risk parity
Geometrically, a covariance matrix of the assets in the portfolio is a complete graph (on the left), can we figure out a tree-based model that will be more optimal?
One of the optimization-based portfolio management methods is a risk parity model. It is also stated as an optimization problem, where we allocate rather the risk than the capital resources. The problem of this approach (and, actually, most of the approaches described in this article) matures when we’re working with portfolios of very huge size — if we represent connections between assets geometrically, they will be in the form of the complete graph (see image above), which is an over-complication in the world of hierarchies. The solution lies again in unsupervised learning, but with the exploitation of the hierarchical clustering algorithms applied to the covariance matrix. After finding clusters of the assets, we can re-allocate risk over them recursively. The high-level procedure described below:
A high-level description of the hierarchical risk parity (HRP) portfolio optimization algorithm