Recommender Systems

Recommender systems are one of the most widely used ML implementations in the world today. Although not widely used in finance at present, it is easy to think about where they might be useful once we go through a few examples of where they are being used in the real world at the moment. 

What Are Recommender Systems?

Recommender systems generally recommend items or courses of action to users based upon their past behaviour. They determine a ranking based upon the predicted likelihood of the next choice that user would make. For example, since 2000 Netflix has used a personalised movie recommendation system which suggests new movies according to past viewer activities and the thumbs up or thumbs down rating system. This has been extremely successful, and after several model iterations, around 80% of Netflix’s total stream time is achieved via this recommender system.

In addition to media companies, almost all e-commerce and music sites use recommender systems today. They take into account the purchases made by the user in the past and similar purchases made by others.  

In terms of finance specifically, you could easily envisage a use case for something like a personal news recommender. With the vast amount of news and information available these days, it is often difficult to sift through the numerous available news sources in order to get to the news that you are interested in. A recommender system that is personalised to the user and that takes into account newswire preferences and even companies or sectors of interest could be very useful. A second possible use case could be analysing past client activity in order to make more relevant security suggestions taking into account company sector, maturity, ratings, country or any other number of relevant factors. 

How Do They Work?

Recommender systems are mainly seen in 2 forms:

1. User Based Collaborative Filtering
2. Item Based Collaborative Filtering

User Based Collaborative Filtering

Collaborative filtering is about making recommendations according to a combination of your experience and the experiences of others. A user’s experience can be captured automatically or by their manual inputs for e.g. pressing a Like button.

Let’s look at how it works with reference to movies:

• Firstly we need to make a user vs item matrix to capture the user’s preferences against each item.
  • Each row represents a user’s ranking and each of the columns can be the items they voted for. E.g. a movie, a product or a news source.
    • An item can also be an aggregate, like Drama Movies, where we can calculate weighted average ranking from the underlying movies.
• Secondly, we compute similarity scores between the users.
• Thirdly, we find users who are similar to you based on past behaviour.
• Finally, we suggest an item which the user hasn’t experienced before.
• For e.g.
• Assume that there are two users:
• One watches 2 movies: Lord of the Rings and The Hobbit.
• The other one has only watched The Lord of the Rings.
• User based collaborative filtering performs mathematical calculations between the users and notices a high correlation between them based on their voting scores of Lord of the Rings. It can then recommend to the second user to watch The Hobbit.

Item Based Collaborative Filtering

• In this system, instead of finding relationships between the users, items like movies or news sources are compared with each other based upon their usage or liking.
• Similar to the above, in item based recommendation systems, we need to make an item vs user matrix.
  • Each row is the items (like movies, products or news sources) and each column is the user rating.
  • We then calculate the similarities between rows using mathematical correlation.

Some of the Math Involved

We will be calculating the correlation between the ratings of the different users. The correlation coefficient between the ratings can be calculated using:

• Sx is the standard deviation of X. 
• Sy is the standard deviation of Y.
• X bar is the mean of X.
• Y bar is the mean of Y.

Choosing the Right Model

If there are a lot more users than there are items then User based collaborative filtering will be more accurate as we have more granular inputs. This is however a more computationally expensive approach as we need to calculate the correlation coefficients between each of the users. The situation is reversed when there are significantly more items than there are users. There are many other models and techniques that are being developed and for those keen to explore further, this recent paper¹ explores these and other approaches in more depth.

To conclude, even though recommender systems are not being heavily used on trading floors at present, the models and methodologies are robust and well established. In the coming years we will no doubt see these models being adapted and adopted to help front office professionals make better decisions and serve their clients more efficiently.

1. Zhang, Q., Lu, J. & Jin, Y. Artificial intelligence in recommender systems. Complex Intell. Syst. 7, 439–457 (2021). https://doi.org/10.1007/s40747-020-00212-w