Over the last few decades, digital transformation has taken the lead, and the Internet has become a ubiquitous and the most potent tool in e-commerce and online advertisement. The web services, such as Youtube, Facebook, and Netflix, rely on recommender systems to use data to predict and narrow down user preferences.

Daily, the number of options is exponentially growing, which prompts the need for collaborative filtering to find the relevant items that match the user preferences.

Recommender systems (or recommendation systems) are artificial intelligence algorithms that suggest or recommend relevant products or items to customers. Depending on the industry, content-based filtering helps users to discover services they might otherwise not have found, thus generating a massive amount of income for companies that use particular recommender systems. For instance, 35% of Amazon web sales were generated through their recommended items. 

In addition to generating revenue through collaborative filtering, recommender systems effectively promote and help companies stand out among competitors. Recommender systems research is trained to understand user interest, previous decisions, and user feedback to offer personalized recommendations.

Recommender systems filtering methods: Collaborative vs. content

The two most commonly used recommendation systems are content-based filtering and collaborative filtering. 

Content-based filtering offers recommendations for items to the user based on other, similar items that the user liked in the past (for example, when the rating for the specific item is high). Each of the recommended items has contents, which by the system are considered properties, and thus the items compose a so-called item content matrix.

Collaborative filtering recommends an item to users if their neighbors (other users with similar interests) are interested in a similar item. For this approach, the user’s ratings on an item represent their interest, and the sum-total of the user’s ratings on items creates a so-called rating matrix. 

Both recommender systems have strong points and weaknesses, making them suitable for a specific type of recommendation. The content-based evaluating recommender systems focus on the item’s content and the user’s interest. Accordingly, this user-based filtering method recommends different items to different users, and the uniqueness of this practical recommender system is its strong point. In opposition, collaborative filtering tends towards community, and all of the item possibilities that users may like are hidden. The content-based method cannot discover and recommend indirect items to users, making this feature its weak point. 

Sometimes, a significant amount of content is identified with an item (for instance, when the item has many properties), and collaborative filtering preoccupies system resources and time for analysis. Since it doesn’t encounter analyzing rich content items, this can be considered a strong point of collaborative filtering. However, collaborative filtering can sometimes make unexpected recommendations based on the probability that the item can be suitable to the user while the recommendation doesn’t relate to the user profile. 

Collaborative Filtering Methods

Collaborative filtering is an important approach to the far-reaching scope of a recommender system that uses the interactions and data collected by the system from other users and then filters the information. The main idea is to evaluate items and products about which people agreed in the past and to produce new recommendations for which they presumably will agree again in the future. The evaluating recommender systems produce new recommendations based on past user-item interactions and the estimated proximities. 

The concept of collaborative filtering methods is simple: when we want to find a new TV show to watch, it is highly likely to ask friends for recommendations. Needless to say, we will ask for recommendations from friends with the same taste. In the same way, collaborative-filtering systems are focused on determining the similarity of ratings of items by the users. The user-item interactions are stored in the so-called “user-item interactions matrix.”

There are two main approaches for collaborative filtering: model-based and memory-based. The model-based collaborative filtering method responds to the user’s request instantly by compressing the huge database into a model and performing the user modeling through a reference mechanism. On the other hand, the memory-based algorithm loads the entire database into system memory and makes a recommendation based on the in-line memory database. While simple, this algorithm encounters the problem of loading enormous amounts of data through the system.  

In other words, the memory-based method analyzes the recorded interactions, assuming no model. For example, this approach uses the nearest neighbor search and suggests the most popular item amongst the neighbors. The model-based method is focused on user-item interactions and makes assumptions about an underlying “generative” model to make new recommendations.

Collaborative Filtering: Pros and Cons

The main advantage of collaborative filtering is that these approaches don’t require information about users of items, making them applicable in many situations. These methods become more accurate when there are more interactions between users and items. Each time the users interact with items, the system becomes more and more efficient in making new recommendations.

Considering that collaborative filtering is solely based on past interactions recorded between users and items, there is a flaw in recommendations for new users. In a situation with fewer interactions, it is impossible to recommend new items to any users. The “cold start problem” usually is addressed with specific strategies: random strategy (making random item recommendations to new users or recommending new items to random users), maximum expectation strategy (recommending popular items to new users or new items to most active users), exploratory strategy (making recommendation for a set of various items to new users, or a new item to set of various users) or with a non-collaborative method for the early life of the user or the item. 

Group advantages and disadvantages of collaborative filtering 

All the features in collaborative filtering are learned automatically, without the necessity of domain knowledge, making it the main advantage of this method. This approach recommends new items similar to previous items the users were interested in, which helps users discover new interests, even when they are not actively searching. The algorithm needs a user-item interaction matrix for products or items to train the matrix factorization model, meaning there is no need for in-detail features and contextual data.

On the flip side, because the recommendations are created on historical data and interactions, the lack of data can lead to difficulty making recommendations for new products or users. Over time, with the growth of the user base, algorithms become overwhelmed due to high data and lack of scalability. In the long run, a lack of diversity can be counterproductive since recommending new items to the user is the ultimate goal. The historical ratings will make the popular products even more popular, and due to little or limited data, there will be a need for more new recommendations and a diversity of products and items.

Model-Based Collaborative Approaches

Collaborative filtering methods are popular algorithms for recommender systems. By surveying user communities, this approach recommends items, discovers potential items hidden under the communities, and thus discards limitations.

The requirement of loading a large amount of data of in-line memory is the stumbling block of memory-based approaches. This particularly refers to situations when an overwhelming number of persons are using the system, and the rating matrix becomes huge and consumes computational resources. Once the system’s performance collapses, it cannot respond to the user’s requirements immediately. The model-based approach is successful in solving such problems.

There are four common approaches for model-based collaborative filtering: clustering, classification, latent model, Markov decision process (MDP), and matrix factorization.

Matrix Factorization for Recommendation

Matrix factorization techniques work by multiplying two different sorts of entities to generate dormant features that are still untapped. Collaborative filtering applies matrix factorization to recognize the relationship between the entities of items and users. Namely, based on the user’s ratings of the items, this method makes predictions of how the users would rate the items and makes recommendations based on that prediction.

The primary presumption of the matrix factorization method is that there is a dimensional space of features that is small-scale and untapped and can be used to represent both users and items. In this manner, the interaction between the user and the item can be acquired by evaluating the dot product of corresponding dense vectors in that space.

A matrix factorization-based collaborative approach is used to achieve two goals: reduce the dimension of the rating matrix and discover potential features hidden under the rating matrix and thus produce new recommendations. Low performance occurs due to data sparseness and a huge rating matrix. 

Collaborative filtering methods can be loaded with many users or items that might not contribute to predicting the missing value, meaning there would be unnecessary rows and columns in the matrix. With dimensionality reduction, the inessential rows and columns are disposed. 

Principal Component Analysis (PCA) is a technique of dimensionality reduction that focuses on discovering the significant components known as patterns in data without loss of credible data. In collaborative filtering, patterns are the users who often rate items or items rated by many users.

How it works 

For example, to create a user-based rating matrix for a TV show, there is a need to model the interactions between users and TV shows. The main assumptions are that there are features that describe shows pretty well, and the same features can be used to describe user preferences (high values for features liked by the user and low values if not liked).

The tricky part is not to explicitly give the features to the model since they are created for content-based approaches. Instead, within the matrix factorization techniques, the system is supposed to discover the useful features by itself and make descriptions for both users and items. The aim is to extract features that are learned and not given. Humans cannot understand the mathematical meaning of the features since there is no intuitive interpretation, but very often, the structures that pop up from such emergence are exceptionally close to intuitive decomposition. Needless to say, the use of matrix factorization results in a close representation of users regarding preferences and items regarding characteristics.   

Memory-Based Collaborative Filtering Algorithms

Memory-based collaborative filtering algorithms utilize the user rating to calculate the similarity between users or items based on historical data. The main idea of this collaborative filtering method is to find the most similar unrevealed items based on ranking, find relations of resemblance between users or objects, and recommend new items.

In order to make recommendations, memory-based algorithms use the entire user-item database to generate a prediction. The system analyzes a set of users with similar interests and similar transactions history. 

This method is also known as the nearest-neighbor collaborative filtering method and works by finding the nearest neighbors of the specific user and computing predictive values. The key feature of this prevalent memory-based algorithm is predicting the missing values of the matrix (active user vector / rating vector misses user ratings for certain items).  

There are two main memory-based approaches: user-based filtering and item-based filtering. The main characteristic of these approaches is that they analyze data from the user-item interaction matrix to offer new recommendations. 

The user-based filtering makes recommendations for a particular user by finding similar users and analyzing the similarity of ratings. In contrast, item-based filtering analyzes users who liked an item, finds other items that those and similar users liked, and outputs such items as recommendations. The difference is subtle, but it is crucial to know the most convenient of these memory-based algorithms for practical use.

While being easy to implement and able to aid the new data efficiently, the memory-based collaborative filtering method’s main drawback is the decreased performance in data with high sparsity and impeded scalability for large datasets.

User-Based Model

The user-user method is focused on identifying similar users with a similar profile of interactions and making recommendations for items that are most popular among these neighbors and are new for the specific user.

The slogan that best describes the users-based method is: “Users who are similar to you also liked….”. It is a user-centered method that scans the user interactions with items and evaluates the distance between users to find the neighboring ones. 

How it works 

Each user makes interactions with different items, creating its vector, the so-called “its line” in the interaction matrix. In order to make a recommendation for the specific user, the algorithm computes the similarity between the user’s interest and the interests of other users. The similarity measure works by finding users with similar interactions on the same items and supposes that the users are being close. After computing the similarities between all users, the algorithm selects the nearest neighbors to the specific user and makes suggestions regarding the most popular items among them, but in the process, selects the items that the specific user has yet to interact with. 

When measuring the similarity between users, the number of common interactions must be considered carefully to avoid recommendations that don’t match user preference. For example, some users could have only one interaction with the specific user and still be considered a 100% match and closer than users with 100 interactions and 98% match of agreement. Therefore, the user-adapted interaction considers that two users like similar items if they have the same interactions with many common items.

At the heart of the user-user collaborative filtering algorithm lies the user rating. Without any ratings, the system is unable to produce recommendations, a problem known as a “cold start.” Without the recommendations, users tend to flinch, leading to a complete system halt. This pattern is broken by forcing new users to rate items before entering the system. This strategy solves the problem of offering generalized items when users’ preferences are unavailable because, in such cases, the power of the recommender system becomes pointless. In addition to new users’ ratings, the lack of ratings of new items is another issue of collaborative filtering. If there are many items, out of which fewer are rated, these items will not be recommended to anybody.

Item-Based Model

The main idea of the item-item-based model is to find items similar to the ones that the user previously “positively” interacted with (both purchased and rated items). Two items are considered similar if most user interactions with those items are done similarly. The item collaborative filtering represents item based or “item-centered” by evaluating interactions users have with items and estimating the distances between the items.

This item-to-item-based collaborative filtering can be illustrated with the phrase: “Users who liked this item also liked….”

How it works

The first step toward making a recommendation for a specific user is to consider the item the user liked the most and represent it by its vector of interaction (also known as (“its column” in the interaction matrix) with other users. The next step is to calculate the similarities between the “best item” and all the other items. The basis for making suggestions is keeping the nearest neighbors to the selected “best item” and recommending the new items to the specific user. To get more relevant recommendations, the algorithm builds a similar-items table with all of the preferred items instead of the user’s favorite alone.

Item-to-item collaborative filtering method offers scalability and good performance thanks to the expensive similar-items table offline. The online algorithm looks up similar items for users’ ratings and purchases and solely depends on the amount of data (how many items the user rated or purchased). This makes the algorithm fast regardless of the size of the data sets. Additionally, the quality of recommendations is excellent since the algorithm makes suggestions for highly correlated items.

This algorithm is efficient even when the recommendations are based on several items (as few as one or two), eradicating the cold-start problem. The similarity matrix is smaller (unlike the used-based methods), which reduces the cost of finding neighbors in the matrix. Still, unlike user-based collaborative filtering, lower diversity in the recommendation is often a drawback of the item-based methods.    

Content-Based Methods: Concept

Content-based recommender systems attempt to predict what a user likes based on that user’s activity. This approach makes recommendations using keywords and attributes assigned to items in a database and creates suggestions based on the user’s profile. The creation of a user profile is based on data collected from the user’s actions, including purchases, ratings (both likes and dislikes), downloads, search for specific items, items added to the chart and wish list, clicks on product links, etc.

How content-based methods work: a user purchases a smartphone on a website and searches for smartphone accessories. The user enters keywords such as smartphone manufacturer, model, and phone cases with wallet and card holders. Based on this information, the recommender system may suggest phone cases with sleeves for credit cards with specific attributes, such as an RFID-blocking fabric layer for protection against unauthorized credit card scanning. This recommendation with an additional feature may be something the user didn’t think about but appreciates and will likely make a new order.

The database contains items with assigned attributes based on the products, services, and content. The central concept of content-based methods is the algorithm to make recommendations based on these attributes. Assigning attributes can be a massive project, and many companies reserve this task for subject-matter expert teams, which manually assign item attributes to get the best result.

User profiles are the element that is as important as assigning attributes in content-based recommender systems. The profiles include interactions between users and items. Namely, database objects contain information about interactions such as purchases, browser history, items the user has read, watched, or listened to, and their assigned attributes.

Repetition and frequency of attributes across multiple items have greater significance than those that show up less often. Not all attributes of an item are equally important to the user, and therefore the recommender algorithms need an established degree of importance. User feedback is considered one of the most critical features for creating the database, which is why many websites ask for user ratings for products, services, or content.

The recommender system creates a unique model for each user based on attribute weighting and history and makes recommendations weighted by importance and user preferences. For example, if a user creates a workout playlist, listens to songs that are from pop, hip-hop, and rap genres, and prefers male artists, the recommender system might recognize that the user likes male artists and dance songs. The algorithm will make suggestions for more urban contemporary songs from other male artists. The recommender system may also suggest different types of songs from the new (suggested) artists, but such options would have a lower assigned score in the algorithm.

Benefits of Content-Based Recommender Systems

In comparison with collaborative filtering, content-based methods have many benefits, including:

The recommender system doesn’t need historical data from other users to start making suggestions. Once the specific user starts searching and browsing certain items and starts making purchases, the recommender system will begin making relevant recommendations. This feature is handy for businesses with limited user pools and companies with many users but hardly any interactions between users.

Making suggestions for highly relevant items to the user is another excellent feature of this filtering method. The algorithm produces customized recommendations for the specific user based on its profile. By analyzing the user’s preferences and tastes, the system recommends products with the same attributes as in the user’s profile.

Highly relevant and personalized recommendations create a feeling of transparency, and users tend to accept such suggestions better. On the other hand, collaborative filtering may produce recommendations that are unrelated and irrelevant to the specific user. For example, suppose a group of users who purchased sweatshirts also happen to buy notepads. In that case, the collaborative filtering algorithm may recommend notepads to the specific user, even though the user is uninterested in the products and doesn’t have a history of buying or searching for the item.

Content-based filtering systems need initial inputs from users to start producing suggestions, but unlike the collaborative filtering “cold start” problem, the quality of recommendations is generally more accurate.  

Challenges of Content-Based Filtering

Creating perfect recommender algorithms is an unrealistic expectation, and like other recommender systems, this filtering method has its disadvantages:

Lack of diversity and newness is a common issue of content-based filtering. Creating recommendations is much more than suggesting relevant items to users. The specific user that likes Peaky Blinders will probably also like Money Heist. Still, there is a high chance that the user won’t need a recommender system to suggest the TV show, and the point of practical recommender systems is lost. To offer usefulness, the algorithm must produce diverse and unexpected suggestions.

As mentioned before, to create an efficient content-based recommender system, experts define, and tag attributes to items, and every time there is a new item, there is a need for new content. This burdensome task makes scalability challenging, time-consuming and costly. Speaking of subject-matter experts, the attributes may need to be more accurate or consistent due to the subjective nature of this task. To avoid the possibility of the items being incorrectly tagged and impairing the function of the recommender system, companies need a process to ensure consistent and accurate application of attributes.  

There are two methods used in content-based filtering: the vector spacing method (method 1) and the classification model (method 2).

Method 1: The Vector Space Method

Suppose a user watched a crime TV show and reviewed it on the Internet as good and watched a comedy TV show and reviewed it as bad. The rating system will use the information (for an item, in this case, a TV show) provided by the user and create a rating system. The next recommendation this user will get is another TV show from the crime genre because of the high rating from the user.

For this algorithm, the user vector consists of rankings from information provided by the user, and then an item vector is created, in which the tv shows are ranked according to the genres. The vector space method multiplies the values and makes recommendations based on the user and item vector.  

Method 2: Classification Method

The classification method works by creating a decision tree and deciding whether the user is interested in the item or not. For example, a TV show is considered, let it be Stranger Things. Based on user data, the genre of this show is horror (supernatural). Then, the genre is not a crime nor a user-reviewed horror/ supernatural/ fiction type of TV show. These classifications help the system conclude that the specific TV show is unsuitable for the specific user.   

Knowledge-Based Recommender Systems

Knowledge-based systems make recommendations based on specific queries made by the user. This approach often prompts the user to give guidelines, rules, or examples of an item. Based on the expectations provided by the user, the algorithm searches the database and recommends similar items. In other words, this recommender system produces suggestions based on specific inquiries provided by the user. 

These recommender methods are mainly used in domains with small user purchase histories. In such systems, before producing recommendations, the algorithm takes into account several factors: the knowledge about the items, including their features, user preferences (asked explicitly), and recommendation criteria.

The knowledge-based recommender systems use a knowledge base and an inference engine to perform the problem-solving analysis. The knowledge base contains a collection of information about the items, and the inference engine infers acuities from the gathered information.

How it works

Imagine you are going on a trip and looking to book an apartment. You will input location, price, a number of persons, how many rooms, proximity to landmarks, etc., and the website will return a list of apartments based on the listed requirements. The requirements could also be higher, such as modernly furnished or with good natural lighting. Changes are that this webpage is a knowledge-based recommender system.

From a layman’s perspective, you may wonder why this search should use a recommender system when the webpage could filter specific entries from a database and provide suggestions. Of course, the system could do that. However, the system will miss out on much functionality in such a situation. In most cases, the item spaces are complex, and filtering might return nothing, while the recommender systems will provide personalized results for each user.

Stages of recommendation process: the user begins making a query (by constraints (constraint-based) or by example (case-based)), and the algorithm starts searching the domain knowledge for relevant items and adds them to the query. The single database query consists of all relevant items from the database, which can be modified further by the user (changes of parameters).

A knowledge-based recommender system is a convenient tool that effectively solves specific problems. However, this method can be even more advantageous in conjunction with other recommender systems techniques.  

Hybrid Approaches

A hybrid recommender system is an approach that combines multiple (two or more) recommendation techniques to produce suggestions. The main idea of hybrid approaches is to create an algorithm with higher recommendation accuracy than the methods when used individually. For instance, a hybrid recommender system can effectively solve the lack of information in collaborative filtering and user preferences in content-based recommender systems. 

By combining the strengths of two or more recommender systems, the hybrid approach acquires knowledge increase, and the ultimate result is better recommendations. The merger of knowledge into one database allows the widening of the underlying collaborative filtering algorithm with content data and the broadening content-based algorithms with user behavior data.

The internet-connected world offers lots of information, and a recommender system helps users manage the information explosion and avoid overwhelming digital content. The three most noteworthy recommendation approaches discussed above exploit different information sources and follow different patterns to make recommendations. While all recommender methods produce personalized suggestions, there are variations in degrees of success and individual application domains.

As explained, content-based filtering methods analyze item properties and make predictions, while collaborative filtering scans similarities of different users or items and makes predictions for a specific user. Knowledge-based algorithms utilize explicit knowledge models from the domain. Each approach has pros and cons.

High data sparsity, cold-start problems, and user-item interaction are vital factors for recommendation systems, meaning that a combination of filtering methods will provide the best performance and most accurate recommendations. None of the basic approaches can exploit the available data completely; therefore, by combining the strengths of different models, hybrid approaches can overcome some of the shortcomings.

Evaluation of a Recommender System

Machine learning algorithms are as worthy as the accuracy they provide. By evaluating the performances of recommender systems, experts and companies decide which method assures personalized recommendations that best fit requirements.

Generally speaking, recommender systems have two evaluation methods: metrics-based (evaluation based on well-defined and precise metrics) and human-based evaluation (based on human judgment and satisfaction estimation).

Initially, the prediction power was considered an essential feature of recommender systems, and the algorithms were evaluated on their ability to predict user’s choices accurately and suggest accurate items or products. However, it is now widely agreed that while accuracy is crucial, there are more critical features. Besides exact anticipation of user’s taste, the recommender engines should offer new items, and item diversity, guarantee privacy and respond fast to interactions. 

The easiest way of evaluating recommender systems is to perform offline experiments. The existing pre-collected data set is tested with a protocol that represents user behavior (interaction with items), and recommender performance is evaluated with prediction accuracy. This inexpensive type of experiment doesn’t require interaction with real users, which enables the comparison of a wide range of algorithms. The downside of this method is a narrow set of questions that mainly analyze the recommender system’s prediction power. A user study is a more expensive option requiring a group of users to use the system, perform a set of tasks, and answer questions regarding their experience. Finally, online experiments are large-scale experiments on a deployed system where the evaluation is performed on real users unaware of the experiment. This methodology provides sturdy evidence as to the actual value of the recommender system.  

Metrics Based Evaluation

When the recommender system is based on a model that produces numerical values, such as rating recommendations and matching probabilities, the quality of suggestions can be assessed with a classic technique known as error measurement metric (for example: mean square error MSE). The algorithm is trained to analyze part of the available interactions, and the evaluation is based on the remaining ones. Another evaluation option for a recommender system that outputs numeric values is the classical thresholding approach (values above the threshold are positive, and values below are negative) which provides classification.

Moreover, suppose the suggestions from the recommender algorithms are not numeric values, and the output is a list of recommendations (user-user or item-item models). In that case, the metric approach can still evaluate the precision by estimating the proportion of recommended items that match user needs and preferences. For this purpose, the estimation can be implemented on items from the dataset for which there is user feedback. 

Human-Based Evaluation

The recommender system is expected to offer suggestions for items the user is interested in. Still, the produced recommendations should also be diverse and have so-called explainability.

Practical recommender systems and the algorithms that produce recommendations are created to avoid the “information confinement area.” The concept of “serendipity” can be explained by a particular recommender approach’s tendency to generate a confinement area (diversity of recommendations). 

Serendipity is estimated by calculating the distance between recommended items. The delicate part is that too low serendipity could create confinement areas, while too high would not consider user preferences and interests. A good recommender system suggests items that suit the user’s preferences and are not too similar. For example, instead of recommending a user James Bond films (“Skyfall,” “Spectre,” and “No Time to Die”), it seems better to suggest “Skyfall,” “Mission: Impossible – Fallout,” and “The Coldest Game.”

Explainability is the concept that a recommender system should provide outputs in a way that makes sense to users. If recommendations for items and products are hard to follow and easy to understand, the user tends to lose confidence and take all of the following recommendations with a reserve dose. Therefore, when making recommendations, many models include sentences such as “people who liked’ Spectre” also liked’ The Coldest Game,” “you liked “Skyfall,” and you may be interested in’ Mission: Impossible – Fallout.”

Conclusion  

Recommender engines are becoming essential in many industries, especially in the fields of social networking and e-commerce, and hence, represent one of the most exciting subjects for all parties involved: computer programmers as creators, companies as direct beneficiaries, and users as the most affected party. In this article, we introduced the basic concept of the recommender system for a better understanding of the subject. To better comprehend the questions related to these algorithms, we discussed types of recommender methods and approaches, how they are used, the pros and cons of each filtering method, and the benefits of using these recommendation algorithms.