An Improved Hybrid Recommender System by ... - Semantic Scholar

An Improved Hybrid Recommender System by Combining Predictions Belkacem Chikhaoui Prospectus Laboratory University of Sherbrooke, Canada [email protected]

Mauricio Chiazzaro Department of computer Science University of Sherbrooke, Canada [email protected]

Abstract—Recommender systems are gaining a great importance with the emergence of E-commerce and business on the internet. These recommender systems help users in making decision by suggesting products and services that satisfy the users’ tastes and preferences. Collaborative filtering and content-based recommendation are two fundamental methods used to develop recommender systems. Although, both methods have their own advantages, they fail in some situations such as the ’cold start’ where new users or items are added in the system. In this paper, we propose an approach that combines collaborative filtering, content-based and demographic filtering approaches to develop a recommender system for predicting ratings in a dynamic way. We show through experiments that our approach achieves good accuracy and high coverage and outperforms the conventional filtering algorithms as well as the naive hybrid methods. Moreover, we show how our approach deals with the cold-start problem by incorporating demographic characteristics of users.

I. I NTRODUCTION With the rapid development of the Web and E-commerce, and the huge amount of information available, there is an increased need for creating sophisticated systems that are able to process the existing amount of information on one side, and help the users in making decisions by suggesting them products and services that match their preferences and tastes on the other. Recommender systems are proposed as a promising solution to deal with these issues. Two main techniques are used to develop recommender systems. Content-based (CB) techniques [1] in which the user will be recommended items similar to those the user preferred in the past, and collaborative filtering (CF) techniques [2] in which the user will be recommended items that the other users similar in tastes liked in the past [3]. Each technique has some limitations when it is taken individually such as limited content analysis, new item problem, new user problem, sparsity, etc. To address these limitations, hybrid recommender systems have been proposed where the CB and CF techniques are combined in order to provide more accurate predictions and overcome the limitations of each technique. In this paper we propose a new hybrid approach to build a recommender system. Our approach combines CB, CF and demographic filtering (DF) techniques to improve the prediction accuracy, allow better coverage, and overcome the cold

Shengrui Wang Prospectus Laboratory University of Sherbrooke, Canada [email protected]

start problem. The demographic characteristics of users (e.g. gender, race, age, disabilities, educational attainment, home ownership, employment status, occupation, etc.) are used to overcome the cold start problem by categorizing the users into categories (clusters) using a nearest neighbor technique. Each category holds users sharing similar demographic characteristics. For a new user, items are recommended using only the cluster to which this user belongs. In the same way, the combination of demographic characteristics and CB technique allows to solve the problem of new items added in the system. Our work makes several contributions: • A hybrid approach to build an efficient recommender system that makes a trade-off between prediction accuracy and coverage. • An approach that attributes a confidence measure dynamically to each recommendation algorithm according to the available ratings of a user. • An approach that deals with the cold start problem. • Good coverage thanks to the combination of different recommendation algorithms. The rest of this paper is organized as follows: Section 2 describes the problem definition. The related work is discussed in Section 3. Section 4 presents our model and the experimental results are presented in Section 5. Finally we conclude the paper in Section 6. II. P ROBLEM DEFINITION Given a set of users U = {u1 , u2 , ..., uN }, and a set of items I = {i1 , i2 , ..., iM }. In recommender systems, each user u rates a set of items RIu = {ui1 , ui2 , ..., uiP }. The rating of user u on item i, denoted by ru,i , can be expressed in different ways such as real numbers, expressions, etc. Very often, ratings are mostly expressed using integer values in the range [1, 5]. Let u ∈ U be a user, and i ∈ I be an item. The recommender system predicts the rating rˆu,i (called the predicted rating) of the user u on the item i such that ru,i is unknown. The existing recommender systems predict the rating rˆu,i based on the past ratings of the user u on items similar to the item i (CB techniques), or the ratings of other users similar to the user u expressed on the item i (CF techniques), or both (hybrid techniques).

Another issue in recommender systems is how to improve the coverage. The coverage in a recommender system means the number of pairs < user, item > in the test dataset for which we can predict the rating [4]. Our hybrid approach with the incorporation of demographic filtering can improve the coverage of a recommender system. III. R ELATED WORK In this section we discuss the existing recommender systems with a focus on hybrid systems that combine both content-based and collaborative filtering techniques. Work has been done in hybrid recommender systems by combining content-based and collaborative filtering techniques. [5] proposed an approach that combines the content of items and item-based collaborative filtering using a clustering technique. The clustering results merge the content information into the collaborative filtering in order to solve the cold start problem. However, this approach ignores the demographic information of users which can be helpful to improve the prediction results. [6] proposed a hybrid approach referred to as (Hydra) that combines collaborative and content-based filtering. The authors incorporate the rating data as well as the content information in a unified model. The authors demonstrate that mixing features of users and items achieves good accuracy. Nevertheless, the authors do not exploit all the users and items features like the address and rating time, and they do not indicate how their approach deals with the cold start problem. [7] proposed a framework that combines collaborative, content-based and demographic filtering for recommending information sources such as Web pages or news articles. The author used home HTML pages to gather demographic information of users. The recommender system is tested on very few number of users and items which cannot guarantee the efficiency of the proposed system. Moreover, there is no explanation on how the model is built. M. Lee et al. [8] proposed a collaborative filtering recommender system combined with the SOM Neural Network. The authors categorize the users based on their demographic informations. Then, a clustering technique is used to cluster the users in each category according to their preference to items using the SOM Neural Network. After that, the collaborative filtering is used to recommend items to users. However, the cold start problem is not addressed in the paper. [9] proposed a hybrid recommender system using Boltzmann Machines which are probabilistic models that combine both content-based and collaborative filtering techniques. By using Boltzmann Machines, the authors try to increase the accuracy of the prediction and overcome the cold start problem. Demographic information of users is not considered in the proposed approach. M. Jahrer et al. [10] used many collaborative filtering algorithms such as SVD (Singular Value Decomposition), Neighborhood Based

Approaches, Restricted Boltzmann Machine, Asymmetric Factor Model and Global Effects to build a recommender system. The authors show that linearly combining these algorithms increases the accuracy of prediction. However, the authors do not indicate how their approach deals with the cold start problem, and demographic information of users is not considered in the proposed approach. In addition, the use of all these models leads to significant increase in training time. [11] proposed a model in which the contentbased algorithm is used to enhance the existing user data, then the collaborative filtering is used for rating prediction. [12] proposed a clustering approach applied on the itembased collaborative filtering to deal with cold start problem. An adjusted K-means clustering algorithm is proposed. The prediction is computed by performing a weighted average of deviations from the neighbor’s mean. Other works on hybrid recommender systems can be found in [13]. IV. O UR M ODEL The challenge of recommender systems is not only increasing the precision of the predicted ratings, but also having the best coverage. In this section we discuss our rating prediction model. A. Neighborhood-based collaborative filtering The neighborhood-based approach is commonly used in CF recommender systems. In a neighborhood-based approach, the rating of a user u on item i is calculated based on the ratings of like mined users that rate similarly the item i. Such approach depends strongly on the number of neighbors that rate the item i and share similar ratings profile as the user u. The neighbors of the user u are computed using a similarity measure. Cosine-based and Correlation-based (equation 1 and 2) are the most used similarity measures in recommendation systems [3]. Formally, let u and v be two users, and let RIuv be the set of all items co-rated by both users u and v. In the cosinebased method, users u and v are treated as vectors in the |RIuv |−dimensional space [3]. Therefore, the similarity is computed as follows: → − − u .→ v − − = sim(u, v) = cos(→ u,→ v)= → − − k u k2 × k → v k2 X

ru,i rv,i

i∈RIuv

s X i∈RIuv

2 ru,i

s X

(1) 2 rv,i

i∈RIuv

− − − where → u .→ v denotes the dot-product between the vectors → u → − and v . In correlation-based method, the Pearson correlation coefficient [14] is used to measure the similarity between users

as follows: X

(ru,i − r¯u )(rv,i − r¯v )

i∈RIuv

Pu,v = sim(u, v) = s X

(ru,i − r¯u )2

s X

i∈RIuv

(rv,i − r¯v )2

i∈RIuv

(2)

Here r¯u and r¯v are respectively the average of the u and v user’s ratings. In our model, we employed the cosine-similarity measure (equation 1) to compute the similarity between users. The users and items are represented by a N × M matrix. Thus, the similarity is computed along the rows of the matrix. The KNN algorithm is used to select the most similar users. B. Content-based filtering Unlike the user-based collaborative filtering, the contentbased CF algorithm finds the set of items RI = {ui1 , ui2 , ..., uiP } previously rated by the user u called (active user), and selects the items that are similar to the item i called (target item) using a similarity measure. For example, in the case of MovieLens1 dataset (see section V), we used the movies features that characterize the rated movies such as genre, country and date. We used the KNN (nearest neighbor) algorithm to select the most similar items to the target item i. Unlike the user-based CF, in the content-based algorithm the similarity is computed along the columns of the user-item matrix. One drawback of the cosine-similarity is that it does not take into account the differences in rating scale between users. To deal with this drawback, we employed the Pearson correlation similarity measure shown in equation 2. The Pearson correlation similarity measure deals with the differences in rating scale by substracting the average user’s ratings from each co-rated pairs of items [15]. The predicted rating using the Pearson correlation similarity measure are computed as the weighted average of deviations from the neighbor’s mean [11] using the following formula: N X (rv,i − r¯v ) × Pu,v

ˆru,i = r¯u +

v=1 N X

(3) Pu,v

v=1

where ru,i is the predicted rating of the user u on the item i; r¯u is the mean rating given by the user u; Pu,v is the Pearson correlation similarity between users u and v; and N is the number of users in the neighborhood. C. Demographic filtering Demographic characteristics of users play an important role in identifying categories of users that like certain kinds of items or have similar tastes. In some situations, CF and 1 GroupLens

Research: http://www.grouplens.org

CB algorithms fail to predict ratings or recommend items to users specifically in the case of new users or items (cold start problem). Demographic filtering can be used to overcome the limitations of the CF and CB algorithms. [16] developed a recommender system based only demographic characteristics of users. In our model, we used demographic characteristics of users (e.g. age, gender, occupation and zip code) available in the MovieLens dataset to create categories of users that share the same demographic data. The KNN algorithm is used to create these categories using the cosine-based similarity measure. The rating prediction of a user u on item i is computed using only the category to which the user u is belonging. The accuracy of prediction depends on the number of nearest neighbors of the user u. The number of nearest neighbors will be discussed in section V. D. Rating prediction The most important task of recommender systems is rating prediction. In our model, the rating prediction is obtained by combining the three recommender algorithms described previously: collaborative, content-based and demographic filtering. Each algorithm predicts the rating of a user u on item i independently from the others. To combine linearly all the predictors, we should take into account the contribution of each one in the final rating prediction. For example, if the user u has already rated a collection of items similar to the target item i, then the content-based algorithm will perform better than the other algorithms. In the same way, if the user u has a great number of neighbors that have rated the target item i, then the collaborative filtering will perform better than the other algorithms. In the case of cold start problem, neither content-based nor the collaborative filtering will perform well. However, the demographic filtering can perform well in this situation. To take into account the difference in the contribution of each predictor in the final rating prediction, we associate a parameter to each predictor. The final rating prediction is given by the following formula: ˆru,i =

αDFu,i + βCBu,i + γCFu,i α+β+γ

(4)

To compute the value of each parameter, we should use a function Ψ(n) that gives value 1 for big values of user’s ratings n (n = |RIu |) and a small value for small values of n. The sigmoid function satisfies these constraints for Ψ(n). Therefore, the parameters α, β and γ can be computed using thesigmoid function of n asfollows: 1     γ=  −n  1+e 2 . 1     α = β = 1 − −n  1+e 2 The parameters α, β and γ represent a sort of confidence given respectively to the DF, CB and CF recommendation

algorithms. The greater the number of user’s ratings, the more confidence will be given to the CF algorithm. The values of these parameters change dynamically depending on the available ratings expressed by the user u. E. Cold start problem In order to make good recommendations, the recommender system learns first the user’s preferences and tastes based on his/her ratings. When new items are added in the system, traditional collaborative filtering algorithms fail to recommend these new items to users since no ratings expressed by users on these items. The same problem happens in the case of new users. In our approach, we use the demographic characteristics of users to solve the problem of new users. To do so, we make the following steps: 1) create categories of users that share the same demographic characteristics. 2) find the new users’ nearest neighbors using the KNN algorithm with the cosine-based similarity measure. 3) based on the ratings of the nearest neighbors users, compute the prediction for the new user u on the item i using the equation 3. V. E XPERIMENTS In this section we describe the experimental study and metrics we use to compare our approach with different prediction algorithms. We present first the dateset used in our experiments, followed by the evaluation metrics and comparison, then the results obtained are presented.

where ru,i and rˆu,i denote the actual and predicted rating. The smaller the value of RMSE, the more precise a rating prediction. But, recommender systems try to make a tradeoff between the precision and coverage. As mentioned earlier, the coverage is the number of pairs < user, item > for which we can predict a rating. In order to have the coverage and the precision in the same metric measure, we use the F-measure as follows: F-Measure = (2.P recision× Coverage)/(P recision + Coverage). We compare the results of our approach with different existing mainstream algorithms described below: 1) collaborative filtering using the Pearson correlation similarity measure (CF). 2) content-based filtering using the cosine-similarity measure (CB). 3) demographic filtering using the cosine-similarity measure (DF). 4) content-based and collaborative filtering hybrid method (CB+CF). 5) collaborative and demographic filtering hybrid method (CF+DF). 6) content-based and demographic filtering hybrid method (CB+DF). 7) collaborative, content-based and demographic filtering naive hybrid method (CF+CB+DF). In this method, the rating prediction is computed as the mean of the rating prediction of each predictor. 8) Our approach. C. Experimental results

A. Dataset We performed our experiments on the MovieLens dataset. The dataset contains 943 users and 1682 movies with 100.000 ratings. Each user rates minimum 20 movies using integer values in the range [1, 5] . We note that not all movies are rated by all users. There are 19 movie genres. The movie can belong to more than one genre. A binary value (0 or 1) is used to indicate whether a movie belongs to a specific genre. The dataset is splited into 5 disjoint training (80 %) and test (20 %) sets. B. Evaluation metrics and comparison To evaluate a recommender system, the cross validation (including the leave-one-out method [17]) is used. We make a 5-fold cross validation for rating prediction. The rating prediction is the average of experiments results over 5 training and test datasets. We use the Root Mean Squared Error (RMSE) to measure the error in our recommender system as follows: v X u u (ru,i − rˆu,i )2 u t (u,i)∈T estSet (5) RMSE = | T estSet |

In this section we present the results obtained and the comparison with the different methods mentioned above. Before that, we should find the optimal values of the parameter K in the neighborhood computation. 1) Neighborhood size: The neighborhood size (K parameter) plays an important role in the prediction results. We varied the number of neighbors from 10 to 100 and computed the RMSE. The RMSE decreases between k=10 and k=90, after that it tends to be stable. We observe that the DF is strongly affected by the value of K. Therefore, we select k=100 as the optimal value of neighborhood size. 2) Comparison: Once the optimal value of K is obtained, we compare our approach with the other methods. Table I shows the RMSE, coverage and F-Measure for all methods. Our approach outperforms all the other methods according to the trade-off made between the coverage and precision. We note that the collaborative filtering method performs well and outperforms all the other methods in terms of precision and F-Measure except our approach (precision: 0.718 vs 0.721 and F-Measure: 0.706 vs 0.707). We also observe that the naive hybrid method (CF+CB+DF) does not perform well comparing to our approach or to the collaborative filtering method.

Table I R ESULTS OF DIFFERENT METHODS FOR ALL USERS Algorithm Content-Based Collaborative filtering Demographic filtering CB+CF CB+DF CF+DF CF+CB+DF Our approach

RMSE 0.644 0.552 0.61 0.598 0.625 0.582 0.581 0.554

Precision 0.618 0.718 0.645 0.641 0.61 0.649 0.692 0.721

Coverage (%) 53.6 69.4 62.7 64.2 60.9 66 66.1 69.4

F-Measure 0.574 0.706 0.636 0.641 0.609 0.655 0.676 0.707

the collaborative filtering that performs poorly in the cold start problem. Figure 2 shows the RMSE for all the methods for the cold start problem. We note that our approach makes the smallest value of RMSE amongst all the other methods.

Figure 2. Comparison of RMSE of all methods for the cold start problem

Figure 1. Comparison of Coverage, Precision and F-Measure of all methods for all users

3) Overcoming the cold start problem: As mentioned earlier, in pure collaborative filtering the prediction of ratings for new users or items cannot be made since there is not enough past ratings by other users. In our approach, the incorporation of the demographic filtering algorithm allows to overcome the cold start problem. To test our approach on the cold start problem, we choose the users from the training set and delete all the ratings of those users. Therefore, we treat them as new users. In our experiments, we randomly select 500 users from the training set and delete their ratings. Table II shows the results obtained by all methods on the cold start problem. We observe from Table II that our approach outperforms all Table II R ESULTS OF DIFFERENT METHODS FOR THE COLD START PROBLEM Algorithm Content-Based Collaborative filtering Demographic filtering CB+CF CB+DF CF+DF CF+CB+DF Our approach

RMSE 0.682 0.638 0.621 0.862 0.82 0.811 0.86 0.586

Precision 0.626 0.528 0.639 0.367 0.563 0.542 0.595 0.677

Coverage (%) 53.4 59.1 61.3 25.6 32.6 34.1 25.9 65.5

F-Measure 0.576 0.558 0.626 0.301 0.413 0.419 0.361 0.666

the other methods and achieves good results comparing to

Figure 3. Comparison of Coverage, Precision and F-Measure of all methods for the cold start problem

In summary, our approach performs well for all users as well as for the cold start problem comparing to the other methods. We observe from Figures 1 and 3 that the demographic filtering performs well in terms of precision and coverage for all users (precision: 0.645, coverage: 62.7 %) and for the cold start problem (precision: 0.639, coverage: 61.3 %). The results obtained by the demographic filtering method demonstrate the importance of taking into account the demographic characteristics of users and how these characteristics can improve the rating prediction in recommender systems. Our approach demonstrates the ability of the collaborative filtering algorithm in making good predictions for users that have more ratings (precision: 0.718, coverage: 69.4 %).

The collaborative filtering contributes greatly in the results of our approach in the case of users with a large number of ratings, which increases the value of γ and decreases the two other parameters α and β. However, for the cold start problem, the demographic content-based filtering will contribute more given the small value of γ. As another contribution in our paper, our approach allows to compute rating predictions for all users as well as for the cold start problem using one formula which integrates parameters that take values according to the actual state of the user in terms of the number of available ratings. The dynamic way in computing the parameters α, β and γ makes our approach flexible, adaptable and more practical in real systems.

[5] Q. Li and B. M. Kim, “An approach for combining contentbased and collaborative filters,” in Proceedings of the sixth international workshop on Information retrieval with Asian languages. Morristown, NJ, USA: Association for Computational Linguistics, 2003, pp. 17–24.

VI. C ONCLUSION AND PERSPECTIVES

[7] M. J. Pazzani, “A framework for collaborative, content-based and demographic filtering,” Artif. Intell. Rev., vol. 13, no. 5-6, pp. 393–408, 1999.

In this paper we introduced an efficient hybrid approach for recommender systems. Three recommendation algorithms are incorporated in our approach to predict the ratings. We attributed a confidence measure to each recommendation algorithm. This confidence measure is computed according to the number of ratings available for a particular user. Hence, the confidence values change dynamically according to the available ratings, which allow to solve the cold start problem. Our work indicates that the correct combination of collaborative, content-based and demographic filtering recommender algorithms can improve the performance of recommender systems in terms of precision of rating prediction as well as the coverage. We validated our approach on real data for the rating prediction for normal users and cold start problem. The results obtained show that our approach outperforms the classic filtering algorithms as well as the naive hybrid methods. Our work suggests an interesting future extensions and directions. First of all, we want to test the efficiency of our approach on several other datasets. Second, in our approach, we focused on one important task of recommender systems which is the prediction of ratings. We plan to extend our approach to cover the second important part of recommender systems, that is the recommendation of a list of top items to users. R EFERENCES

[4] M. Jamali and M. Ester, “Trustwalker: a random walk model for combining trust-based and item-based recommendation,” in Proceedings of the SIGKDD conference. New York, NY, USA: ACM, 2009, pp. 397–406.

[6] S. Spiegel, J. Kunegis, and F. Li, “Hydra: a hybrid recommender system [cross-linked rating and content information],” in CIKM-CNIKM, 2009, pp. 75–80.

[8] M. Lee, P. Choi, and Y. Woo, “A hybrid recommender system combining collaborative filtering with neural network,” in AH, 2002, pp. 531–534. [9] A. Gunawardana and C. Meek, “A unified approach to building hybrid recommender systems,” in RecSys, 2009, pp. 117– 124. [10] M. Jahrer, A. T¨oscher, and R. Legenstein, “Combining predictions for accurate recommender systems,” in Proceedings of the SIGKDD conference. New York, NY, USA: ACM, 2010, pp. 693–702. [11] P. Melville, R. J. Mooney, and R. Nagarajan, “Contentboosted collaborative filtering for improved recommendations,” in proceedings of AAAI/IAAI, 2002, pp. 187–192. [12] Q. Li and B. M. Kim, “Clustering approach for hybrid recommender system,” in Web Intelligence, IEEE / WIC / ACM International Conference on. Los Alamitos, CA, USA: IEEE Computer Society, 2003, pp. 33–38. [13] R. D. Burke, “Hybrid recommender systems: Survey and experiments,” User Model. User-Adapt. Interact., vol. 12, no. 4, pp. 331–370, 2002. [14] J. L. Herlocker, J. A. Konstan, A. Borchers, and J. Riedl, “An algorithmic framework for performing collaborative filtering,” in Proceedings of the SIGIR conference. New York, NY, USA: ACM, 1999, pp. 230–237.

[1] R. J. Mooney and L. Roy, “Content-based book recommending using learning for text categorization,” in DL ’00: Proceedings of the fifth ACM conference on Digital libraries. New York, NY, USA: ACM, 2000, pp. 195–204.

[15] B. M. Sarwar, G. Karypis, J. A. Konstan, and J. Riedl, “Itembased collaborative filtering recommendation algorithms,” in proceedings of WWW, 2001, pp. 285–295.

[2] J. A. Konstan, B. N. Miller, D. Maltz, J. L. Herlocker, L. R. Gordon, and J. Riedl, “Grouplens: applying collaborative filtering to usenet news,” Commun. ACM, vol. 40, no. 3, pp. 77–87, 1997.

[16] E. A¨ımeur, G. Brassard, J. M. Fernandez, and F. S. M. Onana, “Privacy-preserving demographic filtering,” in Proceedings of the ACM symposium on Applied computing. New York, NY, USA: ACM, 2006, pp. 872–878.

[3] G. Adomavicius and A. Tuzhilin, “Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions,” IEEE Trans. Knowl. Data Eng., vol. 17, no. 6, pp. 734–749, 2005.

[17] J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. T. Riedl, “Evaluating collaborative filtering recommender systems,” ACM Trans. Inf. Syst., vol. 22, no. 1, pp. 5–53, 2004.