1 Regularized Gibbs Sampling for User Profiling with Soft Constraints Nicola Barbieri Department of Electronics, Informatics and Systems (DEIS) University of Calabria - Italy via Bucci 41c, 87036 Rende (CS) and Institute for High Performance Computing and Networks (ICAR-CNR) Italian National Research Council Email: nbarbieri@deis.unical.it; barbieri@icar.cnr.it Abstract—In this paper we extend the formulation of the User Rating Profile model, providing a Gibbs Sampling derivation for parameter estimation. Validation tests on Movielens data show that the proposed approach outperforms significantly the variational version in terms of both prediction accuracy and learning time. Gibbs Sampling provides a simple and flexible learning procedure which can be extended to include external ev- idence, in the form of soft constraints. More specifically, given a- priori information about user-neighbors, we propose an effective regularization technique that drives the first sampling iterations pushing the model towards a state which better represents the user-neighborhoods specified in input. I. I NTRODUCTION. The goal of a recommender system is to provide users with a list of products/information/services that will prob- ably meet his/her interests. The role and impact of those systems on everyday web navigation and e-commerce are increasing and recommendation techniques have been also applied in search engines to provide more personalized results and thus to improve their accuracy. As the volume of the catalog increases, Collaborative Filtering (CF) is becoming the dominant approach to recommendations. In fact, while content based techniques need to process huge amount of information concerning user-profiles, i.e demographic infor- mation, or products, CF techniques relies exclusively on a dataset of preferences expressed by users. Those preferences can be implicit or explicit: in the first case data are collected in the form of co-occurrence pairs during the browsing session of the user. This approach does not require an active role for the user, the collection phase is completely automatic but it can result in extremely noisy data. On the other hand, to improve the quality of the recommendation list, the user is asked to provide an explicit preference value, named rating, for each purchased item. A common approach to evaluate the predictive skills of a recommender systems is to minimize statistical error metrics, such as the Root Mean Squared Error (RMSE). The common assumption is that small improvements in RMSE would reflect into important improvements of the accuracy of the recommen- dation lists. Latent factor models, based on matrix factorization [1], [2] or Latent Semantic Indexing [3], [4] techniques are able to achieve significant results in terms of prediction accuracy. The underlying idea is to introduce a finite set of hidden factors which are used to capture hidden relationships among users and items. Each user/item is mapped in a more compact latent space, and their representation in this space is used to compute predictions. The major drawbacks of such models relie (i) on the complexity of the learning phase, which typically involves an Expectation Maximization algorithm, and (ii) on the lack of intuitiveness of the recommendations, which is a desirable property in order to gain users’ trust. On the other side, neighborhood based [5], [6] approaches, like K-Nearest Neighbors, are able to provide more intuitive recommendations, usually in the form of ‘Customers Who Bought This Item Also Bought...’, but they achieve a predic- tion accuracy lower than the one achieved by latent factor models. In this work we focus on a probabilistic approach for user profiling, which can be considered as the extension of the La- tent Dirichlet Allocation [7] to preference data. The contribute of the work is twofold: (i) we extend the original formulation of the model introduced in [8], by providing a Gibbs Sampling derivation of parameter estimation; (ii) we design and study the effects of a regularization procedure that include a-priori knowledge in the learning process. In particular, we are interested in using background information coming from a neighborhood model, to push the Gibbs Sampling towards an ‘useful’ clustering solution which increase the number of neighbors mapped to the same cluster of the considered user, thus preserving neighborhoods. This approach can be recalled as clustering with constraints, following the ideas proposed in [9], [10]. However, in this case, constraints satisfaction is not a primary concern but a desirable property, mainly because neighborhood models assumed as prior-knowledge produce low prediction accuracy and thus could deteriorate the overall performances. The rest of the paper is organized as follows: the formal framework for representing preference values and a summary of state-of-art approaches to the recommendation problem are given in Sec. II. The User Rating Profile model [8] is then presented in Sec. III, where we also provide a full derivation of Gibbs Sampling parameter estimation and introduce a neighborhood-preserving regularization technique. Finally, the performances achieved by the proposed approach in predic-