IEEE Proof IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 1 CommTrust: Computing Multi-Dimensional Trust by Mining E-Commerce Feedback Comments Xiuzhen Zhang, Lishan Cui, and Yan Wang, Senior Member, IEEE Abstract—Reputation-based trust models are widely used in e-commerce applications, and feedback ratings are aggregated to compute sellers’ reputation trust scores. The “all good reputation” problem, however, is prevalent in current reputation systems— reputation scores are universally high for sellers and it is difficult for potential buyers to select trustworthy sellers. In this paper, based on the observation that buyers often express opinions openly in free text feedback comments, we propose CommTrust for trust evaluation by mining feedback comments. Our main contributions include: 1) we propose a multidimensional trust model for computing reputation scores from user feedback comments; and 2) we propose an algorithm for mining feedback comments for dimension ratings and weights, combining techniques of natural language processing, opinion mining, and topic modeling. Extensive experiments on eBay and Amazon data demonstrate that CommTrust can effectively address the “all good reputation” issue and rank sellers effectively. To the best of our knowledge, our research is the first piece of work on trust evaluation by mining feedback comments. Index Terms—Electronic commerce, text mining 1 I NTRODUCTION A CCURATE trust evaluation is crucial for the success of e-commerce systems. Reputation reporting systems [1] have been implemented in e-commerce systems such as eBay and Amazon (for third-party sellers), where overall reputation scores for sellers are computed by aggregating feedback ratings. For example on eBay, the reputation score for a seller is the positive percentage score, as the percent- age of positive ratings out of the total number of positive ratings and negative ratings in the past 12 months. 1 A well-reported issue with the eBay reputation man- agement system is the “all good reputation” problem [1], [2] where feedback ratings are over 99% positive on aver- age [1]. Such strong positive bias can hardly guide buyers to select sellers to transact with. At eBay detailed seller ratings for sellers (DSRs) on four aspects of transactions, namely item as described, communication, postage time, and postage and handling charges, are also reported. DSRs are aggregated rating scores on a 1- to 5-star scale. Still the strong posi- tive bias is present – aspect ratings are mostly 4.8 or 4.9 stars. One possible reason for the lack of negative ratings at e-commerce web sites is that users who leave negative feedback ratings can attract retaliatory negative ratings and thus damage their own reputation [1]. 1. http://pages.ebay.com/help/feedback/allaboutfeedback.html • X. Zhang and L. Cui are with the School of Computer Science and IT, RMIT University, Melbourne, VIC 3001, Australia. E-mail: {xiuzhen.zhang, lishan.cui}@rmit.edu.au. • Y. Wang is with the Macquarie University, Sydney, NSW 2109, Australia. E-mail: yan.wang@mq.edu.au. Manuscript received 26 Mar. 2013; revised 24 Sep. 2013; accepted 5 Nov. 2013. Date of publication xxx. Date of current version xxx. Recommended for acceptance by F. Bonchi. For information on obtaining reprints of this article, please send e-mail to: reprints@ieee.org, and reference the Digital Object Identifier below. Digital Object Identifier 10.1109/TKDE.2013.177 Although buyers leave positive feedback ratings, they express some disappointment and negativeness in free text feedback comments [3], often towards specific aspects of transactions. For example, a comment like “The products were as I expected.” expresses positive opinion towards the product aspect, whereas the comment “Delivery was a little slow but otherwise, great service. Recommend highly.” expresses negative opinion towards the delivery aspect but a pos- itive opinion to the transaction in general. By analysing the wealth of information in feedback comments we can uncover buyers’ detailed embedded opinions towards dif- ferent aspects of transactions, and compute comprehensive reputation profiles for sellers. We propose Comment-based Multi-dimensional trust (CommTrust), a fine-grained multi-dimensional trust eval- uation model by mining e-commerce feedback comments. With CommTrust, comprehensive trust profiles are com- puted for sellers, including dimension reputation scores and weights, as well as overall trust scores by aggregating dimension reputation scores. To the best of our knowledge, CommTrust is the first piece of work that computes fine-grained multidimension trust profiles automatically by mining feedback comments. In later discussions, we use the terms reputation score and trust score interchangeably. In CommTrust, we propose an approach that combines dependency relation analysis [4], [5], a tool recently devel- oped in natural language processing (NLP) and lexicon- based opinion mining techniques [6], [7] to extract aspect opinion expressions from feedback comments and iden- tify their opinion orientations. We further propose an algorithm based on dependency relation analysis and Latent Dirichlet Allocation (LDA) topic modelling tech- nique [8] to cluster aspect expressions into dimensions and compute aggregated dimension ratings and weights. We call our algorithm Lexical-LDA. Unlike conventional 1041-4347 c  2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.