Improving Patient Opinion Mining through Multi-step Classification Lei Xia 1 , Anna Lisa Gentile 2 , James Munro 3 , and Jos´ e Iria 1 1 Department of Computer Science, The University of Sheffield, UK {l.xia, j.iria}@dcs.shef.ac.uk, 2 Department of Computer Science, University of Bari, Italy al.gentile@di.uniba.it, 3 Patient Opinion, http://www.patientopinion.org.uk/ james.munro@patientopinion.org.uk Abstract. Automatically tracking attitudes, feelings and reactions in on-line fo- rums, blogs and news is a desirable instrument to support statistical analyses by companies, the government, and even individuals. In this paper, we present a novel approach to polarity classification of short text snippets, which takes into account the way data are naturally distributed into several topics in order to obtain better classification models for polarity. Our approach is multi-step, where in the initial step a standard topic classifier is learned from the data and the topic labels, and in the ensuing step several polarity classifiers, one per topic, are learned from the data and the polarity labels. We empirically show that our approach improves classification accuracy over a real-world dataset by over 10%, when compared against a standard single-step approach using the same feature sets. The approach is applicable whenever training material is available for building both topic and polarity learning models. 1 Introduction Opinion mining or extraction is a research topic at the crossroads of informa- tion retrieval and computational linguistics concerned with enabling automatic systems to determine human opinion from text written in natural language. Re- cently, opinion mining has gained much attention due to the explosion of user- generated content on the Web. Automatically tracking attitudes and feelings in on-line blogs and forums, and obtaining first hand reactions to the news on- line, is a desirable instrument to support statistical analyses by companies, the government, and even individuals. Patient Opinion 4 is a social enterprise pioneering an on-line review service for users of the British National Health Service (NHS). The aim of the site is to enable people to share their recent experience of local health services on- line, and, ultimately, help citizens change their NHS. Online feedback is sent to 4 http://www.patientopinion.org.uk