Adv Data Anal Classif
DOI 10.1007/s11634-012-0122-2
REGULAR ARTICLE
Non parametric statistical models for on-line text
classification
Paola Cerchiello · Paolo Giudici
Received: 29 December 2011 / Revised: 28 June 2012 / Accepted: 1 August 2012
© Springer-Verlag Berlin Heidelberg 2012
Abstract Social media, such as blogs and on-line forums, contain a huge amount of
information that is typically unorganized and fragmented. An important issue, that has
been raising importance so far, is to classify on-line texts in order to detect possible
anomalies. For example on-line texts representing consumer opinions can be, not only
very precious and profitable for companies, but can also represent a serious damage if
they are negative or faked. In this contribution we present a novel statistical methodol-
ogy rooted in the context of classical text classification, in order to address such issues.
In the literature, several classifiers have been proposed, among them support vector
machine and naive Bayes classifiers. These approaches are not effective when coping
with the problem of classifying texts belonging to an unknown author. To this aim,
we propose to employ a new method, based on the combination of classification trees
with non parametric approaches, such as Kruskal–Wallis and Brunner–Dette–Munk
test. The main application of what we propose is the capability to classify an author
as a new one, that is potentially trustable, or as an old one, that is potentially faked.
Keywords Non parametric statistical models · Kruskal–Wallis test ·
Brunner–Dette–Munk test · Text analysis · Opinion spam detection
Mathematics Subject Classification 62G10 · 62H30
P. Cerchiello (B ) · P. Giudici
University of Pavia, Pavia, Italy
e-mail: paola.cerchiello@unipv.it
P. Giudici
e-mail: giudicio@unipv.it
123