ImageAlert: Credibility Analysis of Text-Image Pairs on the Web Yusuke Yamamoto * Graduate School of Informatics, Kyoto University Yoshida-Honmachi, Sakyo, Kyoto, Japan yamamoto@dl.kuis.kyoto-u.ac.jp Katsumi Tanaka Graduate School of Informatics, Kyoto University Yoshida-Honmachi, Sakyo, Kyoto, Japan tanaka@dl.kuis.kyoto-u.ac.jp ABSTRACT Although the Web lets users freely browse and publish infor- mation, most Web information is unauthorized in contrast to conventional mass media. Therefore, it is not always cred- ible or correct. We propose a model to solve these problems by enabling the credibility of text-image pairs on the Web to be analyzed. We propose a bipartite graph model, in which one set of nodes corresponds to a set of text data, and the other corresponds to a set of images. That is, each text-image pair is represented by an edge. We introduce the notion of “supportive relationships” among edges in our bipartite graph model. Intuitively, our hypothesis is that the more “supportive” text-image pairs a target text-image pair has, the more credible it is. Although such a bipartite graph model is by itself not necessarily new, one of its most notable features is that we take into consideration the sim- ilarity (or dissimilarity) among nodes in each node set to compute supportive relationships. As our model is generic, it can be applied to a variety of types of Web information represented by text-image pairs. We especially focus on the analysis of text-image pairs on the Web in this paper and de- scribe a practical implementation of our model for analyzing credibility, ImageAlert. Categories and Subject Descriptors H.3.3 [Information Systems]: Information Search and Re- trieval; I.4.9 [Computing Methodologies]: Image Pro- cessing and Computer Vision General Terms Algorithm Keywords Credibility of Web information, image credibility, image-text pairs analysis, image processing * He also works as a Research Fellow (DC2) of the Japan Society for the Promotion of Science. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC’11 March 21-25, 2011, TaiChung, Taiwan. Copyright 2011 ACM 978-1-4503-0113-8/11/03 ...$10.00. Samurai warrior Ashikaga Takauji was designated as Shogun and began government in Kyoto. Page A The person in this portrait is not Ashikaga Takauji but Kohno Moronao, a follower of Takauji. Page B Figure 1: Example where different names for people conflict in similar images. 1. INTRODUCTION Research aimed at evaluating the credibility of content on the Web has recently become even more crucial because it has started to influence our daily lives. The abundance of content on the Web, the lack of publishing barriers, and poor control of the quality of Web content has raised issues with credibility. Several researchers have reported that users perceive Web information to be somewhat credible [13, 14]. If users are not aware of the credibility of Web information, they can easily be misled, and this can sometimes endanger them. For example, there are more than twenty thousand health-related sites on the Web, but more than half such sites have not been reviewed by medical specialists [16]. For example, Figure 1 shows horseback scenes of a certain Japanese historical personage on two Web pages. The por- traits are of the same person in both these images, but their captions actually name two different people. A recent widely accepted interpretation actually explains that the person in these portraits is Kohno Moronao, not Ashikaga Takauji. Then, it follows that the caption on page A is incorrect. Figure 2 shows another example of an exaggerated adver- tising illustration on the Web. For the same hamburger H, ad (a) exaggerates the value of hamburger H by using an attractive depiction and a catch phrase, while a consumer’s blog (b) indicates that the real hamburger H is awful unlike hamburger H in the ad. Thus, if captions of images provide incorrect facts like these, users are faced with the possibility of mistakenly identifying images. It is therefore important to ensure that images on Web pages are credible. Three possible solutions are for users to check the credi- bility of images themselves, ask trustworthy people, or check for their credibility as exhaustively as possible from printed