Evaluating Web Document Quality with Linguistic Variables: Combining Informative and Page Design Quality Elena Garc´ ıa Computer Science Department University of Alcal´a Ctra. Barcelona km.33.6 28871 - Alcal´a de Henares Madrid (Spain) elena.garciab@uah.es Miguel- ´ Angel Sicilia Computer Science Department University of Alcal´a Ctra. Barcelona km.33.6 28871 - Alcal´a de Henares Madrid (Spain) msicilia@uah.es Tomasa Calvo Computer Science Department University of Alcal´a Ctra. Barcelona km.33.6 28871 - Alcal´a de Henares Madrid (Spain) tomasa.calvo@uah.es Abstract The concept of Web document qual- ity is multidimensional and difficult to characterize. For assessment pur- poses, it can be separated in two main concerns: content or informa- tive quality and design quality, being the latter essentially connected to the concept of Web usability. In ad- dition, usability assessment is usu- ally obtained from linguistic judge- ments, vague guidelines or uncer- tain indicators, which points out the suitability of a linguistic assessment framework. In this paper, a defini- tion for such linguistic framework is described, along with an exploration of the possible approaches to com- bine linguistic usability assessments with content-quality judgements. Keywords: Usability Analysis, Fuzzy Set Theory, Web Page Qual- ity, Linguistic Information Process- ing. 1 Introduction Web documents can be considered as hyper- media nodes that expose information by using a purposeful interface design for which a sig- nificant amount of widely accepted guidelines and principles have been proposed [18, 19]. Thus, the usability of Web pages is a prin- cipal quality criteria that can be considered as orthogonal to informative quality, which concentrates on the quality of the contents. Many usability evaluation methods are cur- rently used both by researchers and by prac- titioners [16, 3], but there exists a lack of commonly accepted metrics or benchmarks, which hampers the feasibility of building sys- tems that automate the filtering of Web pages according to their usability. A number of re- cent tools [11] and studies [12] focus on the building of usability analysis tools, that use usability indicators to obtain automated as- sessments of Web pages. But in any case, us- ability evaluation is inherently subject to un- certainty and vagueness. Uncertainty comes from subjectivity in expert or user judgements — arising, for example, in inspection meth- ods [17] or usability questionnaires [14]—, and vagueness is related to the approximate na- ture of most usability guidelines [5]. As pointed out elsewhere [10], fuzzy mod- els can be used to approach the problem of identifying the quality of Web documents by explicitly considering linguistic assessments and imprecise feedback, but existing content- oriented frameworks [8] should be extended to consider also the “form” of the documents, that is, their properties w.r.t. usability. Moreover, current content-presentation sepa- ration practices for Web documents recom- mend separating content and presentation physically, resulting in two distinct sets of ar- tifacts. On the one hand, content is expressed in some XML 1 sub–language, following a con- crete semantic schema. On the other hand, several presentations are generated for those 1 http://www.w3.org/TR/REC-xml