Evaluating Real-time Online Research Data (RORD) and Verbatim Quotient Detection (VQD): low inference tools to monitor outcomes of unconstrained authentic Internet research Nasiroh Omar 1 , Colin Higgins 1 , Colin Harrison 2 , Diego Campo Millan 1 1 School of Computer Science and Information Technology {nxo, cah}@cs.nott.ac.uk 2 School of Education Colin.Harrison@nottingham.ac.uk University of Nottingham Jubilee Campus, Wollaton Road, Nottingham NG8 1BB, United Kingdom Abstract This research evaluated a prototype Internet research support system based on earlier work defining online reading and a model of online reading behavior. The prototype system is embedded with Real- time Online Research Data (to capture the participants’ online reading processes and written output) and Verbatim Quotient Detection (to monitor the proportion of verbatim material copied from Internet sources into a final essay). Fifty-seven undergraduate students and one faculty member participated in the experiment. Analysis of online reading activity using the real time data collection system and the verbatim detection tools demonstrated that even with limited time and a common task, when the whole Internet was available there was significant variation in participants’ search strategies, search terms and research processes. 1. Background Different online readers carry out Internet research in different styles. The research styles while working on goal-directed tasks display distinctive navigation characteristics [1]. Many studies analysing online reader behaviour reported that some of the online readers are in need of various types of support while navigating the web[2]. Furthermore, McEneaney argued that navigational competence makes a significant contribution to the online reading effectiveness. If this shortcoming were left unsolved, the second-level digital divide [3] in which users who have the ability to access online information but who will fail to comprehend the content [4] will become a serious problem. With the unprecedented pace of digital text spreading [5], the medium of reading is changing [6, 7], and the need to have access to online reading support tools is increased. In order to provide an online reading support tool, we need first to develop tools to evaluate online reading performance. Most of current computer-based online reading comprehension assessment is based on multiple-choice type of questions [8], non multiple-choice questions and short essays [9-11] and comprehension summary [11]. But Harrison [12] suggested that the most useful reading assessment is based on performance with an authentic tasks. Such tasks may be devised used in digital text reading due to the capability of computer to capture the process of reading processes in digital contexts, and especially in web reading [13]. The focus of this paper is to analyze and understand the general behaviour of the online readers while using a prototype of Internet Research Support System that will incorporate the authentic performance-based online reading evaluation. 2. Understanding of what online reader does while reading online We started our work by observing the complexities of online reader behaviour while reading online and defining what we meant by online reading. The term ‘online reading behaviour’ refers to the set of observable actions readers undertake while completing an online reading task. The task is internally explained