Systematic review of the types of methods and approaches used to assess the effectiveness of healthcare information websites Jennifer Tieman A,B and Sandra L. Bradley A A Palliative and Supportive Services, School of Medicine, Flinders University, Health Sciences Building, Repatriation General Hospital Campus, Daw Park, SA 5041, Australia. B Corresponding author. Email: Jennifer.Tieman@flinders.edu.au Abstract. The aim of this systematic review was to identify types of approaches and methods used to evaluate the effectiveness of healthcare information websites. Simple usage data may not be sufficient to assess whether desired healthcare outcomes were achieved or to determine the relative effectiveness of different web resources on the same health topic. To establish the state of the knowledge base on assessment methods used to determine the effectiveness of healthcare websites, a structured search of the literature was conducted in Ovid Medline, resulting in the retrieval of 1611 articles, of which 240 met the inclusion criteria for the present review. The present review found that diverse evaluation methods were used to measure the effectiveness of healthcare websites. These evaluation methods were used during development, before release and after release. Economic assessment was rare and most evaluations looked at content issues, such as readability scores. Several studies did try to assess the usefulness of websites, but few studies looked at behaviour change or knowledge transfer following engagement with the designated health website. To assess the effectiveness of the knowledge transfer of healthcare information through the online environment, multiple methods may need to be used to evaluate healthcare websites and may need to be undertaken at all stages of the website development process. Received 28 February 2013, accepted 6 August 2013, published online 5 September 2013 Introduction Given the diversity of information needed by health professionals and health consumers within a range of care settings, the online environment offers a powerful means by which to disseminate and maintain information currency, as well as encourage engagement with healthcare knowledge. Information seekers now have an unprecedented ability to access vast amounts of information on any health issue via a keyboard or touch screen. Online healthcare information dissemination is powerful because of its reach, its relative cost advantages and its immediate availability. As a result, there is growing interest in the role of e- health and telehealth within the healthcare system, particularly within primary healthcare (Australian Government 2010). Statistics provide evidence of the rapid uptake and use of the Internet by all age groups, geographic regions and countries for accessing information, including healthcare information. Figures from the Internet World Stats website showed that, in June 2012, there were 2.45 billion people, or 34.3% of the world’s population, with access to the Internet (Internet World Stats 2013). According to the Australian Bureau of Statistics, during the same time period in Australia there were over 12 million Internet users, with the number of users increasing at a rate of around 10% annually (Australian Bureau of Statistics 2012). A significant proportion of this Internet activity relates to health issues. A study by Eysenbach and Kohler (2003) examined the prevalence of health-related searches on the web by analysing search terms entered into popular search engines. These authors found that an estimated 4.75% of all searches were health related (Eysenbach and Kohler 2003). The Pew Internet and American Life Project report Health Online 2013 indicated that 72% of Internet users have looked online for health information of one kind or another within the past year; three-quarters of these searches began at a search engine such as Google or Bing (Fox and Duggan 2013). This demand for online health information highlights the importance of information being derived from credible sources. Although advances in technological capabilities are making it increasingly easier to create online resources, developing high- quality content that is readily accessible draws upon a complex range of skills and knowledge. The types of skills and knowledge required can be most readily seen in the US usability guidelines (US Department of Health and Human Services 2006) or the Australian Government’s web publishing guidelines (Australian Government 2012). An integral part of determining the ‘value’ of a website is to formally investigate the resource in a structured, purposeful manner. This form of investigation, or evaluation, can support design integrity, successful development and delivery, and appropriate modification and recognition of the website Journal compilation Ó La Trobe University 2013 www.publish.csiro.au/journals/py CSIRO PUBLISHING Australian Journal of Primary Health, 2013, 19, 319–324 Research http://dx.doi.org/10.1071/PY13030