International Journal of Technology Assessment in Health Care, 31:3 (2015), 138–146. c  Cambridge University Press 2015 doi:10.1017/S0266462314000749 EVIDENCE INFORMED DECISION MAKING: THE USE OF “COLLOQUIAL EVIDENCE” AT NICE Tarang Sharma The Nordic Cochrane Centre-Rigshospitalet, Copenhagen, Denmark ts@cochrane.dk, tarangs@gmail.com Moni Choudhury, Bindweep Kaur, Bhash Naidoo, Sarah Garner National Institute for Health and Care Excellence (NICE), London, United Kingdom Peter Littlejohns King’s College, London, Primary Care and Public Health Sciences, London, United Kingdom Sophie Staniszewska University of Warwick, RCN Research Institute, Warwick Medical School, Coventry, United Kingdom Objectives: Colloquial evidence (CE) has been described as the informal evidence that helps provide context to other forms of evidence in guidance development. Despite challenges around quality, and the potential biases, the use of CE is becoming increasingly important in assessments where scientific literature is sparse and to also capture the experience of all stakeholders in discussions, including that of experts and patients. We aimed to ascertain how CE was being used at the National Institute for Health and Care Excellence (NICE). Methods: Relevant data corresponding to the use of CE was extracted from all NICE technical and process manuals by two reviewers and quality assured and analyzed by a third reviewer. This was considered in light of the results of a focused literature review and a combined checklist for quality assessment was developed. Results: At NICE, CE is utilised across all guidance producing programmes and at all stages of development. CE could range from information from experts and patient/carers, grey literature (including evidence from websites and policy reports) and testimony from stakeholders through consultation. Six tools for critical appraisal of CE were available from the literature and a combined best practice checklist has been proposed. Conclusions: As decisions often need to be made in areas where there is a lack of published scientific evidence, CE is employed. Therefore to ensure its appropriateness the development of a validated CE data quality check-list to assist decision makers is essential and further research in this area is a priority. Keywords: colloquial evidence, checklist, evidence-based healthcare, decision making, guidelines, NICE It is widely agreed that high quality evidence should underpin all healthcare guidance (1), yet there is debate about what con- stitutes as evidence. Almost always the scientific evidence is not complete, or does not address areas of importance for clinicians or patients to allow for a decision to be made solely on such ev- idence alone, necessitating the input from other sources within a deliberative process (2). There is often a need to contextualize this evidence and to understand how it should be implemented in healthcare practice, as without contextualization, guidance and policies may fail to produce the desired results. This shift from evidence-based to evidence-informed decision making has been reflected in the definition of evidence and methodological practices of leading guidance producing organizations such as the Health Evidence Network (HEN) of the World Health Or- ganization that define evidence as “findings from research and other knowledge that may serve as a useful basis for decision making in public health and health care”(3). Evidence has been conceptualized in a range of ways and one approach developed by Lomas and colleagues has described how three forms of evidence are used within a deliberative process for healthcare decision making, namely: “scientific evidence on effectiveness, scientific evidence on context and colloquial evidence”(3;4). • Scientific context-free evidence: evidence that helps determine the poten- tial benefit/ efficacy and safety of the health technology. This is likely to be universal and not subjective to specific geographical scenarios. The au- thors primarily refer to this being evidence from good quality randomized controlled trials (RCTs) (3). One can argue, however, whether context-free evidence can truly exist. This is especially true with respect to the choice of comparator, as often the standard care or usual treatment differs between countries and, therefore, the geographical scenarios plays an important role in setting the context of any treatment alternatives. In addition, these RCTs may not include outcomes that are of importance to patients (5). • Scientific context-sensitive evidence: evidence specific to particular real word scenarios and is less likely to be generalizable (3). It could be argued that to ensure the implementation of recommendations based on the best RCT evidence, some contextual evidence is needed to ensure effectiveness. • Colloquial evidence (CE): evidence that helps support, supplement or refute the scientific evidence and is often used to augment an evidence landscape. CE is an umbrella term and consists of different types of data including informal expert opinion from clinicians and/ or patients, their views and narratives, electronic data from Web sites, policy documents, and other reports etc (3). “Colloquial” has been defined by the Oxford English dictio- nary as “used in ordinary or familiar conversation; not formal or literary” and its synonyms include “informal”,“unofficial”, and “popular”(6), and, therefore, can be understood in this context as informal evidence. It could be argued that CE should not be considered as evidence, as it may not be collected in a rigorous or systematic manner. However, we would assert that it is not appropriate to conceptualize CE from a research perspective in this way, as it has a different role, attributes, characteristics, and contribution. It has been suggested that CE should be understood as the additional “knowl- edge” or “factors” alongside scientific evidence, which is considered in a deliberative process (7). The key issue for decision makers is to balance these diverse evidence types together, assess the weight to place on each, and more importantly let each specific type of evidence contribute appropriately to the final decision (2). 138