On ensuring a higher level of data quality when documenting human rights violations to support research into the origin and cause of human rights violations Romesh Silva School of International and Public Affairs Columbia University rs2032@columbia.edu ABSTRACT This paper reviews some of the measurement challenges posed when documenting large-scale human rights violations and considers a number of approaches. Practical methods and techniques based on recent field experience in Sri Lanka are presented which, when employed, will significantly improve the quality of human rights violations data. These improvements in data quality can enhance the ability of researchers to analyze the factors, origins and causes of human rights violations. Furthermore, through a review of the current literature on reliability measurement techniques, consideration is given to desirable statistical properties for reliability measures when applied to data on human rights violations. INTRODUCTION Human rights researchers collecting data in conflict zones face challenges of accuracy, precision, validity and reliability when documenting violations. These challenges arise from the need for a robust system which accurately records the nature, scope and intensity of violations. However the very nature of a conflict zone where large-scale human rights violations are being perpetrated makes it difficult to meet these demands. With the establishment of a permanent International Criminal Court and increases in domestic legal human rights prosecutions 1 , there is a pressing need for rigorous research into the origins and causes of human rights violations. The standard of evidence demanded by these forums therefore requires human rights researchers to not only explain the methods of data collection and statistical analysis used in amassing statistical human rights evidence but will also require them to provide complimentary scientific measures of the quality of their data. These requirements are now prompting the development of reliability measures for human rights data and statistical methods for operational use in the collection of data in the field. The conceptualization of human rights is constantly evolving as evidenced by the continual evolution of 1 Refer to Henkin (2000) and United Nations (1996). human rights norms and standards in both customary international law and treaty law both internationally and regionally. However, as Henkin (2000) notes, the modern conception of rights has evolved principally from a legal and political basis through the International Bill of Rights. As a result only over the last ten years have statisticians started to formulate a statistical framework for human rights monitoring and reporting. The Universal Declaration of Human Rights (UDHR) is a good starting point and provides for a general reference framework, in that it articulates a widely agreed set of standards of civil and political rights and economic and social rights. However, as it stands, the UDHR does not constitute a comprehensive framework for a statistical nomenclature and measurement system. Based on fieldwork experiences in Sri Lanka and the lessons learned from Ball, Spirer and Spirer (2000) in studying large-scale violations in Guatemala, El Salvador and South Africa, it appears unrealistic to assume one can develop a universal statistical methodology to study human rights violations. Instead any Information System applied in a human rights setting needs to be custom-designed by a multidisciplinary group so as to yield high explanatory power of the nature and causes of large-scale violations while also accommodating local strengths, needs, weaknesses and conditions. In this paper we consider three inter-related areas concerning the quality of human rights data: (1) the design of Human Rights Information Management Systems, (2) the theoretical concept of data quality and (3) the application of data quality measures to human rights settings. DESIGN OF AN INFORMATION SYSTEM FOR HUMAN RIGHTS MONITORING As Ball (1996) has noted there are essentially four basic steps in any Information System for human rights monitoring or human rights documentation project, namely (1) collection of information, (2) data processing, (3) database representation and (4) generation of analytic reports. This paper focuses on human rights data quality issues which arise principally in the data processing stage of an Information System. According to Spain and Hollenbeck (1975) systematic coding systems provide the most mathematically sound methods for observational research. For the human rights field, such coding systems provide an explicit reference framework for studying violations. In particular they provide a Joint Statistical Meetings - Social Statistics Section 3242