Incorporating Diversity in Academic Expert Recommendation Omar Salman, Susan Gauch, Mohammed Alqahatani, Mohammed Ibrahim, Reem Alsaffar Computer Science and Computer Engineering Department University of Arkansas Fayetteville, AR, USA e-mail:{oasalman, sgauch, ma063, msibrahi, rbalsaff}@uark.edu Abstract—Expert recommendation is the process of identifying individuals who have the appropriate knowledge and skills to achieve a specific task. It has been widely used in the educational environment mainly in the hiring process, paper- reviewer assignment, assembling conference program committees, etc. In this paper, we highlight the problem of diversity and fair representation of underrepresented groups in expertise recommendation, factors that current expertise recommendation systems rarely consider. We present a novel way to model experts in the academic setting by considering the demographic attributes in addition to skills. We use the h- index score to quantify skills for a researcher and we identify five demographic features with which to represent a researcher's demographic profile. We highlight the importance of these features and their role in bias within the academic environment. We present three different algorithms for scholar recommendation: expertise-based, diversity-based, and a hybrid approach. To evaluate the ranking produced by these algorithms, we propose a modified normalized Discounted Cumulative Gain (nDCG) version that supports multi- dimensional features and we report the diversity gain from each method. We used a tuning parameter to calibrate the balance between expertise loss and diversity gain. Our results show that we can achieve the best diversity gain increase when the tuning parameter value is set around 0.4, giving nearly equal weight to both expertise and diversity. Keywords-Expert Recommendation; Diversity; Fairness; nDCG. I. INTRODUCTION We are witnessing a significant change in the amount of the available information. The introduction of social media, blogs, the internet of things, and knowledge sharing communities have dramatically increased the amount of the available knowledge online [1]. This has led modern economies to shift to knowledge-based economies where the intellectual capabilities and expertise of the people determine their values in their enterprise and society [2]. However, determining the level of a person's expertise is a significant challenge because it is quite difficult to assess the amount of knowledge that individuals carry in their minds. Hence, enterprises and companies are beginning to rely on documenting people’s expertise, and expert recommendation systems have been developed to identify the right individuals for a task. These systems are mainly dependent on the written artifacts of the experts to determine their expertise. For example, early systems consider the internal documents othe enterprise to extract the skills of individual employees [3]. Expert recommendation systems have been used in academia in the hiring process or finding reviewers or assembling a conference program committee. Although there have been promising developments, most expert recommender systems have not addressed the issue of demographic discrepancies and the need to have a diverse team [3]. Additionally, systems based on machine learning trained on biased training data perpetuate that bias in their recommendations, damaging underrepresented groups [4]. To address this issue, we propose three different approaches with the aim of providing accurate expertise, team diversity, and fair recommendation. Our contribution can be summarized as below:  Propose a novel way to model an expert in the educational setting using a multivariate profile.  Present new expert recommendation algorithms that consider different demographic attributes.  Propose a modified metric that evaluates ranking based on different attributes. II. LITERATURE REVIW The process of expertise recommendation has been extensively surveyed by Balog et. al in [3]. The interest in expertise recommendation and expert modeling in academia has been discussed in [3][5]. Although there has been little attention to study an expert demographic profile in academia, Cochran-Smith and Zeichner [6] defined it to include the status of an individual with respect to gender, race, ethnicity, socioeconomic background, and age. Any attempt to model an individual’s demographics is complicated due to the fact that people tend not to explicitly provide such information. Hence, there are several approaches to predicting their demographic information from publicly available information such as their name [7]-[13]. Bias in expertise recommendation within academia has received a fair amount of attention by many researchers. One study published by Nature magazine [14] shows that women are usually underrepresented in the peer review where only 20% of the reviewers are women. A similar study [15] shows that women and authors from emerging countries were underrepresented as editors and in peer reviewers. Because this problem is a focus area for the National Science Foundation (NSF), they developed an automated reviewer selection system that considers different demographic features when selecting reviewers [16]. The problem of bias in a peer review process is not limited to gender and race, but it can be seen from other angles, such as the geolocation of the reviewer. For example, a study in [17] shows that the US dominated the peer review process by 32.9% while its 102 Copyright (c) IARIA, 2020. ISBN: 978-1-61208-765-8 eKNOW 2020 : The Twelfth International Conference on Information, Process, and Knowledge Management