Machine Generation of Audio Description for Blind and Visual Impaired People VIRGÍNIA P. CAMPOS and LUIZ M. G. GONÇALVES, Federal University of Rio Grande do Norte, Brazil WESNYDY L. RIBEIRO, TIAGO M. U. ARAÚJO, THAÍS G. DO REGO, PEDRO H. V. FIGUEIREDO, SUANNY F. S. VIEIRA, THIAGO F. S. COSTA, CAIO C. MORAES, ALEXANDRE C. S. CRUZ, FELIPE A. ARAÚJO, and GUIDO L. SOUZA FILHO, Federal University of Paraiba, Brazil Automating the generation of audio descriptions (AD) for blind and visually impaired (BVI) people is a diicult task since it has several challenges involved, such as: identifying gaps in dialogues; describing the essential elements; summarizing and itting the descriptions into the dialogue gaps; generating an AD narration track, and synchronizing it with the main soundtrack. In our previous work (Campos et al. [8]), we propose a solution for automatic AD script generation, named CineAD, which uses the movie’s script as a basis for the AD generation. This paper proposes extending this solution to complement the information extracted from the script and reduce its dependency based on the classiication of visual information from the video. To assess the viability of the proposed solution, we implemented a proof of concept of the solution and evaluated it with eleven blind users. The results showed that the solution could generate a more succinct and objective AD but with a similar users’ level of understanding compared to our previous work. Thus, the solution can provide relevant information to blind users using less video time for descriptions. CCS Concepts: · Human-centered computing → Accessibility systems and tools. Additional Key Words and Phrases: Audio Description, Automatic, Accessibility, People with visual impairment 1 INTRODUCTION Blind and visually impaired (BVI) people have diiculty participating in and understanding the contents of ilms, television, and theater, among other visual media. In this context, audio description (AD) emerges as an accessibility resource that provides additional information to this audience. According to the American Council of the Blind (ACB) [1], the AD is a narration that guides the users through the presentation with concise, objective descriptions of new scenes, settings, costumes, and body language, among others, all slipped in between portions of dialogue or songs. Professionals usually produce AD scripts by hand, which is ultimately very time-consuming [25]. According to Lakritz and Salway [25], it requires approximately 60 hours on average for a professional to describe a two-hour movie. Szarkowska [38] noticed that the lengthy preparation process and associated high production costs are the Authors’ addresses: Virgínia P. Campos, vcampos@dca.ufrn.br; Luiz M. G. Gonçalves, lmarcos@dca.ufrn.br, Federal University of Rio Grande do Norte, 3000, Campus Universitario, Natal, Brazil; Wesnydy L. Ribeiro, wesnydy@lavid.ufpb.br; Tiago M. U. Araújo, tiagomaritan@lavid.ufpb.br; Thaís G. do Rego, thais@ci.ufpb.br; Pedro H. V. Figueiredo, pedroigueiredo@lavid.ufpb.br; Suanny F. S. Vieira, suanny@lavid.ufpb.br; Thiago F. S. Costa, thiago.ilipe@lavid.ufpb.br; Caio C. Moraes, caio.cesar@lavid.ufpb.br; Alexandre C. S. Cruz, santacruzalexandre@gmail.com; Felipe A. Araújo, felipealves@lavid.ufpb.br; Guido L. Souza Filho, guido@lavid.ufpb.br, Federal University of Paraiba, R. dos Escoteiros, s/n, Mangabeira, João Pessoa, Brazil. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proit or commercial advantage and that copies bear this notice and the full citation on the irst page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior speciic permission and/or a fee. Request permissions from permissions@acm.org. © 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM. 1936-7228/2023/4-ART $15.00 https://doi.org/10.1145/3590955 ACM Trans. Access. Comput.