A Critical Review of Multimodal-Multisensor Analytics for Anxiety
Assessment
HASHINI SENARATNE, SHARON OVIATT, and KIRSTEN ELLIS, Monash University, Australia
GLENN MELVIN, Deakin University, Australia
Recently, interest has grown in the assessment of anxiety that leverages human physiological and behavioral data to address
the drawbacks of current subjective clinical assessments. Complex experiences of anxiety vary on multiple characteristics,
including triggers, responses, duration and severity, and impact diferently on the risk of anxiety disorders. This article
reviews the past decade of studies that objectively analyzed various anxiety characteristics related to ive common anxiety
disorders in adults utilizing features of cardiac, electrodermal, blood pressure, respiratory, vocal, posture, movement and
eye metrics. Its originality lies in the synthesis and interpretation of consistently discovered heterogeneous predictors of
anxiety and multimodal-multisensor analytics based on them. We reveal that few anxiety characteristics have been evaluated
using multimodal-multisensor metrics, and many of the identiied predictive features are confounded. As such, objective
anxiety assessments are not yet complete or precise. That said, few multimodal-multisensor systems evaluated indicate
an approximately 11.73% performance gain compared to unimodal systems, highlighting a promising powerful tool. We
suggest six high-priority future directions to address the current gaps and limitations in infrastructure, basic knowledge and
application areas. Action in these directions will expedite the discovery of rich, accurate, continuous and objective assessments
and their use in impactful end-user applications.
CCS Concepts: • Applied computing → Health informatics;• Human-centered computing;
Additional Key Words and Phrases: Anxiety, Multimodal Analytics, Physiological and Behavioral Data, Ubiquitous Technology
1 INTRODUCTION
Anxiety is a normal human emotion, but excessive and prolonged anxiety may indicate anxiety disorders that are
highly prevalent, impairing, and costly for society. Anxiety disorders are a group of mental health conditions
characterized by fearful vigilance, worry and somatic symptoms [8]. About 284 million people worldwide are
afected by these disorders [175]. Despite being treatable and likely preventable [21, 106], the prevalence of
anxiety disorders is increasing at a rate of 14.9% per decade [156]. According to the Diagnostic and Statistical
Manual of Mental Disorders (DSM-5), ive common anxiety disorder types occur in adults: social anxiety disorder,
speciic phobia, generalized anxiety disorder, agoraphobia, and panic disorder [8] (see Table 1 for a glossary of
terms). These disorders impair people in social, educational, occupational and family functioning, and increase
the risk of unemployment, lower incomes and problematic relationships [11]. Afected people are also highly
vulnerable to other mental disorders [111], physical illnesses such as cancers [17] and suicidal ideation and
behaviors [136]. Health care costs in afected people are twice as much as in healthy individuals [128]. Indirect
costs (due to morbidity and mortality) of these disorders are about three times the direct medical costs [6].
Given the above context, early and continued access to assessments that can accurately detect and predict
the onset and maintenance of anxiety disorders is crucial. However, current standard clinician-administered
Authors’ addresses: Hashini Senaratne; Sharon Oviatt; Kirsten Ellis, Monash University, Australia; Glenn Melvin, Deakin University, Australia.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that
copies are not made or distributed for proit or commercial advantage and that copies bear this notice and the full citation on the irst page.
Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy
otherwise, or republish, to post on servers or to redistribute to lists, requires prior speciic permission and/or a fee. Request permissions from
permissions@acm.org.
© 2022 Copyright held by the owner/author(s). Publication rights licensed to ACM.
2637-8051/2022/8-ART $15.00
https://doi.org/10.1145/3556980
ACM Trans. Comput. Healthcare