UMBC faculty receive NSF award for audio deepfake detection

UMBC professors Vandana Janeja, Anita Komlodi, Christine Mallinson, and Sanjay Purushotham received an NSF award to develop a prototype Community Infrastructure to Strengthen AI for Audio Deepfake analysis (CISAAD).  Deepfakes, or AI-generated content, are widely recognized as a major societal concern and challenge and audio deepfakes are increasingly being used to spread disinformation on the web and social media and for identity theft and financial scams.

The CISAAD project builds on a 2022 NSF EAGER grant to Drs. Janeja and Mallinson that studied and evaluated listener perceptions of audio deepfakes and novel interdisciplinary methods for detection. The new effort has three main goals:

• Helping to solve the challenges of limited data availability for audio deepfakes and human-augmented data about them through open datasets shared by the community

• Enabling  both single and multi-speaker deepfake analysis across various use cases 

• Addressing the ethical, social, and political challenges associated with deploying deepfake technology developed from open-sourced community data

Their work will inform our understanding of mis/dis-information as a significant societal concern and challenge. It will also help understand the opportunities for content generation in positive applications such as voice restoration and smart and connected community research. 

With an interdisciplinary team across AI, linguistics, cyber infrastructure, and human-centered computing, the project develops an innovative infrastructure for expanding research informed by types of audio deepfakes. Their research and dissemination efforts will expand formal and informal learning in AI and STEM fields related to cybersecurity analytics at the intersection of technology, language, behavior, and society. The principles developed through this project will expand to multiple types of deepfakes and support media and communications experts working to address challenges related to information integrity.

CISAAD will be developed as a prototype community resource. It will include a deep fake data catalog and repository for English audio data, tools, models for deep fake audio analysis use cases, and educational material.