P2-14: Multi-Task Loss for Spike based Sound Source Localization & Detection
Kshirasagar, Shreya*, Mayr, Christian, Guntoro, Andre
Subjects (starting with primary): Source localization and tracking ; Microphone array processing ; New and emerging topics in speech and audio processing
Presented in Poster Session 2
This work presents a multi-task framework using spiking architectures for sound source localization and detection. Our approach uses a dual-feature stream, processing Mel-Frequency Cepstral Coefficients (MFCCs) for Sound Identification (SID) and Generalized Cross-Correlation (GCC) features for Angle-of-Arrival (AoA) estimation. We explore recurrent and convolutional Spiking Neural Network (SNN) designs trained with a unified multi-task loss (MTL). Our proposed recurrent SNN achieves a Mean Absolute Error (MAE) of <2° on the localization task, outperforming the baseline with single injections. Furthermore, we demonstrate the capability of our networks to generalize to complex audio scenes with multiple non-overlapping sources.
This PDF is password protected. Enter the proceedings password in the PDF viewer when prompted.