P6-09: Real-time Speech Restoration using Data Prediction Mean Flows

Braun, Sebastian*

Subjects (starting with primary): New and emerging topics in speech and audio processing ; speech restoration

Presented in Poster Session 6

Abstract:

Generative models are capable to address difficult problems with non-unique solutions like bandwidth extension and gap filling, removing highly non-linear artifacts from codecs, clipping and distortion, as opposed to removing linear additive components like noise and reverb. While large offline processing models have shown impressive results, these tasks have not been solved with real-time capable models with low latency and compute. We propose a few-step flow matching model using Data Prediction Mean Flows in combination with suitable novel low-latency architecture to make flow matching models an attractive choice under theses constraints. Compared to state-of-the-art, our proposed mean flow model uses 120x less compute and introduces no algorithmic latency other than the STFT, while achieving similar audio quality.

This PDF is password protected. Enter the proceedings password in the PDF viewer when prompted.