P2-07: TARG-Speech: Task-Aligned Representation Geometry for Speech Enhancement Evaluation

Goes, Nina*, Kist, Andreas

Subjects (starting with primary): New and emerging topics in speech and audio processing ; Datasets for speech and audio ; Noise reduction

Presented in Poster Session 2

Abstract:

Standard speech enhancement (SE) evaluation relies on signal-level metrics such as PESQ and SI-SNR. These track downstream model performance well under additive noise, but we show they break down under convolutive distor- tion, where the relevant question is whether the enhanced signal preserves task-relevant structure rather than wave- form fidelity. We propose TARG-Speech, a clean-referenced representation-space diagnostic that projects wav2vec 2.0 embeddings into a linear discriminant analysis (LDA) space fitted on labeled clean training data of PERCEPT-R, and quantifies label-wise class separability there. Across the non- clean condition × processing-state panels in our benchmark, the Q1–Q4 centroid distance in LDA space is strongly asso- ciated with downstream macro-F1 with Spearman ρ = 0.954 over n = 35 panels. As the subspace comes from a dis- joint speaker-independent partition, this association is out- of-sample. The comparison is two-regime: under additive noise PESQ, STOI and SI-SNR match or beat it, whereas under reverberation STOI and SI-SNR become uninforma- tive (−0.333, 0.048), PESQ retains no significant association (0.476), and only the geometry remains significantly associ- ated with downstream performance (0.952). The same pattern holds for the binary-label LDA-space silhouette across three self-supervised learning (SSL) backbones (wav2vec 2.0, Hu- BERT, WavLM; ρ = 0.929–0.963). The best front-end per regime recovers 39–50% of the clean-to-degraded gap under noise but only 17–19% under reverberation, and one system falls below the degraded baseline under noise. Supervised by task-aligned clean labels, the diagnostic complements classifier-based evaluation rather than replacing it.

This PDF is password protected. Enter the proceedings password in the PDF viewer when prompted.