P1-04: LaserSpeech: Physics-Based Modeling of Optical Vibrometry for Large-Scale Laser Speech Recognition

bederov, emily*, berdugo, baruch, cohen, israel

Subjects (starting with primary): Datasets for speech and audio ; automatic speech recognition

Presented in Poster Session 1

Abstract:

Laser Doppler vibrometry (LDV) can recover speech from vibrating surfaces with high precision, yet the prohibitive cost of research-grade LDV hardware $10,000-$100,000 per unit) has kept the field starved of training data, leaving laser-based automatic speech recognition (ASR) and reconstruction far behind their radar counterparts. In this paper, we introduce LaserSpeech, a large-scale synthetic dataset and open-source benchmark designed to close this gap. Our physics-based synthesis pipeline transforms clean speech into realistic laser-like recordings by modeling the five principal degradation mechanisms of optical vibrometry: low-pass bandwidth limiting, frequency-dependent (purple) sensor noise, broadband measurement noise, Nyquist anti-aliasing, and multipath frequency smearing. Applied to the 460 hour LibriSpeech (train-clean-460) split, the pipeline yields 460 hours of paired clean/synthetic-laser speech at no hardware cost. We validate the synthetic data by measuring PESQ, STOI, and Whisper WER on the synthesized LibriSpeech utterances relative to their clean originals and compare those metrics to the same measurements on real LDV recordings across 2 experimental sessions. A systematic joint parameter ablation over 18 configurations, evaluated on 1,585 LibriSpeech utterances (4.3h), identifies the synthesis operating point whose acoustic quality profile best matches the real LDV target distribution across two surface conditions (Wooden Box and Kappa), and reveals which parameters control degradation difficulty versus speech preservation. The synthesis code and the real LDV recordings are released publicly to support research on cross-speaker and cross-domain generalization and on pre-training for laser-based speech sensing.

This PDF is password protected. Enter the proceedings password in the PDF viewer when prompted.