Remote microphone arrays offer a promising solution for improving speech intelligibility in assistive listening scenarios, particularly in multi-speaker environments such as group conversations. However, beamforming with such arrays typically produces monaural signals that are not temporally aligned with the signals received at the listener’s ears and lack binaural spatial cues, resulting in degraded spatial perception. In this paper, we propose a geometry-driven framework for binaural reconstruction using a remote microphone array. Experiments demonstrate that the proposed approach improves spatial consistency while maintaining strong signal enhancement compared to adaptive baselines. In addition, the method enables faster switching between talkers, making it well-suited for dynamic conversational settings.