We propose a deep unfolded Recursive EM (REM) network for robust tracking of a single moving speaker in mild reverberant environments. Unlike classical REM algorithms, which rely on fixed-step-size decay schedules, the proposed architecture learns an adaptive update policy by unfolding the iterative procedure into differentiable layers. We introduce a Step Size Network that leverages Feature-wise Linear Modulation (FiLM) and Positional Encoding (PE) to dynamically adjust the recursion weights based on temporal context and convergence state. Experimental results for tracking a single speaker under reverberant conditions demonstrate that the proposed unfolded network outperforms the classical Cappé and Moulines REM (CREM) baseline, which employs a spatial grid search to map the estimated centroids to physical positions. In the single-speaker tracking task, the proposed method achieves a lower Root Mean Square Error (RMSE) than the CREM baseline, highlighting its potential for dynamic acoustic scenarios.