We present a real-time system for computing user location and orientation under constraints of memory, computation, and latency. The system utilizes a set of distributed microphone arrays with independent clocks. It computes angle-of-arrival (AoA) and time-difference-of-arrival (TDoA) information asynchronously, and deploys late fusion to compute the maximum likelihood user position. The user orientation is then computed using a model for the radiation pattern of the human voice. We also describe the Cramer-Rao Bound of the estimation, and a deep-neural-network (DNN) based model for computing the estimation confidence. The effectiveness of the proposed system is established using large datasets of real-life recordings under different room environments.