RemoteGesture: Room-scale Acoustic Gesture Recognition for Multiple Users

As a promising way, controlling smart devices through gestures offers the benefits of non-contact interaction, efficiency and convenience. Previous researches on acoustic-based gesture recognition have mostly focused on near-field gestures within 1 meter and for a single user only. However, such a nearfield sensing scheme is inadequate to meet the growing demands for multi-person human-computer interaction in far-field spaces. In this paper, we present a novel acoustic-based room-scale gesture recognition system that is capable of recognizing gestures simultaneously performed by multi-user. Our approach achieves far-field sensing by examining the relationship between acoustic signal frame length and sensing range, and overcoming a series of practical challenges incurred by far-field sensing. To simultaneously detect and distinguish gestures of multiple persons, we divide the sensing area into multiple beamforming sub-scanning areas and apply binary search to detect multiple users, which allows for an efficient scanning process and facilitates real-time detection. Finally, we conduct a data augmentation scheme to enlarge the training data and apply a lightweight deep learning framework to classify different gestures. Extensive experiments confirm that our system enables multi-user gesture detection and can recognize nine gestures at a distance up to 7 meters.