Push the Limit of Acoustic Gesture Recognition

Yanwen WANG, Jiaxing SHEN, Yuanqing ZHENG*

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

28 Citations (Scopus)


With the flourish of the smart devices and their applications, controlling devices using gestures has attracted increasing attention for ubiquitous sensing and interaction. Recent works use acoustic signals to track hand movement and recognize gestures. However, they suffer from low robustness due to frequency selective fading, interference and insufficient training data. In this work, we propose RobuCIR, a robust contact-free gesture recognition system that can work under different practical impact factors with high accuracy and robustness. RobuCIR adopts frequency-hopping mechanism to mitigate frequency selective fading and avoid signal interference. To further increase system robustness, we investigate a series of data augmentation techniques based on a small volume of collected data to emulate different practical impact factors. The augmented data is used to effectively train neural network models and cope with various influential factors (e.g., gesture speed, distance to transceiver, etc.). Our experiment results show that RobuCIR can recognize 15 gestures and outperform state-of-the-art works in terms of accuracy and robustness.

Original languageEnglish
Pages (from-to)1798-1811
Number of pages14
JournalIEEE Transactions on Mobile Computing
Issue number5
Early online date19 Oct 2020
Publication statusPublished - May 2022
Externally publishedYes

Bibliographical note

The authors would like to thank the editor and reviewers for their help and insightful comments. This work was supported in part by the National Nature Science Foundation of China under grant 61702437 and Hong Kong GRF under Grant PolyU 152165/19E, and the Fundamental Research Funds for the Central Universities 531118010612.


  • Acoustic sensing
  • contact-free
  • data augmentation
  • gesture recognition
  • smart devices


Dive into the research topics of 'Push the Limit of Acoustic Gesture Recognition'. Together they form a unique fingerprint.
  • Push the Limit of Acoustic Gesture Recognition

    WANG, Y., SHEN, J. & ZHENG, Y., 2020, INFOCOM 2020 - IEEE Conference on Computer Communications. IEEE, p. 566-575 10 p. (IEEE Annual Joint Conference: INFOCOM, IEEE Computer and Communications Societies).

    Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

    70 Citations (Scopus)

Cite this