User profiling based on nonlinguistic audio data

Jiaxing SHEN, Oren LEDERMAN, Jiannong CAO, Shaojie TANG, Alex Lsandy PENTLAND

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review


User profiling refers to inferring people's attributes of interest (AoIs) like gender and occupation, which enables various applications ranging from personalized services to collective analyses. Massive nonlinguistic audio data brings a novel opportunity for user profiling due to the prevalence of studying spontaneous face-to-face communication. In this poster, we are the first to build a user profiling system to infer gender and personality based on nonlinguistic audio. Instead of linguistic or acoustic features which are unable to extract, we focus on conversational features that could reflect AoIs. We firstly develop an adaptive voice activity detection algorithm that could address individual differences in voice and false-positive voice activities caused by people nearby. Secondly, we propose a gender-assisted multi-task learning method to combat dynamics in human behavior by integrating gender differences and the correlation of personality traits. The experimental evaluation of 100 people in 273 meetings indicates the superiority of the proposed method in gender identification and personality recognition respectively.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE 37th International Conference on Data Engineering (ICDE)
Number of pages6
ISBN (Electronic)9781728191843
ISBN (Print)9781728191850
Publication statusPublished - 2021
Externally publishedYes
Event37th IEEE International Conference on Data Engineering, ICDE 2021 - Chania, Greece
Duration: 19 Apr 202122 Apr 2021

Publication series

NameProceedings - International Conference on Data Engineering
ISSN (Print)1084-4627


Conference37th IEEE International Conference on Data Engineering, ICDE 2021


  • Gender identification
  • Multi-task learning
  • Nonlinguistic audio
  • Personality recognition
  • User profiling


Dive into the research topics of 'User profiling based on nonlinguistic audio data'. Together they form a unique fingerprint.

Cite this