How are prosodic features processed cognitively? A case study of the multimodal cognitive dynamics in simultaneous interpreting with text

This paper examines the effect of prosodic features on trainee interpreters’ cognitive effort, cognitive coordination and interpreting output in simultaneous interpreting with text. Sixteen trainee interpreters were recruited and randomly divided into two conditions to simultaneously interpret 27 sentences from English (L2) to Chinese (L1), with controlled prosodic features. Their interpreting output and eye-movements were recorded with the Eyelink 1000 Plus eye-tracking system. Various eye-metrics and temporal measures (i.e., ear-voice span, eye-voice span and ear-eye span) were analysed. The Simplified Chinese LIWC2015 Dictionary was employed to analyse the impact of prosodic features on the interpreting output. The findings indicate that prosodic features mitigate cognitive effort and facilitate the integration of auditory and visual inputs, as evidenced by the eyetracking measures. While some prosodic features function as a predictive cue for content shifts, enhancing cognitive coordination and information processing efficiency, others hinder the coordination process due to their signalling of uncertainty or emphasis. This study reveals the role of prosodic features in enabling interpreters to produce linguistically and thematically rich outputs, indicating that interpreters interpret not only the linguistic meaning but also the underlying sense.