The cocktail party effect revisited in older and younger adults: When do iconic co-speech gestures help?

Understanding speech in noisy surroundings is notoriously difficult, especially for older adults. Previous research suggests that visual cues such as the speaker’s articulatory lip movements can significantly improve comprehension in both older and younger adults. Another potentially valuable visual cue for understanding speech in noise are iconic co-speech gestures, hand movements that depict semantic information related to the content of speech, e.g. tracing a circle while saying “ball”.

The current study investigates whether such gestures improve older and younger adults’ comprehension of speech presented in multi-talker babble noise beyond the benefit of visible lip movements, and whether this is modulated by the signal-to-noise ratio (SNR). Older and younger adults were presented with video clips of an actress uttering an action verb. After each video, participants had to select the uttered verb out of four verbs presented in a cued recall task. Videos were presented in three visual conditions (mouth blocked/audio only, visible lip movements, visible lip movements + co-speech gesture) and four audio conditions (clear speech, SNR -18, SNR -24, no audio).

Response accuracies showed no age-related differences in trials where either only auditory or only visual information was presented. Hence, older adults perform as well as younger adults for speech comprehension, lip reading, and gesture interpretation. However, older adults performed significantly worse than younger adults in those trials with combined visual and auditory input. Yet, both age groups benefitted equally from the presence of gestures in addition to visible lip movements as compared to visible lip movements without gestures; this benefit was significantly larger at the worst noise level.

These results suggest an overall age-related deficit in comprehending multi-modal language in noisy surroundings, potentially due to age-related declines in cognitive abilities (processing speed, working memory). Yet, iconic co-speech gestures provide additional semantic cues that can help both younger and older adults in disambiguating visible speech, particularly as the level of background noise increases.