Advances in person perception research: Processing identity and emotion in faces and voices

While other-“race” (OR) faces and caricatures of same-“race” (SR) faces evoke similar ERP patterns, behavioural effects differ (advantage for SR caricatures, disadvantage for OR faces). These paradoxical findings might be explained by qualitatively similar processes at learning, which have different consequences for recognition: Norm deviations are used for forming basic face representations. In caricatures, distinctive information is idiosyncratic and helpful, but for OR faces, salient deviations are misleading, as they are unidirectional relative to the norm. In three experiments, we simulated an OR effect with highly distinctive same-“race“ faces: In each experiment, one characteristic in SR faces was manipulated, always in a uniform direction (E1: big noses, E2: freckled skin, E3: distinctive blue eyes), resulting in deceptive distinctiveness. In a learning/recognition task, we compared performance and ERPs for these faces to veridical SR and OR faces. We found strikingly similar ERPs for OR and manipulated SR faces, accompanied by comparable costs in performance, in contrast to veridical same-“race” faces, supporting a perceptual account of the OR effect. This suggests qualitatively similar processes mediating the learning of unfamiliar SR and OR faces, but with different consequences due to differences in the usefulness of respective distinctive information. Deceptive distinctiveness can explain the phenomenon that is it harder to learn faces of people who look systematically different from what we are used to. This phenomenon is not specific to (the questionable concept of) other “races”, and is related to deviations from an observer´s mental norm in physical appearance instead.

This talk will outline our recent findings regarding differences between familiar and unfamiliar face processing. By using a Fast Periodic Visual Stimulation (FPVS) paradigm and measuring steady-state visual evoked potentials (ssVEP), different cortical responses were detected in the past for objects and faces as well as for identical or different faces. In the current study we tested if a differential ssVEP response is also present for familiar when compared to unfamiliar faces in two populations (German and Belgian) of participants (n = 30). We presented faces of celebrities either familiar (e.g. German celebrities to Germans) or unfamiliar (e.g. French celebrities to Germans; presentation frequency 0.85 Hz) among similar, but unknown faces (presentation rate 6 Hz) in two experiments. During the first experiment, multiple familiar faces were presented to evaluate familiarity while in the second experiment faces of one single familiar person were shown to test the differential ssVEP response to the individual identity. We could observe an enhanced ssVEP amplitude over the occipital-temporal (OT) regions to familiar when compared to unfamiliar faces in both participant groups. Importantly, we could demonstrate different ssVEP responses to the exact same face within the two cultural participant groups, signaling that familiarity determined the observed differential ssVEP responses. Moreover, a positive correlation between subjective familiarity rating and the magnitude of ssVEP response was found. Our results confirm further the theory that familiar and unfamiliar faces are processed differentially in the ventral stream.

The human face informs about the identity and emotion of other people. Responding to such social signals depends on what we have previously learned about these persons. Combining instructed threat and reversal learning paradigms, three studies examined the capability of facial information (person identity and emotional expression) as signals for threat-of-shock or safety. To this end, threat/safety contingencies were verbally instructed (e.g., Person A and B indicate threat, C and D safety), and partially reversed across the experiment (e.g., now Person B cues safety and D threat). Study 1 examined whether facial expressions are equally effective in cueing threat or safety. Study 2 focused on electrocortical processing while viewing face identities that were explicitly instructed as threat or safety cues, and Study 3 followed up on psychophysiological responding to attachment figures (i.e., pictures of loved people) serving as threat or safety-cues. Taken together, main effects of threat instructions and facial emotion were confirmed (e.g., threat-potentiated startle reflex; emotion-enhanced EPN and LPP components). Moreover, happy and angry facial expressions served equally well as instructed threat cues (Study 1). Regarding perceptual processing (Study 2), facilitated face encoding of instructed and reversed threat-cues was observed (e.g., indicated by N170 and EPN). Finally, threat instructions do not spare beloved ones (Study 3), who may serve as threat cues similar as pictures of unknown people. In conclusion, perceptual processing and responding to facial information varies according to the mere verbal communication about whether a person is dangerous or safe.

The aim of this talk is to demonstrate how parameter-specific morphing techniques can enhance our understanding of the role of sensory information for the processing of social and affective signals from faces and voices. Following the inventions of visual (image) morphing technology by Benson and Perrett around 1990, and of auditory morphing 〜ten years later by Kawahara, these techniques can be used beyond classical morphing, averaging, or caricaturing. Specifically, we use them to selectively manipulate independent parameters (e.g., 3D-shape and texture/colouration of faces, or fundamental frequency, timbre, and temporal aspects of voices). This allows us to determine the relative importance of these image (or sound) characteristics for social perceptions of age, gender, identity or affect in neurotypical participants, but also in individuals with sensory or central impairments. For example, experiments with faces consistently reveal a dominant role of texture information over shape in most individuals, whereas individuals with poor face recognition skills appear to rely disproportionately on shape -at variance with traditional claims that spatial configural information is crucial to familiar face recognition. For voices, we present current results that provide relevant information about how hearing-impaired cochlear implant users, compared to normal hearing listeners, use acoustic information for perceiving emotions and other social signals in voices. Overall, we demonstrate how parameter-specific morphing is a promising novel approach to objectively assess profiles of face and voice perception abilities.

Although previous research has demonstrated perceptual aftereffects in emotional voice adaptation, the contribution of different vocal cues to this effect is unclear. We used parameter-specific morphing of adaptor voices to investigate the relative roles of fundamental frequency (F0) and timbre in vocal emotion adaptation, using angry and fearful utterances. Thirty young adults (15 females, 18-26 years [M = 21.6; SD = 2.3]) adapted to morphed voices containing 100% emotion-specific information in either F0 or timbre, with all other parameters kept at an intermediate 50% morph level. As reference conditions, full adaptors (with all parameters at an emotion-specific level of 100%) and ambiguous adaptors (with all parameters non-informative at 50%) were used. Stimuli were created using TandemSTRAIGHT software. Consistent aftereffects were found in all three conditions (dFull = 1.83 , dTimbre = 1.31, dF0 = 0.51). Crucially, aftereffects following timbre adaptation were much larger than in the F0 condition (t(29) = -3.36, p = .002, d = 0.56) and only nonsignificantly smaller than those elicited by full adaptors (t(29) = -1.95, p = .060, d = 0.35). These results suggest a prominent role of timbre information, and a smaller role of F0, in vocal emotion adaptation. Although these findings are limited to angry and fearful voices, they add to the growing body of evidence suggesting a major role of timbre in auditory adaptation.