Journal of Autism and Developmental Disorders, Vol. 32, No. 4, August 2002 (© 2002)
Visual Scanning of Faces in Autism Kevin A. Pelphrey,1,2,4 Noah J. Sasson,1 J. Steven Reznick,1 Gregory Paul,3 Barbara D. Goldman,1 and Joseph Piven2,3
The visual scanpaths of five high-functioning adult autistic males and five adult male controls were recorded using an infrared corneal reflection technique as they viewed photographs of human faces. Analyses of the scanpath data revealed marked differences in the scanpaths of the two groups. The autistic participants viewed nonfeature areas of the faces significantly more often and core feature areas of the faces (i.e., eyes, nose, and mouth) significantly less often than did control participants. Across both groups of participants, scanpaths generally did not differ as a function of the instructions given to the participants (i.e., “Please look at the faces in any manner you wish.” vs. “Please identify the emotions portrayed in these faces.”). Autistic participants showed a deficit in emotion recognition, but this effect was driven primarily by deficits in the recognition of fear. Collectively, these results indicate disorganized processing of face stimuli in autistic individuals and suggest a mechanism that may subserve the social information processing deficits that characterize autism spectrum disorders. KEY WORDS: Face processing; autism; eye movements; visual scanpaths; social perception.
and lack of interest in the human face, appearing very early in ontogeny (Baird et al., 2000; Baron-Cohen et al., 1996; Cohen & Volkmar, 1997; Kanner, 1943; Osterling & Dawson, 1994). The ability to judge facial expressions and derive other socially relevant information from faces is a fundamental requirement for normal reciprocal social interactions and interpersonal communication. Even in the earliest stages of postnatal development, faces are highly salient to typically developing individuals. For instance, newborn infants will spend more time fixating canonical face-like stimulus than scrambled patterns resembling faces to varying degrees (Goren, Sarty, & Wu, 1975; Johnson, Dziurawic, Ellis, & Morton, 1991; Johnson & Morton, 1991; Morton & Johnson, 1991). This early preferential behavior is further refined as infants experience a variety of conspecific faces (Morton & Johnson, 1991). Indeed, the face-directed behaviors of the newborn mark the beginning of a protracted developmental pathway by which a small repertoire of basic abilities is transformed into a highly sophisticated and mature face processing system. Faces may represent an exceptional class of stimuli for normal adults (Farah, Wilson, Drain, & Tanka,
INTRODUCTION Autism is a pervasive neurodevelopmental disorder associated with considerable impairment. Prevalence is estimated to be approximately 1:1000 children (Centers for Disease Control and Prevention, 1997). Deficits in the use of nonverbal social communicative behaviors (e.g., eye-to-eye gaze and facial expression) and stereotyped patterns of behavior (e.g., persistent preoccupation with parts of objects) are two characteristics of the autism phenotype (DSM-IV; American Psychiatric Association, 1994). These characteristics show a distinct developmental course, with preliminary indications, such as a preference for inanimate objects 1
Department of Psychology, University of North Carolina at Chapel Hill. 2 UNC Neurodevelopmental Disorders Research Center, University of North Carolina at Chapel Hill. 3 Department of Psychiatry, Division TEACCH, University of North Carolina at Chapel Hill. 4 Correspondence should be addressed to Kevin A. Pelphrey, Brain Imaging and Analysis Center, Duke University Medical Center, 163 Bell Building, P.O. Box 3918, Durham, NC 27710; e-mail:
[email protected]
249 0162-3257/02/0800-0249/0 © 2002 Plenum Publishing Corporation
250 1998; Kanwisher, McDermott, & Chun, 1997). For example, face perception is usually holistic or configural rather than elemental or piecemeal. To identify a particular face or a facial expression, normally developing individuals generally rely on the spatial configuration of the major features of the face, including the eyes, nose, and mouth (Diamond & Carey, 1986; Farah et al., 1998; Gauthier, Anderson, Tarr, Skudlarski, & Gore, 1999; Gauthier & Tarr, 1997; Valentine, 1988). This holistic processing strategy may be contrasted with the kind of processing strategy typically involved in nonface object recognition, which characteristically relies on the detection of individual features (i.e., a segmental processing strategy) and not the overall configuration of an object (Diamond & Carey, 1986; Farah et al., 1998; Gauthier & Tarr, 1997; Gauthier et al., 1999; Tarr & Bulthoff, 1998; Valentine, 1988). Evidence for the holistic processing of faces comes primarily from studies of the “face inversion effect.” Holistic processing is disturbed when a face stimulus is inverted and the normal configuration of the core facial features is changed, thereby making face perception more difficult (Haxby et al., 1999; Valentine, 1988). Inverting a face stimulus may force individuals to adopt the type of segmental processing strategies that are more characteristic of nonface object perception. Because objects are processed in a more segmental than holistic fashion, object perception shows less of an inversion effect (Farah et al., 1998; Valentine, 1988). Other research indicates that particular cortical regions are specialized for the processing of faces. For instance, neuropsychological studies indicate that some subjects with damage to particular regions of the brain show impaired face recognition but intact object recognition (prosopagnosia). Other patients show the reverse pattern of deficits. This double dissociation has been interpreted as evidence that face and nonface object recognition rely on different neural substrates (De Renzi et al., 1994). The results of neuroimaging investigations using positron emission tomography (PET) (Sergent, Ohta, & MacDonald, 1992), functional magnetic resonance imaging (fMRI) (Kanwisher et al., 1997), magnetoencephalography (MEG) (Sams et al., 1997), and event-related potentials (ERPs) (Bentin et al., 1996) demonstrate that occipitotemporal cortical areas, particularly parts of the fusiform gyrus, activate more in response to faces than to other stimuli. In addition to the evidence concerning cortical specialization, the right hemisphere of the brain may be relatively more involved than the left hemisphere in processing facial information such as identity and
Pelphrey et al. emotion. Ellis and Shepard (1975) demonstrated that faces are recognized more quickly and more accurately when presented to the right hemisphere (left visual field) than when they are presented to the left hemisphere (right visual field). Studies of cortical lesions in prosopagnosia indicate that damage limited to the right hemisphere is sufficient to cause impairments in face processing (De Renzi et al., 1994). Finally, studies using neuroimaging techniques show greater activation in the right than in the left hemisphere areas during face processing tasks (e.g., Kanwisher et al., 1997). This hemispheric difference may occur because relational encoding is mediated primarily by the right hemisphere and feature based encoding is mediated by the left (Farah, 1990) and/or because processing of socioemotional information is mediated primarily by the right hemisphere (Cicone, Wapner, & Gardner, 1980). Investigations that have directly measured point of regard during face perception indicate that normal adults direct most of their attention to the core features of the face (i.e., eyes, nose, and mouth) and spend very little time on nonfeature areas (e.g., Luria & Strauss, 1978; Mackworth & Bruner, 1970; Manor et al., 1999; Mertens, Siegmund, & Grusser, 1993; Noton & Stark, 1971; Phillips & David, 1997; Streit, Wölwer, & Gabel, 1996; Walker-Smith, Gale, & Findlay, 1977; Yarbus, 1967). For example, Walker-Smith and colleagues (1977) recorded visual scanpaths in normal adults as they viewed a face stimulus. Participants devoted the vast majority of their fixations to the eyes, nose, and mouth, with nearly 70% of these fixations directed to the eyes. Given the results inidicating both qualitative and quantitative differences between the face and nonface object perception strategies adopted by normal adults, it is interesting to note that autistic individuals may rely more on individual parts of the face for identification (e.g., the lower face and mouth area) than the overall configuration (e.g., Hobson, Ouston, Lee, 1988; Langdell, 1978). That is to say, individuals with autism may spontaneously process different aspects of facial information than do individuals without autism, perhaps adopting a segmental approach to inspecting faces. Consistent with this hypothesis, studies have found that individuals with autism spectrum disorders show less of an inversion effect for faces and better object perception than expected based on their ability with faces (e.g., Boucher & Lewis, 1992; Davies, Bishop, Manstead, & Tantam, 1994; Hauck et al., 1998; Hobson et al., 1988; Langdell, 1978; Tantam et al., 1989). Individuals with autism also show deficits in their ability to recognize familiar faces (Boucher & Lewis, 1992;
Visual Scanning of Faces in Autism Braverman et al., 1989; Klin et al., 1999; Langdell, 1978; Ozonoff, Pennington, and Rogers, 1990; Tantam et al., 1989) and in their ability to identify emotional expressions (Capps, Yirmiya, & Sigman, 1992; Celani, Battachi, & Arcidiacono, 1999; de Gelder, Vroomen, & van der Heide, 1991; Hobson, 1986a, 1986b; Hobson et al., 1988, 1989; Loveland et al., 1997; Ozonoff et al., 1990). Although it should be noted that some studies have failed to find differences in affect recognition performance between groups of high-functioning autistic and control participants when the identification task involved very simple or basic emotions such as happiness (e.g., Adolphs, Sears, & Piven, 2001; Baron-Cohen, Spitz, & Cross, 1993; Baron-Cohen, Wheelwright, & Jolliffe, 1997; Volkmar, Sparrow, Rende, & Cohen, 1989). The behavioral findings concerning deficits in face perception in autistic individuals are complemented by results obtained from a recent neuroimaging investigation. Schultz and his colleagues (2000) used fMRI to examine face and common object perception in highfunctioning individuals with autism or Asperger syndrome and normal controls. During face but not object discrimination, individuals with autism and Asperger syndrome had significantly greater activation in the right inferior temporal gyri and less activation of the right fusiform gyrus than nonautistic controls. In contrast, the more typical pattern of greater fusiform gyrus activation was found in control participants during face processing, with greater right inferior temporal gyri activation observed during object processing. These findings suggest that individuals with autism spectrum disorders may process faces as though they were objects, adopting the segmental perceptual strategies that are more characteristic of those adopted by normal individuals during nonface object perception. In summary, converging sources of evidence indicate that autistic individuals perceive faces differently than do normally developing individuals and that these differences may underlie deficits in face perception and facial affect recognition among autistic individuals. However, no study to date has specifically compared the visual scanpaths of autistic and nonautistic individuals in response to face stimuli. The most direct, realtime method for assessing these processing strategies is to record visual scanpaths (Noton & Stark, 1971), the pattern of eye movements that occur when an individual processes a complex stimulus. This comparison is valuable because it may reveal clues to understanding the autistics’ often reported difficulties with social communication and thus add to our understanding of the autism phenotype. In the present investigation, the vi-
251 sual scanpaths of five high-functioning adult males with autism and five adult males without autism were recorded using an infrared corneal reflection technique as they viewed photographs of human faces. These data can address an important research question. How do adult males with autism scan human faces? METHODS Participants Five adult males with a diagnosis of autism (mean age 25.2 years, range 19.1 to 30.2 years) were recruited through the Treatment and Education of Autistic and Related Communication Handicapped Children (TEACCH) program in Chapel Hill, North Carolina. All members of the autism group met DSM-IV/ICD-10 diagnostic criteria for autism. Diagnoses were confirmed with the Autism Diagnostic Interview-Revised (ADI-R) (Lord, Rutter, & LeCouteur, 1994) and/or the Autism Diagnostic Observation Schedule (ADOS) (Lord, Rutter, DiLavore, & Risi, 1999). All had IQs in the normal range and thus comprised a group of high-functioning autistic participants. The mean full-scale Wechsler Adult Intelligence Scale-Revised (WAIS-R) (Wechsler, 1981) score was 100.75 (SD ⫽ 7.69). Average performance and verbal WAIS-R scores were 86.50 (SD ⫽ 9.57) and 117.00 (SD ⫽ 23.12), respectively. For the comparison group, five unaffected adult males (mean age 28.2 years, range 25.2 to 32.2 years) were recruited from the local community. These participants had no history of neurological or psychiatric illness. On average, members of the comparison group had completed a college degree and some had undertaken postgraduate work. Before participation, all individuals gave informed written consent. The study was approved by the Human Participants Investigations Committee of the University of North Carolina at Chapel Hill School of Medicine. All participants were tested individually in the same laboratory room by the same three experimenters. Apparatus and Materials An ISCAN series RK-464 remote infrared pupilcorneal reflection eye movement monitoring system (ISCAN Inc., Cambridge, MA) was used to record participant point of regard data. The eye movement monitoring system consisted of a dark pupil/corneal reflex videooculographic eye gaze camera, an infrared light source, a host computer, and auxiliary video display units for observing the monitored eye and participant point of regard on the stimulus scene. A data stream
252 representing the participant’s point of regard within the stimulus scene was transmitted (in real time at 60 Hz) to the host computer for recording. The error resolution for the reported point of regard data was less than 0.50° of visual angle over ⫾20° horizontal and vertical ranges. Stimuli consisted of high-resolution monochromatic digital photographs selected from the standardized set of Ekman and Friesen’ (1976) “pictures of facial affect.” This collection of posed facial expressions contains the six basic emotions of fear, anger, disgust, happiness, sadness, and surprise. Extensive investigation has shown high agreement across cultures in the assignment of these basic emotions to corresponding facial expressions (see Ekman, 1989, for a detailed review of these studies). The stimuli were presented on a 69-cm video monitor and subtended a horizontal visual angle of 10.7° and a vertical visual angle of 14.2°. Procedure Assessment took place in a darkened room. After the participants were acquainted with the apparatus and experimental set-up, they were seated in a comfortable chair in front of the video monitor and were asked to place their chin in a chin rest. The chin rest aided in the processes of point of regard tracking and ensured a distance of 80 cm between the center of the stimulus display screen and the participant’s eyes. After a brief calibration procedure, the stimulus pictures were exposed. The experimental procedure was divided into two phases. In phase I, participants were shown 12 faces from the Ekman and Friesen series with one male and one female face for each of the six basic emotions. Participants were instructed to look at the photographs in any manner they selected (i.e., “Please look at the faces in any manner you wish.”). Each picture was presented for 2 seconds with a 2-second interstimulus interval. Eye movement data were recorded for post experimental processing. During phase II, participants were shown 24 additional faces from the Ekman and Friesen series. These faces were selected from among the photographs that approximately 90% or more of the Ekman and Friesen (1976) normative sample had identified as portraying a particular emotion. The 24 photographs were balanced by gender and emotion so that they consisted of two male and two female faces for each of the six basic emotions. Participants were instructed to identify the emotion portrayed in each picture (i.e., “Please identify the emotions portrayed in these faces.”). Each picture was presented for 2 seconds with a 5-second interstimulus interval. A list of the six basic emotions was
Pelphrey et al. presented during the interstimulus interval to assist the participants in selecting from among the possible choices. Two practice trials (one trial with a face portraying happiness and another with a face portraying sadness) were administered before the start of this phase to ensure that each participant understood the emotion recognition task. All of the participants indicated that they understood the task and the meanings of the six emotion terms. Participants’ verbal responses were recorded by an experimenter and the eye movement data were recorded for post experimental analysis. Eye Movement Data Filtering, Reduction, and Analysis Each 2-second epoch of eye movement data was manually checked for tracking integrity. Eye blinks were identified by a loss of corneal reflection and were excluded from subsequent data analysis, as were off screen gazes. Point of regard data were collected at a rate of 60 samples per second, which provided up to 120 data points during each of the twelve 2-second epochs for each participant. Regions of interest were defined for each face based on the location of the core facial features (i.e., eyes, nose, and mouth). To define these regions, each face was divided into 48 equally sized blocks subtending horizontal and vertical visual angles of 1.8°. Then, for each face, the blocks covering the core facial features were identified. On average, 13 blocks (27%) outlined the key feature regions of the eyes, nose, and mouth, whereas 35 blocks (73%) defined the remaining nonfeature regions of the face. The fundamental variables for quantitative comparative analyses included the percentage of time (i.e., number of sample points out of the points collected) in which the participant’s point of regard was on primary feature versus nonfeature areas of the face as well as the percentage of fixations made on primary facial features versus nonfeature areas of the face. Moreover, the average duration of fixations and the mean number of fixations per stimulus face were calculated for each subject. A fixation was defined as a set of consecutive gaze coordinates, confined within a diameter of 1° of visual arc for a duration of 200 milliseconds or more (Noton & Stark, 1971). A K-means cluster analysis algorithm was developed and used to compute the number of fixations and fixation duration for feature (eyes, nose, mouth) and nonfeature (remaining facial regions) areas (see Latimer, 1988, for a detailed review of similar cluster analytic techniques for use with eye movement data). Finally, in addition to these quantitative indices, each scanning pattern was plotted for inspection and qualitative analysis.
Visual Scanning of Faces in Autism RESULTS Preliminary analyses indicated no differences in scanpaths as a function of stimulus face gender, identity, or emotion portrayed. Hence, subsequent analyses were conducted with the eye movement data collapsed across these three variables. Phase I: Visual Scanning of Faces Typical scanpaths from phase I for three participants with autism and three controls are shown in Figure 1. Each symbol represents a data-sampling point (0.0167 second). The lines connecting these points trace the visual scanpath taken by the subject over the face stimulus. As can be seen in the figure, qualitative differences in the scanpaths of autistic and control participants were evident. As illustrated in the top panel of Figure 2, individuals in the autism group devoted a smaller percentage of time (M ⫽ 56.17%, SD ⫽ 7.08%) to examining the core features of the faces than did participants in the control group (M ⫽ 91.28%, SD ⫽ 6.66%). This effect appeared to be strongest for the percentage of time spent examining the eyes (autism: M ⫽ 50.87%), SD ⫽ 9.16%, control: M ⫽ 79.04%, SD ⫽ 6.49%). Group differences were smaller for the percentage of time spent on the nose (autism: M ⫽ 4.86%, SD ⫽ 4.79%); control: M ⫽ 10.63%, SD ⫽ 2.81% and the mouth (autism: M ⫽ 0.43%, SD ⫽ 0.36%; control: M ⫽ 1.60%, SD ⫽ 1.02%). Similarly, as shown in the top panel of Table 1, autistic participants devoted a smaller percentage of their fixations to the core facial features during Phase I (M ⫽ 71.98%, SE ⫽ 7.62%) than did control participants (M ⫽ 92.62%, SE ⫽ 3.15%). The largest group difference was observed for the percentage of fixations on the eyes (autism: M ⫽ 65.56, SE ⫽ 5.39%; control: M ⫽ 81.64, SE ⫽ 1.87%). The average number of fixations per face (autism: M ⫽ 4.28, SE ⫽ 0.12; control: M ⫽ 4.60, SE ⫽ 0.24) and the average fixation duration (autism: M ⫽ 0.35 second, SE ⫽ 0.01 second; control: M ⫽ 0.33 second, SE ⫽ 0.02 second) did not appear to differ as a function of group membership. Phase II: Visual Scanning of Faces during an Emotion Recognition Task Figure 3 shows typical scanpaths for three participants with autism and three controls from phase II of the experimental procedure. Qualitative differences in the scanpaths of autistic and control participants were once more evident and the qualitative characteristics of
253 the scanpaths did not appear to change across the two phases of the experiment. As illustrated in the bottom panel of Figure 2, individuals in the autism group again spent a smaller percentage of time (M ⫽ 61.58%, SD ⫽ 19.42%) examining the core features of the faces than did participants in the control group (M ⫽ 83.22%, SD ⫽ 8.92%). This effect was once more particularly strong for the percentage of time spent examining the eyes (autism: M ⫽ 42.84%, SD ⫽ 18.52%; control: M ⫽ 56.28%, SD ⫽ 13.43%). Group differences were again evident but somewhat smaller for the percentage of time spent on the nose (autism: M ⫽ 10.76%, SD ⫽ 4.79%; control: M ⫽ 19.84%, SD ⫽ 5.88%) and the mouth (autism: M ⫽ 8.00%, SD ⫽ 5.80%; control: M ⫽ 7.14%, SD ⫽ 4.88%). As shown in the bottom panel of Table 1, autistic participants once more devoted a smaller percentage of their fixations to the core facial features (M ⫽ 65.06%, SE ⫽ 8.76%) during phase II than did control participants (M ⫽ 84.40%, SE ⫽ 3.23%). However, both groups appeared to devote fewer fixations to primary facial features during this phase of the experiment. During phase II, the largest group difference was observed for the percentage of fixations on the nose (autism: M ⫽ 12.36, SE ⫽ 2.83%; control: M ⫽ 20.42, SE ⫽ 3.86%). The average number of fixations per face (autism: M ⫽ 3.71, SE ⫽ 0.40; control: M ⫽ 4.01, SE ⫽ 0.24) and the average fixation duration (autism: M ⫽ 00.33 second, SE ⫽ 0.01 second; control: M ⫽ 00.33 seconds, SE ⫽ 0.01 second) did not appear to differ as a function of group membership. Parametric Analyses of Scanpath Data The observations of group differences for the percentage of time spent on core facial features were explored further in the context of a 2 (Group: autism vs. control) ⫻ 2 (Phase: I vs. II) mixed-design analysis of variance (ANOVA) procedure. The main effect of Group was significant, F(1, 8) ⫽ 19.37, p ⬍ .05. That is, across both phases of the experiment, autistic participants devoted a significant smaller percentage of time to the examination of core feature regions compared with control participants (autism: M ⫽ 58.87%, SE ⫽ 4.56%; control: M ⫽ 87.25, SE ⫽ 4.55%). The percentage of time spent examining core features did not differ as a function of procedural phase (phase I: M ⫽ 73.72%, SE ⫽ 2.17%; phase II: M ⫽ 72.40, SE ⫽ 4.78%), F(1, 8) ⫽ 0.13, p ⬎ .05, and the group ⫻ phase interaction was not significant, F(1, 8) ⫽ 3.35, p ⬎ .05. After collapsing across the two experimental phases, one-way ANOVAs were calculated to explore
254
Pelphrey et al.
Fig. 1. Sample scanpaths from phase I of the experiment for three autistic participants (first column) and three control participants (second column). Participants were instructed to examine the faces in any manner they selected.
Visual Scanning of Faces in Autism
255
Fig. 2. Distribution of visual scanning on and off core facial feature regions by experimental group and phase of investigation.
group differences at the level of the individual core facial feature (i.e., eyes, nose, mouth). These analyses are summarized in Table 2 and indicated that control participants spent a greater proportion of their scanning time examining the eyes (autism: M ⫽ 46.86%, SE ⫽ 5.28%; control: M ⫽ 67.66%, SE ⫽ 4.16%) and the nose (autism: M ⫽ 7.81%, SE ⫽ 2.14%; control: M ⫽
15.24%, SE ⫽ 1.56%) than did autistic participants. Differences in the percentage of time spent examining the mouth were not significant (autism: M ⫽ 4.22%, SE ⫽ 1.33%; control: M ⫽ 4.37%, SE ⫽ 1.17%) Observations of group differences for the percentage of fixations devoted to core facial features were explored further through a 2 (Group: autism vs. control) ⫻
256
Pelphrey et al. Table I. Descriptive Statistics for the Fixation Variables by Experimental Phase Variable Phase I No. of fixations per face Fixation duration (seconds) Fixations on features (%) Fixations off features (%) Fixations on eyes (%) Fixations on nose (%) Fixations on mouth (%) Phase II No. of fixations per face Fixation duration (seconds) Fixations on features (%) Fixations off features (%) Fixations on eyes (%) Fixations on nose (%) Fixations on mouth (%) a
Autism groupa
Control groupa
04.28 (00.27) 00.33 (00.03) 71 .98 (17.04) 28.02 (17.04) 65.56 (12.05) 06.00 (07.18) 00.36 (00.81)
04.60 00.35 92.62 07.38 81.64 08.84 02.16
(00.55) (00.03) (07.04) (07.04) (04.18) (06.78) (01.98)
03.71 00.33 65.06 34.94 47.16 12.36 07.54
04.01 00.33 84.40 15.60 58.16 20.42 05.84
(00.53) (00.02) (07.23) (07.23) (14.56) (08.64) (04.17)
(00.89) (00.03) (19.59) (19.59) (18.64) (06.32) (05.15)
Values given as M (SD) for N ⫽ 10.
2 (Phase: I vs. II) mixed ANOVA. The main effect of Group was significant, F(1, 8) ⫽ 5.53, p ⬍ .05. Autistic participants devoted a significantly smaller percentage of their fixations to the core feature regions compared with the nonautistic participants (autism: M ⫽ 68.52%, SE ⫽ 6.01%; control: M ⫽ 88.51%, SE ⫽ 6.01%). However, the main within-subjects effect of Phase was also significant, F(1, 8) ⫽ 10.79, p ⬍ .05. Both groups of participants significantly decreased the percentage of fixation devoted to core feature regions during the second phase of the experiment (Phase I: M ⫽ 82.30%, SE ⫽ 4.12%; Phase II: M ⫽ 74.73, SE ⫽ 4.67%). The Group ⫻ Phase interaction was not significant, F(1, 8) ⫽ 0.08, p ⬎ .05. A series of one-way ANOVAs were calculated to explore group differences in the percentages of fixations devoted to individual core facial feature (i.e., eyes, nose, mouth) for each of the two procedural phases. These analyses are summarized in Table 3, and indicated that control participants devoted a significantly larger percentage of their fixations to the eyes during Phase I (autism: M ⫽ 65.56, SE ⫽ 5.39%; control: M ⫽ 81.64, SE ⫽ 1.87%). Group differences were not significant when examined at the level of the individual core facial feature. Potential group and experimental phase differences in the average number and duration of fixations were explored through a 2 (Group: autism vs. control) ⫻ 2 (Phase: I vs. II) mixed-design ANOVA. The main effect of Group was not significant, F(1, 8) ⫽ 0.92, p ⬎ .05. Autistic and nonautistic participants made equivalent numbers of fixations across the two experimen-
tal phases (autism: M ⫽ 4.00, SE ⫽ 0.23; control: M ⫽ 4.30, SE ⫽ 0.23). However, the main effect of Phase was significant, F(1, 8) ⫽ 8.22, p ⬍ .05. Both groups significantly decreased the average number of fixations made per face during the second phase of the experiment (Phase I: M ⫽ 4.44, SE ⫽ 0.14; Phase II: M ⫽ 3.86, SE ⫽ 0.23). The Group ⫻ Phase interaction was not significant, F(1, 8) ⫽ 0.002, p ⬎ .05. Emotion Recognition Data One-way ANOVAs were used to examine group differences in the mean percentage of emotions correctly identified by participants. Table 4 provides a summary of these analyses as well as the descriptive statistics by group. Across the six different emotions, individuals in the autism group correctly identified a smaller percentage (M ⫽ 76.00%, SD ⫽ 11.53%) of the emotions portrayed in the stimulus photographs than did control participants (M ⫽ 92.60%, SD ⫽ 8.11). This effect was driven primary by a difference in the percentage of faces correctly identified as portraying fear (autism: M ⫽ 65.00%, SD ⫽ 22.36%; control: M ⫽ 95.00%, SD ⫽ 11.18%). When the autistic participation misidentified fear, they distributed their errors among anger (29%), surprise (42%), and disgust (29%). Statistically reliable differences were not observed for the other five emotions. However, there was a strong trend toward a difference for the identification of anger and smaller but consistent trends for the remaining four emotions. The autistic participants tended to confuse anger with fear (60% of their errors).
Visual Scanning of Faces in Autism
Fig. 3. Sample scanpaths from phase II of the experiment for three autistic participants (first column) and three control participants (second column). Participants were instructed to identify the emotion portrayed in each face.
257
258
Pelphrey et al.
Table II. Evaluating Group Differences on Individual Facial Features across the Two Experimental Phases a
Variable
Autism group
On eyes On nose On mouth
46.86 (11.80) 07.81 (04.78) 04.22 (02.97)
Control group
a
67.66 (09.30) 15.24 (03.49) 04.37 (02.62)
Table IV Evaluating Group Differences in Emotion Recognition Emotion
Autism groupa
Control groupa
Anger Disgust Fear Happy Sad Surprise
60.00 70.00 65.00 95.00 85.00 80.00
90.00 80.00 95.00 100.00 95.00 95.00
All emotions
76.00 (11.53)
F Statistic
F statistic 09.58b 07.89c 00.01
a
Values given as M (SD) percent of sample points (60 sample points ⫽ 1 second). N ⫽ 10. b p ⬍ .05. c p ⬍ .01.
(28.50) (20.92) (22.36) (11.15) (22.36) (27.39)
(13.69) (32.60) (11.18) (00.00) (09.21) (31.87)
04.50 00.33 07.20b 01.00 00.80 01.29
92.60 (08.11)
06.93b
Values given as M (SD) for N ⫽ 10, percent of emotions correctly named. b p ⬍ .05. a
DISCUSSION How do adult males with autism scan human faces? When their scanpaths were compared with the scanpaths of nonautistic participants, a number of differences were apparent. In comparison with control participants, autistic participants spent a greater proportion of their inspection time viewing nonfeature areas of the faces and spent a smaller percentage of time examining core features such as the nose, mouth, and, in particular, the eyes. Similarly, when the percentage of fixations devoted to core features of the face was taken as the index of performance, the results indicated that autistic participants devoted a smaller percentage of their fixations to primary facial features than did control participants. In general, the scanpaths of the participants with autism seemed erratic, undirected, and disorganized, often reflecting the processing of only one or two relatively unimportant features of the face (e.g., an ear, the chin, or region of the hair line). In contrast, the scanpaths of control participants seemed strategic and controlled, generally tracing a triangle that subtended the Table III. Evaluating Group Differences in the Percentage of Fixations Devoted to Individual Facial Features by Experimental Phase Variable Phase I On eyes On nose On mouth Phase II On eyes On nose On mouth a b
Autism groupa
Control groupa
F statistic
65.56 (12.05) 06.00 (07.18) 00.36 (00.81)
81.64 (04.18) 08.84 (06.78) 02.16 (01.98)
07.95b 00.41 03.54
47.16 (18.65) 12.36 (06.32) 07.54 (05.16)
58.16 (14.55) 20.42 (08.64) 05.84 (04.17)
01.08 02.84 00.33
Values given as M (SD) for N ⫽ 10. p ⬍ .05.
eyes, nose, and mouth. Across the two groups, scanpaths did not generally differ as a function of instructions given to the participants (i.e., “Please look at the faces in any manner you wish.” vs. “Please identify the emotions portrayed in these faces.”). However, the autistic and comparison groups both made significantly fewer fixations and devoted a significantly smaller percentage of their fixations to core feature regions during phase II of the experimental procedure. Notably, the average number of fixations per face and the average duration of fixations did not differ as a function of group membership or phase of the experiment. Differences in the accuracy of judgments concerning basic emotional expressions were observed. However, these differences were primarily confined to a deficit among autistic individuals in the identification of fear. The two groups performed at statistically equivalent levels when identifying the other five basic emotions, although there was a consistent trend towards lower levels of emotion recognition performance among the autistic participants. This pattern of findings from the emotion recognition data is consisted with studies that have failed to find deficits in affect recognition among groups of high-functioning autistic individual when the identification task involved very simple or basic emotions (e.g., Adolphs et al., 2001; Baron-Cohen et al., 1993, 1997; Volkmar et al., 1989). At least two explanations may be offered to account for the qualitative and quantitative differences in the scanpaths between the two groups, and these explanations are likely overlapping rather than mutually exclusive. First, the overall pattern of results suggests that autistic individuals examine faces differently than do normally developing individuals and that these differences may underlie deficits in face perception and facial affect recognition among autistic individuals. The present results are consistent with previous research suggesting that autistic individuals may rely more on
Visual Scanning of Faces in Autism individual parts of the face for identification rather than the overall configuration (e.g., Hobson et al., 1988; Langdell, 1978). This conclusion parallels and extends the broader literature on face perception in autism that suggests the presence of specific disturbances in face processing among individuals with autism spectrum disorders (e.g., Hobson et al., 1988; Langdell, 1978; Schultz et al., 2000). These findings could also be explained by the presence of a more general information processing disturbance affecting both face and object perception in autistic participants. For example, Frith (1989) has proposed that the pattern of uneven cognitive functioning (i.e., deficits in social information processing versus relatively intact spatial skills) sometimes seen in high-functioning autistic individuals is caused in part by “weak central coherence,” a style of cognitive processing that is biased towards local rather than global information processing. Happé (1996) compared the low-level visual integration processes of children with autism with those of typically developing children and children with moderate learning difficulties, in the context of response to visual illusions. Her findings were consistent with the central coherence account of autism, indicating that the children with autism were less likely to succumb to visual illusions. Nevertheless, it should be noted that the evidence for the weak central coherence account of autism remains equivocal and there are many outstanding questions with respect to this theory (e.g., Mottron, Belleville, Menard, 1999; Mottron Burack, Stauder, & Robaey, 1999; Mottron, Peretz, & Menard, 2000; Ropar & Mitchell, 1999). The findings from the present investigation cannot be used to distinguish between face specific and domain general explanations primarily because the scanpaths were not measured in response to non-face object stimuli. However, results from the broader literature on face perception in autism suggest that the information processing deficits are confined to faces and cannot be generalized to nonface objects (e.g., Boucher & Lewis, 1992; Davies et al., 1994; Hauck et al., 1998; Hobson et al., 1988; Langdell, 1978; Tantam et al., 1989). Future studies could be profitably directed toward disentangling face-specific and more general perceptual hypotheses by examining the scanpaths of autistic and nonautistic individuals in response to face and nonface object stimuli. It may also be of interest to consider the implications of the present findings in light of research concerning the broader autism phenotype (e.g., Briskman, Happé, & Frith, 2001; Folstein & Rutter, 1977; Happé, 1999; Briskman, & Frith, 2001; Piven & Palmer, 1997;
259 Piven et al., 1997; Piven, 2001). Collectively, this research has provided considerable evidence that parents and siblings (particularly male relatives) of children with autism demonstrate a higher incidence of autistic like personality and cognitive traits (e.g., communication deficits, aloofness, rigidity, shyness, anxiousness, and cognitive deficits in performance IQ and executive function) in comparison to various kinds of control families. Future research within the current eye tracking paradigm may contribute to efforts aimed at further characterizing the broader autism phenotype by providing an objective measure of social cognitive processing that can be administered to family members of autistic individuals. The present research, while intriguing, is limited in at least two respects. First, participants in the control and autism groups were not matched on IQ. Factors such as somewhat greater intelligence among members of the control group may have contributed to the effects observed here. Second, this study involved a relatively small sample of participants. The relative lack of statistical power to detect effects may have contributed to the failure to find significant group differences in the analyses of facial affect recognition performance. These limitations notwithstanding, the results from the present study advance our understanding of the autism phenotype and open several paths for future research. ACKNOWLEDGMENTS This research was supported in part by grants HD03110 and MH01568 to Joseph Piven and MH19111 to the department of Psychiatry at the University of North Carolina at Chapel Hill. We are grateful to the participants for their willingness to donate their time and effort to this research. We would also like to thank Gary B. Mesibov and the staff of the TEACCH program for their assistance in recruiting participants and Ralph Adolphs for his help in obtaining stimuli. Finally, we would like to thank the Frank Porter Graham Child Development Center for providing laboratory facilities for this project. REFERENCES Adolphs, R., Sears L., & Piven, J. (2001). Abnormal processing of social information from faces in autism. Journal of Cognitive Neuroscience, 3, (2), 232–240. American Psychiatric Association. (1994). Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, DC: American Psychiatric Association.
260 Baird, C., Charman, T., Baron-Cohen, S., Cox, A., Swettenham, J., Wheelwright, S., & Drew, A. (2000). A screening instrument for autism at 18 months of age: a 6-year follow-up study. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 694–702. Baron-Cohen, S., Cox, A., Baird, G., Swettenham, J., Nightingale, N., Morgan, K., Drew, A., & Charman, T. (1996). Psychological markers in the detection of autism in infancy in a large population. British Journal of Psychiatry, 168, 158–163. Baron-Cohen, S., Spitz, A., & Cross, P. (1993). Can children with autism recognize surprise? Cognition and Emotion, 7, 507–516. Baron-Cohen, S., Wheelwright, S., & Jolliffe, T. (1997). Is there are a “language of the eyes”? Evidence from normal adults and adults with autism or Asperger Syndrome. Visual Cognition, 4, 311–332. Bentin, S., Allison, T., Puce, A., Purez, E., & McCarthy, G. (1996). Electrophysiological studies of face perception in humans. Journal of Cognitive Neuroscience, 8, 551–565. Boucher, J., & Lewis, V. (1992). Unfamiliar face recognition in relatively able autistic children. Journal of Child Psychology and Psychiatry, 33, 843–859. Braverman, M., Fein, D., Lucci, D., & Waterhouse, L. (1989). Affect comprehension in children with pervasive developmental disorders. Journal of Autism and Developmental Disorders, 19, 301–315. Briskman, J., Happé, F., & Frith, U. (2001). Exploring the cognitive phenotype of autism: weak “central coherence” in parents and siblings of children with autism: II. Real-life skills and preferences. Journal of Child Psychology and Psychiatry, 42, (3), 309– 316. Capps, L., Yirmiya, N., & Sigman, M. D. (1992). Understanding of simple and complex emotions in non-retarded children with autism. Journal of Child Psychology and Psychiatry, 33, 1169–1182. Celani, G., Battacchi, M. W., & Arcidiacono, L. (1999). The understanding of emotional meaning of facial expressions in people with autism. Journal of Autism and Developmental Disorders, 29, 57– 66. Cicone, M., Wapner, W., & Gardner, H. (1980). Sensitivity to emotional expressions and situations in organic patients. Cortex, 16, 145–158. Cohen, D. J., & Volkmar, F. R. (1997). Handbook of Autism and Pervasive Developmental Disorders. 2nd Ed. New York, NY: John Wiley & Sons. Davies, S., Bishop, D., Manstead, A. S. R., & Tantam, D. (1994). Face perception in children with autism and Asperger’s syndrome. Journal of Child Psychology and Psychiatry, 35, 1033– 1057. De Gelder, B., Vroomen, J., & van der Heide, L. (1991). Face recognition and lip reading in autism. European Journal of Cognitive Psychology, 3, 69–86. De Renzi, E., Perani, D., Carlesimo, G. A., Silveri, M. C., & Fazio, F. (1994). Prosopagnosia can be associated with damage confined to the right hemisphere: an MRI and PET study and a review of the literature. Neuropsychologia, 32, 893–902. Diamond, R., & Carey, S. (1986). Why faces are and are not special: an effect of expertise. Journal of Experimental Psychology, 115, 107–117. Dolan R. J., Fink G. R., Rolls E., Booth M., Holmes, A., Frackowiak, R. S., & Friston, K. J. (1997). How the brain learns to see objects and faces in an impoverished context. Nature, 389, 596 –599. Ekman, P. (1989). The argument and evidence about universals in facial expressions of emotion. In H. Wagner and A. Manstead (Eds.), Handbook of social psychophysiology (pp. 143–164). Chichester: Wiley. Ekman, P., & Friesen, W. V. (1976). Pictures of facial affect. Palo Alto, CA: Consulting Psychologists Press. Ellis, H. D., & Shepard, J. W. (1975). Recognition of upright and inverted faces presented in the left and right visual fields. Cortex, 11, 3–7.
Pelphrey et al. Farah M. J., Wilson, K. D., Drain M., & Tanaka, J. N. (1998). What is special about face perception? Psychological Review, 105, 482– 498. Farah, M. J., (1990). Visual agnosia. Cambridge, MA: MIT Press. Folstein, S. E., & Rutter, M. L. (1977). Infantile autism: a genetic study of 21 twin pairs. Journal of Child Psychology and Psychiatry, 18, 297–321. Frith, U. (1989). Autism: Explaining the enigma. Maiden, MA: Blackwell Publishers. Gauthier, I., & Tarr, M. J. (1997). Becoming a “Greeble” expert: exploring mechanisms for face recognition. Vision Research, 37, 1673–1681. Goren, C. C., Sarty, M., & Wu, P. Y. (1975). Visual following and pattern discrimination of face-like stimuli by newborn infants. Pediatrics, 56, 544–549. Happé, F., Briskman, J., & Frith, U. (2001). Exploring the cognitive phenotype of autism: weak “central coherence” in parents and siblings of children with autism: I. Experimental tests. Journal of Child Psychology and Psychiatry, 42, (3), 299–307. Happé, F. (1999). Autism: Cognitive deficit or cognitive style? Trends in Cognitive Sciences, 3, (6), 216–222. Happé, F. (1996). Studying weak central coherence at low levels: children with autism do not succumb to visual illusions. A research note. Journal of Child Psychology and Psychiatry and Allied Disciplines, 37, (7), 873–877. Hauck, M., Fein, D., Maltby, N., Waterhouse, L., & Feinstein, C. (1998). Memory for faces in children with autism. Child Neuropsychology, 4, 187–198. Haxby, J. V., Ungerleider, L. G., Clark, V. P. Schouten, J. L., Hoffman, E. A., & Martin, A. (1999). The effect of face inversion on activity in human neural systems for face and object perception. Neuron, 22, 189–199. Hobson, R. P. (1986a). The autistic child’s appraisal of expressions of emotion. Journal of Child Psychology and Psychiatry, 27, 321–342. Hobson, R. P. (1986b). The autistic child’s appraisal of expressions of emotion: a further study. Journal of Child Psychology and Psychiatry, 27, 671– 680. Hobson, R. P., Ouston, J., & Lee, A. (1988). Emotion recognition in autism: coordinating faces and voices. Psychological Medicine, 18, 911–923. Hobson, R. P., Ouston, J., & Lee, A. (1989). Naming emotion in faces and voices: abilities and disabilities in autism and mental retardation. British Journal of Developmental Psychology, 7, 237–250. Johnson, M. H., & Morton, J. (1991). Biology and cognitive development: The case of face recognition. Oxford: Basil Blackwell. Johnson, M. S., Dziurawic, H., Ellis, M., & Morton, J. (1991). Newborns’ preferential tracking of face-like stimuli and its subsequent decline. Cognition, 40, 1–19. Kanner, L. (1943). Autistic disturbances of affective contact. Nervous Child, 2, 217–250. Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17, 4302– 4311. Klin, A., Sparrow, S., de Bildt, A., Cicchetti, D., Cohen, D., & Volkmar, F. (1999). A normed study of face recognition in autism and related disorders. Journal of Autism and Developmental Disorders, 29, 499–508. Langdell, T. (1978). Recognition of faces: an approach to the study of autism. Journal of Child Psychology and Psychiatry, 19, 255– 268. Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 659– 685. Lord, C., Rutter, M., DiLavore, P., & Risi, S. (1999). Autism Diagnostic Observation Schedule (ADOS). Los Angeles, CA: Western Psychological Services.
Visual Scanning of Faces in Autism Loveland, K., Tunali-Kotoski, B., Chen Y., Ortegon J., Pearson, D., Brelsford, K., & Gibbs, M. (1997). Emotion recognition in autism: verbal and nonverbal information. Development and Psychopathology, 9, 579–593. Luria, S. M., & Strauss, M. S. (1978). Comparison of eye movements over faces in photographic positives and negatives. Perception and Psychophysics, 2, 547–552. Mackworth, N. H., & Bruner, J. S. (1970). How adults and children search and recognize pictures. Human Development, 13, 149– 177. Manor, B. R., Gordon, E., Williams, L. M., Rennie, C. J., Bahramali, H., Latimer, C. R., Barry, R. J., & Meares, R. A. (1999). Eye movements reflect impaired face processing in patients with schizophrenia. Biological Psychiatry, 46, 963–969. Mertens, I., Siegmund, H., & Grusser, O. (1993). Gaze motor asymmetries in the perception of faces during a memory task. Neuropsychologia, 31, 989–998. Morton, J., & Johnson, M. H. (1991). CONSPEC and CONLERN: a two-process theory of infant face recognition. Psychological Review, 98, 164–181. Mottron, L., Belleville, S., & Menard, E. (1999). Local bias in autistic subjects as evidenced by graphic tasks: perceptual hierarchization or working memory deficit? Journal of Child Psychology and Psychiatry and Allied Disciplines, 40, (5), 743– 755. Mottron L., Burack, J. A., Stauder, J. E. A., & Robaey, P. (1999). Perceptual processing among high-functioning persons with autism. Journal of Child Psychology and Psychiatry and Allied Disciplines Volume 40, (2), 203–211. Mottron, L., Peretz, I., & Menard, E. (2000). Local and global processing of music in high-functioning persons with autism: beyond central coherence? Journal of Child Psychology and Psychiatry and Allied Disciplines, 41, (8), 1057–1065. Noton, D., & Stark, L. (1971). Eye movements and visual perception. Scientific American, 224, 35– 43. Osterling, J., & Dawson, G. (1994). Early recognition of children with autism: a study of first birthday home videotapes. Journal of Autism and Developmental Disorders, 24, 247–257. Ozonoff, S., Pennington, B. F., & Rogers, S. J. (1990). Are there emotional perception deficits in young autistic children? Journal of Child Psychology and Psychiatry, 31, 343–361. Phillips, M. L., & David, A. S. (1997). Visual scanpaths are abnormal in deluded schizophrenia. Neuropsychologia, 35, 99– 105. Piven, J., & Palmer, P. (1997). Cognitive deficits in parents from
261 multiple-incidence autism families. Journal of Child Psychology and Psychiatry, 38, 1011–1021. Piven, J., Palmer, P., Landa, R., Santangelo, S., Jacobi, D., & Childress, D. (1997). Personality and language characteristics in parents from multiple-incidence autism families. American Journal of Medical Genetics (Neuropsychiatric Genetics), 74, 398–411. Piven, J. (2001). The broad autism phenotype: a complementary strategy for molecular genetic studies of autism. American Journal of Medical Genetics (Neuropsychiatric Genetics), 106, 34–35. Ropar, D., & Mitchell, P. (1999). Are individuals with autism and Asperger’s syndrome susceptible to visual illusions? Journal of Child Psychology and Psychiatry and Allied Disciplines 40, (8), 1283–1293. Sams, M., Hietanen, J. K., Hari, R., Ilmoniemi, J. H., & Lounasmaa, O. V. (1997). Face-specific response from the human inferior occipito-temporal cortex. Neuroscience, 77, 49–55. Schultz, R. T., Gauthier, I., Klin, A., Fulbright, R., Anderson, A., Volkmar, F., Skudlarski, P., Lacadie, C., Cohen, D. J., & Gore, J. C. (2000). Abnormal ventral temporal cortical activity during face discrimination among individuals with autism and Asperger syndrome. Archives of General Psychiatry, 57, (4) 1–23. Sergent, J., Ohta, S., & MacDonald, B. (1992). Functional neuroanatomy of face and object processing: a positron emission tomography study. Brain, 115, 15–36. Streit, M., Wölwer, W., & Gabel, W. (1996). Facial-affect recognition and visual scanning behavior in the course of schizophrenia. Schizophrenia Research, 24, 311–317. Tantam, D., Monaghan, L., Nicholson, H., & Stirling, J. (1989). Autistic children’s ability to interpret faces: a research note. Journal of Child Psychology and Psychiatry, 30, 623 – 630. Tarr, M. J., & Bulthoff, H. H. (1998). Image-based object recognition in man, monkey, and machine. Cognition, 67, 1–20. Valentine, T. (1988). Upside-down faces: a review of the effect of inversion upon face recognition. British Journal of Psychology, 79, 471– 491. Volkmar, F. R., Sparrow, S. S., Rende, R. C., & Cohen, D. J. (1989). Facial perception in autism. Journal of Psychological Psychiatry, 30, 591–598. Walker-Smith, G. J., Gale, A. G., & Findlay, J. M. (1977). Eye movement strategies in face perception. Perception, 6, 313–326. Wechsler, D. (1981). Wechsler Adult Intelligence Scale-Revised. New York: Harcourt Brace Jovanovich. Yarbus, A. L. (1967). Eye movements and vision. New York: Plenum Press.
