Music in the Lives of Deaf Children with Cochlear Implants

THE NEUROSCIENCES AND MUSIC III—DISORDERS AND PLASTICITY

Music in the Lives of Deaf Children with Cochlear Implants Sandra E. Trehub,a Tara Vongpaisal,a and Takayuki Nakatab a b

University of Toronto, Mississauga, Ontario, Canada

Future University–Hakodate, Hakodate, Hokkaido, Japan

Present-day cochlear implants provide good temporal cues and coarse spectral cues. In general, these cues are adequate for perceiving speech in quiet backgrounds and for young children’s acquisition of spoken language. They are inadequate, however, for conveying the rich pitch-patterning of music. As a result, many adults who become implant users after losing their hearing find music disappointing or unacceptable. By contrast, child implant users who were born deaf or became deaf as infants or toddlers typically find music interesting and enjoyable. They recognize popular songs that they hear regularly when the test materials match critical features of the original versions. For example, they can identify familiar songs from the original recordings with words and from versions that omit the words but preserve all other cues. They also recognize theme songs from their favorite television programs when presented in original or somewhat altered form. The motivation of children with implants for listening to music or melodious speech is evident well before they understand language. Within months after receiving their implant, they prefer singing to silence. They also prefer speech in the maternal style to typical adult speech and the sounds of their native language-to-be to those of a foreign language. An important task of future research is to ascertain the relative contributions of perceptual and motivational factors to the apparent differences between child and adult implant users. Key words: music perception; singing; children; deafness; cochlear implants

Introduction: Challenges of Implants Many congenitally deaf children achieve high levels of proficiency in speech perception and production on the basis of electrical input from their cochlear implants.1 Adults who become profoundly deaf escape the challenges of language acquisition, but they still face daunting communication challenges, which are often overcome through the use of cochlear implants. Although typical cochlear prostheses optimize temporal-envelope cues, which are critical for speech perception, they provide limited cues to temporal fine structure, which are critical for precise pitch resolution.2 Address for correspondence: Sandra E. Trehub, Department of Psychology, University of Toronto, 3359 Mississauga Road North, Mississauga, Ontario, Canada L5L 1C6. Voice: +1-905-828-5415; fax: +1-905-569-4850. [email protected]

Limitations in pitch resolution have enormous implications for the perception of music.3 Adult implant users commonly require differences of 4–12 semitones before they can identify one tone as higher or lower in pitch than another.4–6 Such resolution is obviously inadequate for the appreciation of melodies whose successive notes are often separated by 1 or 2 semitones.7 Some sense of the challenges faced by implant users can be gained by listening to simulations of music via cochlear implants.8 Although poor pitch resolution is the principal impediment to music processing in implant users, there are other difficulties. Degraded cues to spectral shape also interfere with timbre perception, specifically, the differentiation of instrumental timbres from one another, including those from different instrument families.9,10 It is not surprising, then, that many adult implant users find music disappointing

The Neurosciences and Music III—Disorders and Plasticity: Ann. N.Y. Acad. Sci. 1169: 534–542 (2009). c 2009 New York Academy of Sciences. doi: 10.1111/j.1749-6632.2009.04554.x

534

Trehub et al.: Children with Cochlear Implants

or unacceptable in the postimplant period.11,12 For a limited number of adult implant users with extensive musical training, heroic efforts coupled with technical and personal accommodations have enabled them to return to careers as performing musicians. In general, however, adult implant users fail to recognize familiar melodies13–15 or to differentiate unfamiliar melodies from one another on the basis of pitch cues alone.3 Child Implant Users Like Music The situation for child implant users, especially those who are congenitally or prelingually deaf, differs substantially from that of adult implant users. Although children must contend with similar device limitations, they do not evaluate music with reference to acoustic standards or the way music used to sound. For the most part, these young implant users find music highly engaging, and they participate in varied musical activities, such as singing, dancing, concert-going, and instrument lessons.16–18 In principle, such favorable attitudes and greater musical involvement could result in better music perception skills than those of adult implant users. In practice, however, children with normal or impaired hearing typically make poorer use of available auditory cues than their older counterparts.19,20 It remains unclear, then, whether children with implants achieve the potential perceptual benefits from their musical engagement. Unfortunately, there have been no direct comparisons of the music perception skills of child and adult implant users, so answers to this question must await further research. Nevertheless, studies of implanted children are shedding light on their music recognition and production skills. Child Implant Users Identify Familiar Songs In one investigation, 8- to 15-year-old implant users had difficulty recognizing simple in-

535

strumental versions of well-known songs such as Happy Birthday, and Row, Row, Row Your Boat.21 In that respect, they seemed much like adult implant users.13–15 For hearing listeners, who have abstract as well as detailed mental representations of familiar music, 22–24 there are multiple routes to identification. For example, most hearing listeners would recognize Happy Birthday in different pitch registers (e.g., high or low) and at different tempi (e.g., fast or slow) when performed with or without variations by solo singers, unaccompanied choral singers, solo instrumentalists, or full orchestras. By contrast, child implant users are likely to have different mental representations that depend less, if at all, on relative pitch cues. If their representations are less abstract or less detailed than those of hearing listeners, they could have difficulty linking instrumental renditions of familiar songs to the sung renditions that they usually hear. Considerations such as these led us to explore the possibility that music recognition in young implant users required all of the features usually heard or a specific subset of those features. Accordingly, we focused on implanted children and adolescents who were fans of popular music and listened regularly to recordings by their favorite vocalists or groups.18 Our plan was to test these implant users and a control sample of age-matched hearing listeners on various renditions of the specific recordings most familiar to them: the original version, which included a single vocalist or group with instrumental accompaniment, an instrumental version that matched the original except for the absence of vocals, and a synthesized piano version of the main (sung) melody that preserved the original pitch, tempo, and rhythm. For each version, listeners were required to identify the target song from choices depicted on a monitor (i.e., song titles and photos of artists or groups). Each listener selected 3–5 songs that they knew best from a set of 14 songs that were highly popular at that time with children and adolescents. First, we established that 5- to 8-year-olds and adults with normal hearing could identify the songs from the aforementioned versions

536

Figure 1. Mean accuracy scores for hearing children and adults on original, instrumental, and melody versions of popular songs. Error bars represent standard errors. (From Vongpaisal et al .18 Reprinted with permission.)

(Fig. 1). Hearing children and adults performed well above chance levels (0 translating to chance performance and 1 to perfect performance), but the performance of both groups declined as the difference increased between the original stimuli and the test stimuli. Children performed more poorly than adults only on the melody version. This finding may reflect children’s inability to use retrieval cues as flexibly as adults do. Alternatively, their mental representations of music may be less robust than those of adults. Subsequently, we found that young implant users 8–18 years of age (mean age of 12 years) performed more poorly than agematched hearing listeners on all versions of the songs (Fig. 2). As with the hearing children, young implant users’ performance was best for test stimuli that matched all features available at the time of encoding (i.e., during recreational listening) and worst for those with the greatest divergence. Specifically, implant users’ performance was well above chance levels on the original versions, less accurate but still above chance levels on the instrumental versions, and at chance levels on the melody versions. Although implanted children with excellent speech perception skills have difficulty decoding the lyrics of unfamiliar songs,25 the lyrics of the famil-

Annals of the New York Academy of Sciences

Figure 2. Mean accuracy scores of hearing (NH) and implanted (CI) children on original, instrumental, and melody versions of popular songs. Error bars represent standard errors.(From Vongpaisal et al .18 Reprinted with permission.)

iar songs obviously contributed to their performance in this study. Child and adolescent implant users’ success on the instrumental versions confirmed that verbal cues were not critical for song identification. This finding implies that despite implant users’ difficulty with timbre (instrument) identification,9,10 timbre cues still contribute to music recognition. Recall that the instrumental renditions matched the original recordings in all respects except for the absence of vocals, so they provided a rich array of timbre and timing cues. By contrast, the melody renditions, which deviated substantially from the original music, were insufficient for song identification even though they retained the pitch and timing cues of the original sung melody. As can be seen in Figure 3, individual differences in the performance of implant users were substantial. Child Implant Users Identify TV Theme Songs If child implant users are able to recognize music that they hear by choice, might they also recognize music that they hear incidentally while watching TV? Children’s TV programs typically feature theme songs that become well


Figure 3. Accuracy scores of individual agematched hearing (NH) and implanted (CI) children and adolescents on the original, instrumental, and melody versions of popular songs.(From Vongpaisal et al .18 Reprinted with permission.)

known to young hearing viewers, but it is unclear whether such songs would be engaging and memorable for young implant users. We examined the ability of Japanese child implant users to recognize such music.26,27 North American rehabilitation practices are relatively similar to those in Japan, with the principal focus on speech and language and little, if any, attention to music. The Japanese children, who were 4–8 years of age, were tested on various renditions of the theme music, including the original version, an instrumental version that preserved all cues except for the vocals, and a synthesized flute version of the main melody.

537

After listening to a musical excerpt, children selected the actors or cartoon characters from the target TV show from alternatives depicted on a monitor. Japanese child implant users reliably identified the original versions, but they showed no recognition of the other versions. Perhaps incidental exposure to the TV theme songs was insufficient for generating robust representations of the music that could generalize to other versions. Alternatively, these children could have had less overall exposure to the music than did the Canadian pop song fans in the previous study. In Japan, the theme songs of children’s TV programs typically change at 3- to 12-month intervals, in contrast to the considerably longer availability of specific popular songs or TV theme songs in North America. When Canadian 4- to 11-year-olds with cochlear implants were given a comparable identification task with TV theme songs, their performance was above chance levels on all versions,28 lending credence to the potential contribution of exposure. Although the Canadian children were older, on average, than the Japanese children, age is unlikely to underlie the observed differences in performance. In fact, the two youngest children in the Canadian sample—one 4-year-old and one 5-yearold—were the best performers. More significantly, perhaps, these children received their implant at 12 months of age, a year or more earlier than other children in the Canadian or Japanese samples. Implantation of congenitally deaf children before 2 years of age is known to facilitate speech perception and language development.29,30 Early implantation may result in comparable processing advantages for music. The most unexpected finding was child implant users’ successful identification of the melody versions of the TV theme songs. Not only did these Canadian children outperform the Japanese TV viewers, but they also outperformed the implanted children and teens who failed to identify the melody versions of familiar pop songs.18 One potentially relevant

538


difference between the Canadian pop song fans and TV viewers is the younger age of implantation of the latter group. Another possible difference is children’s greater cumulative exposure to the TV theme music than to the pop songs. No information on exposure was available, but self-reports or parental reports of cumulative exposure to music listening or TV viewing are unlikely to be reliable in any case. It is important to ascertain the consequences of musical exposure in auditory-alone or auditory-visual contexts by means of comparable stimuli and exposure time across contexts as well as participants of similar age and implant experience. Finally, it is possible that timbre differences between the pop song melodies (piano) and TV melodies (flute) contributed to the differences in performance. According to some reports, piano tones pose particular difficulty for implant users.3,4 Child Implant Users Sing The favorable music evaluations of child implant users16,18,26 could be driven largely by social conventions. Similarly, their varied musical activities could arise from parental pressure. Such external factors, however potent, are unlikely to account for the high incidence of spontaneous singing in this population,26 which attests to their interest in music. Obviously, it is of considerable interest to ascertain the nature of implant users’ singing in view of their limitations in pitch processing3,4,6 and their relative strength in rhythm processing.14 During a routine clinic visit, we asked Japanese children to sing their favorite songs, and they obliged without hesitation. We measured the mean fundamental frequency and duration (inter-onset intervals) for each note sung by these children and a comparison group of hearing children.17 For each child, we compared observed durations to expected durations, basing the latter on notated versions of the song adjusted to the child’s tempo. We also compared observed pitches to target pitches.

The pitch and timing characteristics of song excerpts (first 15 beats) from a subsample of children with cochlear implants and those with normal hearing are depicted schematically in Figure 4. Although the observed timing of songs deviated somewhat from the expected or ideal timing, those deviations were no greater for implanted children than for hearing children. In any case, both groups provided clearly recognizable reproductions of the original timing. By contrast, implanted and hearing children’s songs were dramatically different in pitch patterning. As can be seen in Figure 4, implanted children used a very limited pitch range, roughly a third of the range of hearing children. Whereas hearing children’s pitch directional changes—up or down—matched those of the target song, those of implanted children were unrelated to the target song. Despite their substantial differences in pitch patterning, the sung renditions of child implant users and hearing children revealed comparable energy, vitality, and joy. Listening Preferences of Toddlers with Implants It is clear that deaf children with cochlear implants enjoy music, remember aspects of the music they hear regularly, and, in many cases, sing for their own pleasure. In that sense, their motivation for music is much like that of hearing children. For hearing children, however, interest in music is evident from the earliest days and months of life.31 For example, preverbal infants are tantalized by mothers’ melodious speech, preferring it to adult-directed speech, which is less variable in pitch and dynamics and less affect-laden.32,33 They also prefer the maternal style of singing, which is highly expressive, to typical informal singing, which is much less expressive.34,35 Although maternal speech and singing are both highly engaging to infants, maternal singing sustains their attention for considerably longer periods.36

539


Figure 4. Sample illustrations of expected (open symbols) and observed (solid symbols) pitch and duration (inter-onset intervals) for the first 15 beats of familiar songs sung by children with cochlear implants (CI) and hearing children (NH). (From Nakata et al .17 Reprinted with permission.)

Congenitally deaf children, who typically receive their implants at 12 months of age or later, have not had comparable exposure to maternal speech, singing, and environmental sounds from birth. Their exposure begins about a month after implantation, when their implants are activated. We are currently attempting to assess the listening preferences of implanted toddlers by means of a headturn preference procedure (Volkova et al., manuscript in preparation). Toddlers are pre-

sented with one auditory pattern when they look at one display (e.g., to their left) and a contrasting auditory pattern when they look at another display (e.g., to their right). Over a series of trials, cumulative looking times associated with each of the auditory patterns provide an estimate of their relative interest in those patterns. Preliminary evidence reveals that within a few months of implant activation, toddlers prefer an audiovisual display of a mother (not their own) singing a play song to a soundless

540


version of the same visual display. Well before they show evidence of understanding any words, these toddlers prefer motherese (i.e., speech in the maternal style) to typical adultdirected speech, presumably on the basis of its prosody or music. They also prefer motherese in their language-to-be to motherese in a foreign language, which presumably reflects their familiarity with the speech sounds and prosody in their everyday environment. Preschool Implant Users Identify TV Theme Songs We have begun testing preschool implant users, from 2.5 or 3 years of age, on their recognition of TV theme music from programs watched regularly (Volkova et al., manuscript in preparation). Of the limited number of preschoolers tested to date, most have been successful in identifying original and instrumental versions of these songs. Some have even succeeded in identifying the melody versions. It is as yet unclear whether their uniformly early age of implantation is responsible for their high level of engagement with music and their success on various music recognition tasks. Increasingly, very young deaf children are receiving bilateral implants rather than the unilateral implants that were prevalent for many years. It is too early to know whether bilateral implantation will enhance music perception. Future Directions The available data seem to suggest that music is more engaging and memorable for children with cochlear implants than it is for adult implant users. At this point, the relative contributions of motivational and perceptual factors to these differences remain unclear. Specifying such contributions is an important task for future research. Although limitations of current implant technology interfere with fine pitch res-

olution and musical pitch processing,2,3 these limitations do not preclude rewarding musical experiences, especially for children. We know much more about children’s music perception skills than we did 5 years ago, but many questions remain unanswered. What specific features contribute to child implant users’ engagement with music? Do these children rely primarily, even exclusively, on timing cues? For children with implants who can identify simple melodic renditions of TV theme songs, are they doing so solely on the basis of relative timing patterns rather than the relative pitch patterns that are central for hearing listeners? Can they capitalize on gross spectral cues? Answers to such questions should be forthcoming over the next few years. Acknowledgments

Preparation of this paper was assisted by grants from the Natural Sciences and Engineering Research Council and the Social Sciences and Humanities Research Council of Canada, and from the Japan Society for the Production of Science. Conflicts of Interest

The authors declare no conflicts of interest. References 1. Svirsky, M.A., A.M. Robbins, K.I. Kirk, et al. 2000. Language development in profoundly deaf children with cochlear implants. Psychol. Sci. 11: 153– 158. 2. Shannon, R.V., Q.-J. Fu, J. Galvin, et al. 2004. Speech perception with cochlear implants. In Cochlear Implants: Auditory Prostheses and Electric Hearing, Springer Handbook of Auditory Research. F.-G. Zeng, A.N. Popper & R.R. Fay, Eds.: 334–376. Springer-Verlag. New York. 3. Cooper, W.B., E. Tobey & P.C. Loizou. 2008. Music perception by cochlear implant and normal hearing listeners as measured by the Montreal Battery for Evaluation of Amusia. Ear Hear. 29: 618– 626.

Trehub et al.: Children with Cochlear Implants 4. Fujita, S. & J. Ito. 1999. Ability of nucleus cochlear implantees to recognize music. Ann. Otol. Rhinol. Laryngol. 108: 634–640. 5. Green, T., A. Faulkner & S. Rosen. 2002. Spectral and temporal cues to pitch in noise-excited vocoder simulations of continuous-interleavedsampling cochlear implants. J. Acoust. Soc. Am. 112: 2155–2164. 6. Vandali, A.E., C. Sucher, D.J. Tsang, et al. 2005. Pitch ranking ability of cochlear implant recipients: a comparison of sound-processing strategies. J. Acoust. Soc. Am. 117: 3126–3138. 7. Vos, P. & J. Troost. 1989. Ascending and descending melodic intervals: statistical findings and their perceptual relevance. Music Percep. 6: 383– 396. 8. http://www.healthaffairs.uci.edu/hesp/Simulations/ simulationsmain.htm 9. Gfeller, K., S. Witt, G. Woodworth, et al. 2002. Effects of frequency, instrumental family, and cochlear implant type on timbre recognition and appraisal. Ann. Otol. Rhinol. Laryngol. 111: 349– 356. 10. McDermott, H.J. 2004. Music perception with cochlear implants: a review. Trends. Amplif. 8: 49– 82. 11. Gfeller, K., A. Christ, J.F. Knutson, et al. 2000. Musical backgrounds, listening habits, and aesthetic enjoyment of adult cochlear implant recipients. J. Am. Acad. Audiol. 11: 390–406. 12. Lassaletta, L., A. Castro, M. Bastarrica, et al. 2007. Does music perception have an impact on quality of life following cochlear implantation? Acta. Otolaryngol. 127: 682–686. 13. Galvin, J.J., Q.-J. Fu & G. Nogaki. 2007. Melodic contour identification by cochlear implant listeners. Ear Hear. 28: 302–319. 14. Gfeller, K., G. Woodworth, D.A. Robin, et al. 1997. Perception of rhythmic and sequential pitch patterns by normally hearing adults and cochlear implant users. Ear Hear. 18: 252–260. 15. Kong, Y.-Y., R. Cruz, J.A. Jones, et al. 2004. Music perception with temporal cues in acoustic and electric hearing. Ear Hear. 25: 173–185. 16. Gfeller, K., S.A. Witt, L.J. Spencer, et al. 1998. Musical involvement and enjoyment of children who use cochlear implants. Volta Rev. 100: 213– 233. 17. Nakata, T., S.E. Trehub, C. Mitani, et al. 2006. Pitch and timing in the songs of deaf children with cochlear implants. Music Percept. 24: 147–154. 18. Vongpaisal, T., S.E. Trehub & E.G. Schellenberg. 2006. Song recognition by children and adolescents with cochlear implants. J. Speech Lang. Hear. Res. 49: 1091–1103.

541 19. Eisenberg, L.S., R.V. Shannon, A.S. Martinez, et al. 2000. Speech recognition with reduced spectral cues as a function of age. J. Acoust. Soc. Am. 107: 2704– 2710. 20. Stalinski, S.M., E.G. Schellenberg & S.E. Trehub. 2008. Developmental changes in the perception of pitch contour: distinguishing up from down. J. Acoust. Soc. Am. 124: 1759–1763. 21. Stordahl, J. 2002. Song recognition and appraisal: a comparison of children who use cochlear implants and normally hearing children. J. Music Ther. 39: 2– 19. 22. Dowling, W.J. 1999. The development of music perception and cognition. In The Psychology of Music, 2nd ed. D. Deutsch, Ed.: 603–625. Academic Press. San Diego, CA. 23. Schellenberg, E.G., P. Iverson & M.C. McKinnon. 1999. Name that tune: identifying popular recordings from brief excerpts. Psychon. Bull. Rev. 6: 641– 646. 24. Schellenberg, E.G. & S.E. Trehub. 2003. Accurate pitch memory is widespread. Psychol. Sci. 14: 262– 266. 25. Vongpaisal, T., S.E. Trehub & E.G. Schellenberg. 2005. Challenges of decoding the words of songs. Presented at the 10th Symposium on Cochlear Implants in Children. Dallas, TX, March 17. 26. Nakata, T., S.E. Trehub, C. Mitani, et al. 2005. Music recognition by Japanese children with cochlear implants. J. Physiol. Anthropol. Appl. Human Sci. 24: 29–32. 27. Mitani, C., T. Nakata, T., S.E. Trehub, et al. 2007. Music recognition, music listening, and word recognition by deaf children with cochlear implants. Ear Hear 28: 29S–33S. 28. Vongpaisal, T., S.E. Trehub & E.G. Schellenberg. 2009. Identification of TV tunes by children with cochlear implants. Music Percept. In press. 29. Nicholas, J.G. & A.E. Geers. 2006. Effects of early auditory experience on the spoken language of deaf children at 3 years of age. Ear Hear. 27: 286– 298. 30. Svirsky, M.A., S.-W. Teoh & H. Neuburger. 2004. Development of language and speech perception in congenitally, profoundly deaf children as a function of age at cochlear implantation. Audiol. Neurotol. 9: 224–233. 31. Trehub, S.E. 2008. Music lessons from infants. In The Oxford Handbook of Music Psychology. S. Hallam, I. Cross & M. Thaut, Eds.: 229–234. Oxford University Press. Oxford, UK. 32. Fernald, A. 1991. Prosody in speech to children: prelinguistic and linguistic functions. In Annals of Child Development, Vol. 8. R. Vasta, Ed.: 43–80. Jessica Kingsley Publishers. London, UK.

542 33. Singh, L., J.L. Morgan & C.T. Best. 2002. Infants’ listening preferences: baby talk or happy talk? Infancy 3: 365–394. 34. Trainor, L.J. 1996. Infant preferences for infantdirected versus noninfant-directed playsongs and lullabies. Infant. Behav. Dev. 19: 83–92.

Annals of the New York Academy of Sciences 35. Trainor, L.J., E.D. Clark, A. Huntley, et al. 1997. The acoustic basis of preferences for infant-directed singing. Infant Behav. Dev. 20: 383–396. 36. Nakata, T. & S.E. Trehub. 2004. Infants’ responsiveness to maternal speech and singing. Infant Behav. Dev. 27: 455–464.