THE EFFECTS OF INDUCED EMOTIONAL STATES ON EMOTIONAL

Scientiae Mathematicae Japonicae, 76, No. 1 (2013), 79–86 :e-2013, 79–86

79

THE EFFECTS OF INDUCED EMOTIONAL STATES ON EMOTIONAL EXPERIENCE: A RESEARCH AGENDA Vincenzo Capuano, Maria Teresa Riviello, Gennaro Cordasco and Anna Esposito Received October 15, 2012

Abstract. This work presents a research agenda defining a set of perceptual experiments and behavioral analyses devoted to investigate the effects of induced emotional states and related emotional regulation strategies on word memory recognition performance and related time assessments by the involved participants. The general idea is to define a protocol for data collection that would allow to concurrently account of the four variables at the play: emotion, emotion regulation, memory performance, and time perception. In addition, the recording of the subjects facial expressions during the induction procedure will contribute to the creation of a multi-modal database of quasi-spontaneous facial emotional expressions.

1 Introduction (statement of topic and problem) The psychodynamic interest on human defence mechanisms has contributed to the growing of studies on emotion regulation strategies understood here as the processes through which individuals consciously or unconsciously control their emotional feelings [2, 35] in order to appropriately regulate the magnitude of their emotional experience to environmental demands [2, 5, 35, 27]. According to Gross [15] individuals exert considerable control (either in a conscious or unconscious way) over their emotions, using a wide range of regulation strategies to psychologically deal with and express the emotions they experience. In particular Gross identified two categories of emotion coping strategies: reappraisal (changing the ongoing emotional impact) and suppression (inhibiting the ongoing emotion behavior). Furthermore he defined an instrument to assess both the strategies and characterize individual differences and long-term consequences in everyday life [15]. A great amount of research on emotions is devoted to regulation strategies [17, 21, 26, 27, 39]. Today we know that the different emotion regulation strategies are linked with specific emotional, physiological, cognitive and behavioral responses [16]. Neuroimaging studies identified brain regions engaged in the cognitive control of emotions and highlighted the role of brain areas, such as the lateral prefrontal cortex (PFC) and the anterior cingulate cortex (ACC) region, and their modulatory effects on limbic regions (e.g., amygdala). Such effects are observed both during and just after the regulation phase [6], and regulate individual emotional behaviors in a determinate context [23, 27]. Recent works [29, 30, 36] defined the “circumplex model of affect” to describe all the emotions as a linear combination of two neuropsychological dimensions: valence (negative, positive and neutral) and arousal (i.e., bodily activation) on a specific cognitive performance. Most of the studies that investigate on emotion regulation have been devoted to observe the effects of emotional states in terms of valence only [13, 14, 18, 23, 28, 31, 32]. The proposed research agenda intends to extend these investigations to a larger number of emotional states – happy, anger, fear, disgust, sadness and surprise – accounting 2000 Mathematics Subject Classification. Key words and phrases. Emotion, emotion regulation, memory, time perception, facial expression.

80

V. CAPUANO, M. T. RIVIELLO G. CORDASCO AND A. ESPOSITO

both valence and arousal, and assessing the subject’s performance on different cognitive tasks. Indeed, each of emotional state listed above, also known as “basic emotions” [9], are characterized by both a valence and an arousal intensity. 1.1 Time perception The subjective duration of events is modulated by the existence of an internal clock system that provides the perception of time [41, 11]. The ability of emotions to speed up or slow down this internal clock has been supported by several investigations. Gibbon [11] defined the temporal information–processing model (see Figure 1 (left)). It consists in three different stages (clock, memory and decision) and requires an internal clock containing a pacemaker and an accumulator, linked by a switch: in the clock stage the pacemaker emits pulses, the accumulator collects them until the switch is in a closed state; then, in the memory stage, pulses in the accumulator are stored in the memory and are compared to previously stored durations. The model has been used to show that the perception of time is influenced by the presentation of stimuli.

Figure 1: (left) The temporal information-processing model [11]. A representation of the internal clock. (right) Illustration of the model integration [38]. Schirmer [38] describes three different cognitive mechanisms underlying the emotional effects on time perception: arousal, attention and sentience. Schirmer argues that the arousal mechanism influences the perception at the clock stage, that is it affects the frequency of pulses. Schirmer hypothesis is confirmed by several experiment, done on both non-human animals and humans, showing that artificially changing the arousal impacts the subject time perception. Although several researches confirm the link between arousal and time [25], the relationship between arousal and the presence of an internal clock is still poorly defined. The attention mechanism is based on the way cognitive resources are allocated to stimulus processing, and assumes that the internal clock stages (clock, memory and decision) “share the attentional resource with other ongoing mental processes”. The results of this sharing is that some temporal pulses may be lost. The rationale of this mechanism is that time distortions could be clarified by other processes taking place during rather than after temporal encoding. On the other hand, several studies reveal a correlation between

EFFECTS OF INDUCED EMOTIONAL STATES ON EMOTIONAL EXPERIENCE

81

over/under time estimation and emotional stimuli, while neutral stimuli usually do not affect time perception [12]. The sentient mechanism is closely related to the concept of awareness: all the external and internal signals are incorporated in the brain and give us self-awareness. Emotion induced bodily changes (such as heart–rate changes) increase sentience and consequently sensitivity to time. Hence, the sentient mechanism states that any bodily changes, not only the arousal, can affect time. This proposal was supported by the fact that sentient processing interests the insula, a physical brain region associated to the perception of time [42]. As for the attention model, this mechanism does not take into account the valence of stimuli. The model integration [38], depicted in Figure 1 (right), is able to combine the advantages of all the three mechanisms described above. This model describes the temporal process, which starting from an external event, causes a subjective perception of time, taking into account the characteristics of the triggering event and the way the subject perceives it. When a stimulus appears (e.g. a shrub or a snake), according to its emotional meaning, it can generate bodily changes, which in turn may increase self-awareness or sentience. When stimuli are presented simultaneously, their contribution to bodily and sentient variations will affect the subjective perception of time (e.g., longer for the snake and shorter for the shrub). The aim of this work agenda is to implement Schirmer’s model adding emotional regulation strategy information to the “model integration” process. Although, both psychological and physiological data on time perception exist [8, 13], to the best of our knowledge, there are no research works that separately investigate the effect of each of Ekman’s “basic emotions” [9] on time perception. Moreover, it would be worth defining a paradigm to verify changes in the estimation of time according to the emotional regulation strategy adopted by the subject. 1.2 Memory Memory is the processes by which information is encoded, stored, and retrieved and is the expression of what we have learned. How much can my memories be affected by my mood? Gordon Bower [3] proposed a memory theory based on the context as a facilitator for memory. In one of his studies, he asked a group of subjects to memorize and recall a list of positive or negative colored words, when they were in a bad and in a good mood, separately. Results show that in the recall task, subjects performed better when their moods had the same emotional valence of the words in the list. Research in the field of forensic psychology, where it is necessary to identify the correct time window of a witness episode, have shown that personal characteristics and exposure conditions to a witnessed episode may distort the time perception of the event [24, 33]. An accurate analysis of the literature on this topic indicates that emotional stimuli engage specific cognitive and neural mechanisms that affect memory, through the amygdala functions [1, 34, 20, 19, 7, 22]. According to Hamann [19] emotional arousal influences memory via factors that act during memory encoding (attention and elaboration) and factors that modulate memory consolidation. Brain imaging experiments suggest that the amygdala is involved in emotional responses. The activation of the amigdala – the left part in particular – is significantly higher when the subject watches fearful facial expressions compared to happy ones. Similarly, the left amygdala also activates in response to sad but not angry faces. [37]. Regarding the implication of emotion regulation strategies in the memory, Richards and Gross’s [31, 32] show, through different experimental paradigms, that there is a cognitive cost in using emotion regulation, and that expressive suppression impairs remembrance

82


while reappraisal does not (e.g., a suppression of a facial emotion expression, induced while a subject is watching a movie, reduces memory for the details of the movie). These results emphasize the importance of emotion regulation on emotional memory researches. All the studies, previously described, have analyzed memory performance only in negative contexts: sadness, anger, anxiety and disgust. Moreover the effect of different categories of emotions on memory performance is yet to be considered. 1.3 Aims and research questions The aim of this work is to identify a pattern of behavioral responses to the six basic emotions – happy, anger, fear, disgust, sadness and surprise – contributing to a better understanding of the role of the emotion regulation strategies. Moreover, the recording of the subjects’ facial expressions during the induction procedure will contribute to the creation of a multi-modal database of quasi-spontaneous facial emotional expressions that can be used for experimental emotional research considering the Italian cultural background. The research questions are: 1. Do the different emotional regulation strategies (reappraisal and suppression) play a role on the subjects ability to correctly perform a word memory recognition task? 2. Do they affect the personal perception of time? 3. Are there any specific effects of the emotion categories both for time perception and word memory recognition task? 4. Do the two different emotion regulation strategies produce specific effects on the regulation of emotional facial musculature? 2 Methods, approach: a timescale for research agenda We define a set of psychological experiments and behavioral analyses devoted to investigate the effects of the different induced emotional states and related emotional regulation strategies on word memory recognition performance and related time assessments. The experimental definition consists of six different steps: 1. The identification of video stimuli to elicit emotions, selection and assessment; 2. The identification of a word memory recognition task to assess memory performance; 3. The definition of a time perception task to evaluate time perception; 4. The definition of experimental set–up: participant and procedure; 5. Collection and analysis of emotional facial expressions; 6. Statistical analysis. 2.1 Identification of video stimuli A database of video stimuli inducing specific emotional states will be defined. Video with high emotional content of Fear, Happiness, Sadness, Disgust, Anger, and Surprise will be extracted from YouTube (www.youtube.it) [10]. A neutral state will also be considered, in order to simulate a non emotional scenario. The database will consist of 70 stimuli, 10 videos for each emotion considered. The duration of each video will ranges from 50 to 140 seconds. The collected stimuli will be randomly presented to a group of 20−30 subjects (balanced by gender) in order to be assessed. The subjects will watch the video–clips through a Power Point or a Superlab [43] presentation) and will be asked to label each video according to


83

the seven emotional categories listed above, plus an option for “other emotion” which will be suggested when the video evokes an emotion different from those indicated. Moreover, they will rate, using a Likert scale from 1 (very weak) to 7 (very strong), the intensity of the portrayed emotion. The result of this assessment procedure will be the identification of 35 video stimuli (5 stimuli for each emotional label) selected among those that obtained the highest average intensity rating. 2.2 Word memory recognition task A word memory recognition task, similar to Bower’s paradigm [3], will be defined for this proposal. The procedure will be obtained through an Italian standardization of verbal recognition [40]. About 120 monosyllabic and bi-syllabic Italian words, with low frequency of use will be selected and grouped into 60 target and 60 distractive stimuli. The experimental procedure to perform the word memory recognition task consists of three phases: a) encoding, b) retention interval, c) retrieval. In the encoding phase, the 60 target stimuli will be shown on a computer screen in random order. Each stimulus will be presented for 250 ms with an inter-stimulus interval (ISI) of 2 seconds. Each stimulus is preceded by the presentation of a fixation point lasting 100 ms. The task is to memorize the target stimuli. During the retention interval subjects watch the emotional video stimuli selected in the first step. In the retrieval phase, a combination of the target and distractive stimuli will be displayed on the screen following the same procedure used for the encoding phase. The participants have to provide a YES (if the stimulus was already seen previously in the encoding phase) or NO (otherwise) answer. 2.3 Time perception task The subjects’ time perception will be detected through a personal interview and by reproduction methods [4]. The subjects will be advised to be as accurate as possible in making time judgments. At the end of the word memory recognition task, the subjects will be asked to provide a verbal estimation of how long the task took for them and press a specific key on the keyboard for the same time interval they estimated the task duration. 2.4 Experimental Set–up: participant and procedure The perceptual experiments will involve 256 participants (128 females and 128 males). Before starting the experiment, the participants will be asked to answer the ERQ questionnaire in order to identify which of the two emotion regulation strategies (reappraisal and suppression) they habitually use. Subject will be partitioned into 8 groups (32 subject per group), balanced among gender and emotion regulation strategies: 8 females reappraisal, 8 females suppression, 8 males reappraisal and 8 males suppression. Eight experimental conditions will be performed: for each of the 6 basic emotion, a group will watch videos associated to that emotion; One group will watch videos associated to a neutral state; and one group will participate to the experiment without any video induction (such subjects will perform a distracting task such as, for instance, mathematical operations). Each experiment will consist of an encoding phase, a retention phase, a retrieval phase, and a time perception task. The entire experiment will be recorded by a camera in order to be able to extrapolate subjects facial expressions. 2.5 Collection and analysis of emotional facial expressions The collected facial expressions will be indexed by both emotion and emotion regulation strategy. Thereafter two judges, expert in FACS (facial action coding system), will analyze, classify and qualitatively evaluate the facial muscles changes, either watching the recordings of the subjects in slow motion or identifying the peak of the emotional expression (which will be captured

84


as a JPEG image), its duration and expression intensity. The selected images will be randomly presented, using Super-Lab, to a group of 20 − 30 subjects who will indicate, for each subject and each emotion the intensity, on a Likert scale that ranges from 1 to 7, of emotional expression observed. 2.6 Statistical analysis Statistical analysis of data will analyze the subjects’ performance process and will allow to build a model of cognitive and behavioral responses to the different categories of induced emotion according to the emotional regulation strategies defined by Gross. Memory performance will be evaluated according to the distribution and frequency of responses and the percentage of correct recognition for each of the verbal stimuli proposed. Results will be presented and discussed in terms of confusion matrices. An ANOVA (ANalysis Of VAriance) analysis will be performed to assess the impact of the two emotion regulation strategies and the experimental condition on time perception and memory performances: the emotion regulation strategies will be considered as a 2 level between variable while the experimental conditions will be an 8 level between variable. The response time, emotion-induced, and the subjects’ time estimation will be considered as within variables. The ANOVA will be performed separately by gender, thereafter the data will be combined to verify a gender effect. Other statistical analysis, such as post-hoc correlations will be performed to refine on the experimental results. 3 Conclusion The proposed agenda describes a set of experiments and behavioral analyses devoted to investigate the effects of induced emotional states and related emotional regulation strategies on word memory recognition performance and time perception. Experimental results will be used to characterize the impact of the user’s emotional experience in different emotional contexts and will set the basis for a new clinical psychological orientation to diagnose psychiatric disorders according to time and memory performance and emotion regulation. From a practical point of view, the experiment results could be used to improve human machine interaction interfaces. The recording data (facial emotional expressions) will be available free and will contribute to the creation of a multi-modal database of quasi-spontaneous facial emotional expressions. References [1] M. T. Banich, K. L. Mackiewicz, B. E. Depue, A. J. Whitmer, G. A. Miller, and W. Heller. Cognitive control mechanisms, emotion and memory: a neural perspective with implications for psychopathology. Neuroscience and biobehavioral reviews, 33(5):613–630, 2009. [2] J. A. Bargh and L. E. Williams. The nonconscious regulation of emotion. In J. J. Gross, editor, Handbook of emotion regulation, pages 429–445. New York: Guilford Press., 2007. [3] G. H. Bower. Mood and memory. American Psychologist, 36(2):129–148, 1981. [4] S. Brown. Time perception and attention: The effects of prospective versus retrospective paradigms and task demands on perceived duration. Attention, Perception, & Psychophysics, 38:115–124, 1985. 10.3758/BF03198848. [5] L. Campbell-Sills and D. H. Barlow. Incorporating emotion regulation into conceptualizations and treatments of anxiety and mood disorders. In J. J. Gross, editor, Handbook of emotion regulation, pages 542–559. New York: Guilford Press., 2007. [6] C. M. Deveney and D. A. Pizzagalli. The cognitive consequences of emotion regulation: An erp investigation. Psychophysiology, 45(3):435–444, 2008. [7] R. J. Dolan, R. Lane, P. Chua, and P. Fletcher. Dissociable temporal lobe activations during emotional episodic memory retrieval. NeuroImage, 11(3):203 – 209, 2000. [8] S. Droit-Volet, S. L. Fayolle, and S. Gil. Emotional state and time perception: effects of film-induced mood. Frontiers in Integrative Neuroscience, 5:, 2011. [9] P. Ekman. Are there basic emotions? Psychological Review, 99(3):550–553, 1992.


85

[10] A. Esposito, V. Capuano, J. Mekyska, and M. Faundez-Zanuy. A naturalistic database of thermal emotional facial expressions and effects of induced emotions on memory. In Esposito et al, editor, Cognitive Behavioural System, volume LNCS 7403. Springer Verlag, 2012. [11] J. Gibbon, R. M. Church, and W. H. Meck. Scalar timing in memory. Annals of the New York Academy of Sciences, 423(1):52–77, 1984. [12] S. Gil, S. Rousset, and S. Droit-Volet. How liked and disliked foods affect time perception. Emotion, 9(4):457–463, 2009. [13] P. R. Goldin, K. McRae, W. Ramel, and J. J. Gross. The neural bases of emotion regulation: Reappraisal and suppression of negative emotion. Biol Psychiatry, 63(6):577, 2008. [14] J. J. Gross, L. L. Carstensen, M. Pasupathi, J. Tsai, C. G. Skorpen, and A. Y. Hsu. Emotion and aging: experience, expression, and control. Psychol Aging, 12(4):590–599, 1997. [15] J. J. Gross and O. P. John. Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. Journal of Personality and Social Psychology, 85(2):348–362, 2003. [16] J. J. Gross, G. Sheppes, and H. L. Urry. Emotion generation and emotion regulation: A distinction we should make (carefully). Cognition AND Emotion, 25(5):765–781, 2011. [17] J. J. Gross and R. A. Thompson. Emotion regulation: Conceptual foundations. In J. J. Gross, editor, Handbook of emotion regulation, pages 3–24. New York: Guilford Press., 2007. [18] J.J. Gross and R.W. Levenson. Hiding feelings: the acute effects of inhibiting negative and positive emotion. J Abnorm Psychol., 106(1):95–103, 1997. [19] S. B. Hamann. Cognitive and neural mechanisms of emotional memory. Trends in Cognitive Sciences, 5(9):394 – 400, 2001. [20] S.B. Hamann. Amygdala activity related to enhanced memory for pleasant and aversive stimuli. Nat. Neurosci., 2:289–293, 1999. [21] J. J. Gross J. Gruber, A. G. Harvey. When trying is not enough: Emotion regulation and the effort-success gap in bipolar disorder. Emotion, In Press:, 2012. [22] I. S. Johnsrude, A. M. Owen, N. M. White, W. V. Zhao, and V. Bohbot. Impaired preference conditioning after anterior temporal lobe resection in humans. The Journal of Neuroscience, 20(7):2649–2656, 2000. [23] S. H. Kim and S. Hamann. Neural correlates of positive and negative emotion regulation. J Cogn Neurosci., 106(1):95–103, 2007. [24] E. F. Loftus and J. C. Palmer. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13(5):585 – 589, 1974. [25] W. H. Meck. Selective adjustment of the speed of internal clock and memory processes. Journal of Experimental Psychology: Animal Behavior Processes, 9(2):171–201, 1983. [26] K. N. Ochsner and J. J. Gross. The cognitive control of emotion. Trends in Cognitive Sciences, 9(5):242 – 249, 2005. [27] K. N. Ochsner and J. J. Gross. Cognitive emotion regulation: Insights from social cognitive and affective neuroscience. Current Directions in Psychological Science, 17:153–158, 2008. [28] K..N. Ochsner, R. D. Ray, J. C. Cooper, E. R. Robertson, S. Chopra, J. D. E. Gabrieli, and J. J. Gross. For better or for worse: neural systems supporting the cognitive down- and up-regulation of negative emotion. NeuroImage, 23(2):483 – 499, 2004. [29] J. Posner, J. A. Russell, A. Gerber, D. Gorman, T. Colibazzi, S. Yu, Z. Wang, A. Kangarlu, H. Zhu, and B. S. Peterson. The neurophysiological bases of emotion: An fmri study of the affective circumplex using emotion-denoting words. Human Brain Mapping, 30(3):883–895, 2009. [30] J. Posner, J. A. Russell, and B. S. Peterson. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev Psychopathol., 17(3):715–734, 2005. [31] J. M. Richards and J. J. Gross. Emotion regulation and memory: The cognitive costs of keeping one’s cool. Journal of Personality and Social Psychology, 79:410–424, 2000.

86


[32] J. M. Richards and J. J. Gross. Personality and emotional memory: How regulating emotion impairs memory for emotional events. Journal of Research in Personality, 40(5):631 – 651, 2006. [33] H. L. Roediger and K. B. McDermott. Creating false memories: Remembering words not presented in lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(4):803–814, 1995. [34] B. Roozendaal, B. S. McEwen, and S. Chattarji. Stress, memory and the amygdala. Nature Reviews Neuroscience, 10:423–433, 2009. [35] J. Rottenberg and J. J. Gross. When emotion goes wrong: Realizing the promise of affective science. Clinical Psychology: Science and Practice, 10:227–232, 2003. [36] J. A. Russell. A circumplex model of affect. J Pers Soc Psychol, 39:1161–1178, 1980. [37] D. Sander, J. Grafman, and T. Zalla. The human amygdala: an evolved system for relevance detection. Rev Neurosci., 14(4):303–316, 2003. [38] A. Schirmer. How emotions change time. Front Integr Neuroscience, 5(58):, 2011. [39] G. Sheppes and J. J. Gross. Is timing everything? temporal considerations in emotion regulation. Pers Soc Psychol Rev, 15:319–331, 2011. [40] D. Smirni, P. Turriziani, M. Oliveri, P. Smirni, and L. Cipolotti. Standardizzazione di tre nuovi test di memoria di riconoscimento verbale e non verbale: uno studio preliminare. Giornale italiano di psicologia, 1:227–248, 2010. [41] M. Treisman. Temporal discrimination and the indifference interval: Implications for a model of the “internal clock”. Psychological Monographs: General and Applied, 77:1–31, 1963. [42] M. Wittmann. The inner experience of time. Phil. Trans. R. Soc. B, 364(1525):1955–1967, 2009. [43] http://www.superlab.com/. ?Copyright 2012 Cedrus Corporation.

Communicated by Shunsuke Sato

Vincenzo Capuano, Gennaro Cordasco and Anna Esposito: Second University of Naples and IIASS, Via Vivaldi, 43, 81100 - Caserta Italy Email: {vincenzo.capuano, gennaro.cordasco, anna.esposito}@unina2.it Maria Teresa Riviello: Second University of Naples and IIASS, Via Vivaldi, 43, 81100 - Caserta Italy Email: [email protected]