Regression Modeling of Reader's Emotions ... - Semantic Scholar

5 downloads 128 Views 318KB Size Report
451–470. I-Tech Education and Pub- ... Technical Report A-6, The Center for Research in ... Holsclaw, T.N.: Investigation of repeated measures linear regression ...
Regression Modeling of Reader’s Emotions Induced by Font Based Text Signals Dimitrios Tsonos1, Georgios Kouroupetroglou1, and Despina Deligiorgi2 1 Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, Greece {dtsonos,koupe}@di.uoa.gr 2 Department of Physics, National and Kapodistrian University of Athens, Greece [email protected]

Abstract. In this work we presented a mathematical model for the readers’ emotional state responses triggered by font style, type and color. It is based on multiple regression analysis of the repeated measures from 45 students and for 35 textual stimuli using the Self-Assessment Manikin test. Based on the dimensional theory of emotions, we propose a model on how emotional dimensions Pleasure, Arousal, and Dominance vary according to the typographic text signals: font style, font type and font/background color combinations. We observe that “Pleasure” dimension is affected negatively by font type (“Arial” and “Times New Roman”) and positively by color brightness difference of font/background color combinations. “Arousal” and “Dominance” are affected only by color brightness difference (negative correlation). According to the proposed model, font type “Arial” elicits more pleasant emotional state than “Times New Roman”. The results can be applied to augment user interface experience or to add expressivity in Text-to-Speech systems and provide accessibility of typography induced text signals. Keywords: document accessibility, text signals, Text-to-Speech, Self-Assessment Manikin test.

1

reader’s

emotions,

Introduction

Universal Accessibility of printed and/or electronic documents is an open research topic [1]. Users require documents in alternative formats, depending on her/his interaction preferences - needs and/or device specifications. Text-to-Speech (TtS) is a common software technology that converts in real-time any electronic text into speech [2], that can provide document accessibility through acoustic modality. A document is the “medium” in which a “message” (information) is communicated [3]. The term “signal” is introduced as “the writing device that emphasize aspects of a text’s content or structure without adding to the content of the text” [4]. It attempts to pre-announce or emphasize content and/or reveal content relationship [5-6, 23-26]. The title, heading, typographic cues are considered as signals. Also, “input enhancement” is C. Stephanidis and M. Antona (Eds.): UAHCI/HCII 2013, Part II, LNCS 8010, pp. 434–443, 2013. © Springer-Verlag Berlin Heidelberg 2013

Regression Modeling of Reader’s Emotions Induced by Font Based Text Signals

435

an operation whereby the saliency of linguistic features is augmented through e.g. textual enhancement for visual input (i.e. bold) and phonological manipulations for aural input (i.e. oral repetition) [7]. Tsonos and Kouroupetroglou [1, 18] categorize the signals, focusing on visual presentation of documents, into three layers: logical, layout and typographic: (i) Logical layer: it associates content with architectural elements such as headings, titles/subtitles, chapters, paragraphs, tables, lists, footnotes, and appendices. (ii) Layout layer: it associates content with architectural elements relating to the arrangement on pages and areas within pages, such as margins, columns, and alignment. (iii) Typography layer: it includes font (type, size, color, background color, etc.) and font style such as bold, italics, underline. The term “plain” used in this work indicates text of any font, but without font style. These three layers are complementary and not independent. Typography can be applied to both the logical and the layout layers of a document. For example, a footnote (Logical layer) could be in italics or in smaller font size than the body of the text. The vertical space in a text block, called leading (Layout layer), can be affected by the font type. Moreover, typography can be applied to the body of the text directly, for example, a word in bold can be used either for the introduction of a new term or to indicate a person’s name. All the text signaling devices, either mentioned as signals or layers: a) share the goal for directing the reader’s attention during reading, b) facilitate specific cognitive process occurring during reading, c) ultimate comprehension of text information, d) may influence memory on text and e) direct selective access between and within texts [4]. Current Text-to-Speech (TtS) systems do not include effective provision of the semantics and the cognitive aspects of the visual (such as typography) and non-visual (such as logical structure) text signals [1-2-22]. In order to achieve the automated metadata’s rendition in visual, acoustic and/or haptic modality, it is mandatory to mathematically formulate, either semantic or cognitive responses of user during their interaction with the documents. Emotion variations during reading process are cognitive information that fails to be rendered into acoustic modality. A proposed methodology presented in [1] tries to overcome this obstacle proposing the mathematical formulation of reader’s emotional state response and convey this information into acoustic and/or haptic modality. Focusing on TtS systems, it is proposed the combination of reader’s emotional state response with the mapping rules of existing Expressive Speech Synthesis models e.g. [8]. Thus, we can acoustically map document’s typographic alteration into prosodic variations, namely pitch, rate and volume. This paper proposes a mathematical model of readers’ emotional state response triggered by font style, type and color. It is based on multiple regression analysis of the repeated measures, using the Self-Assessment Manikin test [9]. Based on the dimensional theory of emotions [10], we propose a model on how emotional dimensions (namely Pleasure, Arousal, Dominance) vary according to typographic signals (font style, type and font/background color combinations). We apply multiple

436

D. Tsonos, G. Kouroupetroglou, and D. Deligiorgi

regression analysis on the derived results (based on the methodology described in [11]). A set of three equations describe the relation between the three emotional dimensions (dependent variables) on typographic signals (independent variables).

2

The Experimental Procedure

Following the dimensional approach of emotions [10], Lang [9] introduced the SelfAssessment Manikin test (SAM) to assess the emotional states “Pleasure,” “Arousal,” and “Dominance” of the participants (known also as the “PAD test”). Synonyms are used for the expression of the PAD dimensions. “Pleasure” can be replaced by “valence” or “evaluation,” “Arousal” by “activation” or “activity,” “Dominance” by “power” or “potency.” The participants can choose between at least five selections of manikins. There are no verbal expressions to assess their emotional states. For the emotional state of “Pleasure” the rating spans from a happy (smiling) manikin to an unhappy (frowning) one. For the “Arousal” dimension the one pole is represented by a highly energetic manikin and the other by a relaxed with eyes-closed one, while for “Dominance” the controlled and in-control poles are represented by a small and large manikin, respectively (Figure 1).

Fig. 1. The manikins of SAM Test as they are displayed during the experimental procedure

Regression Modeling of Reader’s Emotions Induced by Font Based Text Signals

2.1

437

Participants

Forty-five students participated in the experiments, ranging from 18 to 36 years old (mean age 24.4 years, SD=4.61). They were 23 male and 22 female undergraduate or postgraduate students and their native language was Greek. All of them have selfreported normal color vision and normal or corrected-to-normal visual acuity. 2.2

Stimuli

Thirty five combinations of font type, style and color attributes were investigated using Greek text (a paragraph with approximately 46 words) from which any content; emotion, and/or domain dependent information were excluded. All stimuli were displayed in a random sequence on a 17-inch LCD display with resolution, in full screen mode, using 32 bit color depth. The 35 combinations are (abbreviations explained in Table 1): • YU, WB, WU, RG, BG, BW, GY on “Times New Roman” using “Plain, “Bold” and “Bold-Italics” • YU, WB, WU, RG, BG, BW, GY on “Arial” using “Bold” and “Bold-Italics” Table 1. The text signals and their corresponding attributes used in the experimental procedure

Font/Background Color Yellow Blue (YU) White on Black (WB) White on Blue (WU) Red on Green (RG) Black on Grey (BG) Black on White (BW) Green on Yellow (GY) 2.3

Font Style Plain Bold Bold-Italics

Font Type Times New Roman Arial

The Experimental Procedure

At the beginning of the experiment, the participants were asked to read carefully the exact instructions provided by the IAPS guidelines [9] and ask the instructor for possible clarifications. Then, they were asked to fill in the electronic form with their demographic information and a declaration that they agree to take part in the experiment. Afterwards, they participated in a demo version of the experiment to be familiarized with the procedure. Each stimulus was displayed for 10 seconds, and after that, participants were asked to assess their emotional state on a 9-point PAD scale using the manikins provided by the SAM test (Figure 1).

438

3

D. Tsonos, G. Kouroupetroglou, and D. Deligiorgi

Multiple Regression Analysis

In multiple linear regression, the value of a variable Y can be predicted using multiple predictor variables (X1, X2, X3, …). The mathematical notation is: ·

·

·

·

(1)

where a1, a2, a3… an are the regression coefficients for the respective variable and b is the regression constant. In this study we refer to Y variable as the dependent variable and Xn variables as the independent. There are many ways to perform multiple (or single) regression analysis, e.g. least squares, means regression, generalized least squares [12]. In this work we use weighted means regression analysis. The responses are averaged on each stimulus over subjects (along with their respective standard error - SE) and then weighted least squares regression on means was applied. According to [12] and [13] this method: a) provides unbiased estimators, b) is simple and fast and c) it is easy to manipulate the regression approach that is described in the following section. For the analysis we used the statistical software Origin [14]. 3.1

The Variables

The dependent variables used in this experimental design are “Pleasure”, “Arousal” and “Dominance” and their values are in the [-1, 1] continuous space. Each variable consist a separate regression analysis. The independent variables are: • Font Style Variables. We have selected two dummy variables, namely “Plain” and “Bold”. These two variables can be assigned with boolean values (0 or 1). The value 1 denotes the existence of the typographic attribute. E.g. if the text is plain then “Plain” variable is 1 and “Bold” variable is 0. Similar it is in the case of bold text. We also want to represent “Bold-Italics”. To obtain this we assign the value 0 to both “Plain” and “Bold” variables. • Font Type. This variable is also a dummy one. The value 1 corresponds to “Times New Roman” font type attribute and value 0 to “Arial”. • Color Brightness Difference (DBrightness). We have selected the color difference between font and background color. The value of the variable can be calculated using equations (2) and (3) [15] by applying the corresponding RGB scale for the selected colors. The variable is continuous ranging [0, 255]. ℎ ℎ



=| =(

×299+



− ×587+

Note: The independent variables are little or no correlated.

|

(2)

×114) /1000

(3)

Regression Modeling of Reader’s Emotions Induced by Font Based Text Signals

3.2

439

Predictor Variable Determination

The multiple regression analysis can be performed in several ways (e.g. forward or backward) [11]. In the present study we follow the backward approach (full model), namely that at the beginning of the regression analysis we add all the independent variables that we want to observe, and the significance level for each predictor determines which variable is excluded. In detail, the next steps were followed: • • • • •

Determine the regression model Determine R2 Determine whether the multiple regression is statistically significant Determine the significance of the predictor variables Recalculate the entire solution without the dropped predictor(s).

Table 2 presents the multiple regression results including all the independent variables. The R-value is 0.78 and adjusted-R2 is 0.556. The overall ANOVA regression is statistical significant (F=11.651, p