Researchers have identified trust as a key ingredient necessary for virtual .... scheme or to use an existing scheme, the codes chosen must correspond to the.
A QUALITATIVE INVESTIGATION OF EMOTIONAL STATEMENTS RELATED TO TRUST DEVELOPMENT Herbert Remidez, Jr., University of Arkansas, Little Rock, USA James M. Laffey, University of Missouri, Columbia, USA Antonie Stam, University of Missouri, Columbia, USA ABSTRACT Researchers have identified trust as a key ingredient necessary for virtual teams to work effectively [2][6]. However, they have not identified scalable methods that consistently promote trust within virtual teams. The purpose of this study was to investigate the impact of a new communication support system for virtual teams on messaging behaviors associated with building trust. Qualitative analysis of text messages exchanged by participants was undertaken as part of this investigation. The results contribute insights for designers of information systems, suggestions for extending a theory of organizational communication, and a detailed description of the use of qualitative methods to investigate phenomenon that are generally investigated using quantitative methods.
INTRODUCTION Today virtual teamwork plays an increasingly important role in the modern enterprise. It is not surprising that the challenge of creating an organizational environment in which virtual teams can flourish and perform effectively has caught attention in both industry and academia. The steady growth of the digital organization with its temporary and distributed work combined with Internet-based tools becoming more sophisticated suggest that distributed teams are likely to become more prevalent [15]. Given that virtual teams interact entirely via computer-based systems, part of the solution to the problem of building a more effective virtual organization might lie in the design of the information systems these teams utilize. Well-designed communication support tools that affect not only the way in which a message is delivered but also the message itself can play a central role in creating an effective virtual team environment. This study examines the impacts of an innovative communication support tool on the communication process.
REVIEW OF THEORETICAL FOUNDATIONS CAMOC-DIT In order to understand how IS might be designed to help promote trust in virtual teams, it is useful to develop insight into how communication strategies, medium selection and other aspects of the communication process impact the communication process. While Jarvenpaa et al. [6] provided insight into how trust develops in virtual teams, Te’eni’s Cognitive-Affective Model of Organizational Communication for Designing Information Technology (CAMOC-DIT) [15] provides a framework for understanding the overall organizational communication process. The overall CAMOC-DIT model consists of three major components, 1) communication inputs, which are composed of the task, sender/receiver distance and values and norms, 2) the communication process, which includes the goal, strategies, medium and message form, and 3) the communication impact on actions and relationships. Communication strategies aimed at reducing communication complexity bond these elements together. This study focused on the impact of an invention on the communication processes related to relationship development.
Trust Research has identified trust as a key element of effective virtual teams [5] [6]. Hence, the question of how managers can and should promote trust within virtual teams is of immense interest, both from a theorist and a practitioner viewpoint. A large and expanding body of literature demonstrates the importance of trust in facilitating cooperation [9], a freer flow of information [12], and self-managed work teams [4]. Building on Mayer et al. [11], Jarvenpaa et al. [6] identified communication characteristics of temporary virtual teams that developed high levels
of trust. The next step is for researchers to understand factors that promote or inhibit these communication patterns. This is the step this study aimed to make a contribution toward. In addition to direct instruction in trust building strategies, another approach to promoting trust development is to borrow a strategy from researchers in the learning sciences and modify a communication tool to promote actions that are believed to lead to stronger trust bonds [10]. However, little is known about the impact of these systems on the content of messages or the impact of the environment on the use of these tools.
Research Question and Hypotheses The purpose of this study was to examine the impact of a template-driven asynchronous communication support tool on the presence of emotional statements in electronic messages. Insights gained from this investigation will prove useful to software developers designing and researching future communication support systems. Specifically, this study addressed the following research question: To what extent do message templates influence communicators’ inclusion of affective statements in their messages? Quantitative methods were employed to test the following hypothesis, which was derived from this question. H1: There is a significant difference in the mean number of affective statements made by temporary virtual team members who used a template-driven messaging system and those that used a non-template-driven messaging system.
RESEARCH METHOD Sample The experiment involved 40 subjects, all of them MBA students at a large mid-western university. Participants were assigned randomly to five member teams, and teams were randomly assigned to treatments. A team size of about five has previously been employed in related studies [6] [14]. Earlier field tests of the study in another academic unit at the same university were used to fine-tune the experiment.
Procedures The study took place over five weeks: an initial pre-test was followed by three weeks of participant interaction, a post-test, and several days of interviews. After the initial face-to-face introduction of the experiment, all additional directions and information were handled through e-mail. The control group used a standard, hierarchicallyformatted, asynchronous discussion board, while the treatment group used a hierarchically-formatted, templatedriven discussion board application that employed templates designed to support them in taking actions characteristic of high trusting teams. The process for creating a template-driven message involved four screens and a structured interaction. Depending on the nature of the message, the user was guided through a specific series of steps and choices, and was prompted for certain types of information and suggestions on how to proceed next. For a detailed description of the system employed, please see [13]. Users of the template-driven discussion board received a description of each template, which including suggestions of statements users might include in their messages. Users of the regular discussion board did not receive any such directions or suggestions.
MEASURES Messages Analysis To investigate emotional statements included in the messages, all of the messages authored by participants in both treatment groups were collected from the database of the communication support system used in this study and saved into separate spreadsheet files. There were a total of 206 messages authored (control group = 86; template
group = 120); the average word length was 526 words per participant (control = 399; template = 653), and the average participant posted 5 messages (control = 4 treatment = 6). A focused approach to Chi’s [3] verbal analysis method was employed to analyze the context of the messages for the presence of affective statements. This technique can be used to identify themes or detect the presence or absence of specific types of statements. The latter case is how it was used in this study. It relies strictly on the qualitative data, but quantifies the analysis. Excluding the initial collection of the data, the verbal analysis technique consists of eight steps. The steps in the verbal analysis technique are: 1. Reducing or sampling the protocols. 2. Segmenting the reduced or sampled protocol (sometimes optional). 3. Developing or choosing a coding scheme or formalism. 4. Operationalizing evidence in the coded protocols that constitute a mapping to some chosen formalism. 5. Depicting the mapped formalism (optional). 6. Seeking pattern(s) in the mapped formalism. 7. Interpreting the patterns. 8. Repeating the process, perhaps coding at a different grain size (optional). The first two steps in the verbal analysis process involve selecting the data to be coded and selecting the unit of analysis. Because verbal data tend to be voluminous, oftentimes the first step is to reduce the amount of data that will be coded. Chi [3] lists three general heuristics for data reduction: 1) random sampling, 2) selecting a subset based on some “noncontent” criterion, and 3) completing preliminary coding of the data and then selecting a subset to code in more detail. The sample for this study was the content of participants’ messages posted to the discussion boards, and all of the messages were coded. The second step, segmenting the protocols, is used to determine the unit that will be analyzed. Two general strategies a researcher can use to segment the protocols include segmenting on non-content features (e.g. page, paragraph, sentence, phrase, etc.) or segmenting on content features (topics of discussion, argument chains, impasses, etc.). Selecting the segment size, or grain size, greatly impacts how timeintensive the analysis will be. One benefit of the verbal analysis technique is that the researcher always can go back and analyze the data using a different segment size if the original coding does not answer the research questions or if it created other questions. Each sentence was chosen as the segment size for this study because this was the level at which the presence of affective statements could best be detected. The third and fourth steps in the verbal analysis technique involve coding the data. The third step is to develop or choose a coding scheme. The selection or development of the coding scheme is one of the most important steps in the verbal analysis technique [3]. This is true because the coding scheme depends on a “researcher’s theoretical orientation, the hypothesis or questions being asked, the task, and the content domain” [3]. Whether one chooses to develop his or her own coding scheme or to use an existing scheme, the codes chosen must correspond to the formalism that will be used to represent the knowledge. Because the purpose of this analysis was to determine if affective statements were present, a rather simple coding scheme of identifying affective statements was adopted. The fourth step, operationalizing evidence for coding, clarifies what qualifies a segment as belonging to a particular category. Examples of affective statements referenced in Jarvenpaa, et al. [6] were used to guide the identification of elements as being affective statements. The fifth and sixth steps in this technique involve representing and making sense of the findings. The fifth step, depicting the mapped formalism, represents the coded data so that it can be looked at in the aggregate. Chi [3] recommends depicting the data for two reasons; the first is to depict the data to the audience and the second is to see if any patterns can be detected. There are numerous formalisms from which a researcher may choose. For example, a researcher could choose a table, a sequence of problem statements, a semantic network, or a tree structure. The formalism used in this study was a simple table listing the various affective statements identified. The sixth step, seeking pattern and coherence in the depicted data, allows the researcher to learn from the formalism that was chosen. The authors estimated that the table representation would help others understand the effectiveness of the selected templates for supporting the inclusion of affectivity.
The seventh and eighth steps in the process involve interpretation and starting over. The seventh step in the process is to interpret the pattern and its validity. In other words, the researcher checks to see if the pattern that was observed in the previous observations is a pattern that can hold up to close scrutiny. An Independent Samples T-test was used to determine if the treatment group shared more affective statements than the control group. The eighth and final step in the process, repeating the whole process, allows the researcher to delve deeper into the data to answer questions that were not answered the first time through or answer questions that emerged during the analytic process. The second or third analysis is often completed at a different level of detail than the first analysis. In this study, because the initial level of analysis was at such a fine grain, there was no need to conduct a finer grain of analysis. The researchers implemented the verbal analysis technique first by saving the messages into a Microsoft Excel worksheet. The sheet was formatted so that each message filled a row. The next step was to read each message and bold each sentence that represented an affective statement. The third step was to have both the researcher and an external reviewer code the messages. After this was completed, the results were compared to identify discrepancies between coders. The researcher and external coder discussed the discrepancies and adjustments were made until the coding was at least 95% consistent. This process required two passes through the data for the researcher, one pass for the external reviewer, and then a meeting to discuss discrepancies. There were so few affective statements that additional passes were not required.
RESULTS The study’s hypotheses stated: “There is a significant difference in the mean number of affective statements made by temporary virtual team members who used a template-driven messaging system and those who used a nontemplate-driven messaging system.” Affective statements were operationalized as expressions of emotions or moods. To conduct this analysis, all of the messages were examined and coded for the presence of affective statements, as described in the Methods section. The researcher and an assistant completed this process independently and then again jointly to resolve discrepancies. A tally of affective statements for each participant was calculated and used for the analysis. The total number of affective statements for the control group was ten, and the template-driven group had twenty affective statements. Neither of the group’s distributions of scores met the normality assumption. A number of transformation strategies were tried with no success. An Independent Samples T-test was conducted to determine if one of the groups posted significantly more affective statements than the other. Below are the results. With 20 participants in each group, the mean number of emotional statements for the control group was .5 (SD .761) and 1.0 (SD 1.17) for the treatment group. The Levene’s test was significant at the .174 level (equal variance assumed) with an F score of 1.923.
Interpretation The results of the Independent Samples T-test did not indicate a statistically significant difference between the two groups on the number of affective statements. Therefore, the hypothesis cannot be accepted. It is worth noting that the treatment group did have twice as many statements as the control group. However, there were so few statements relative to the size of the group that the total difference did not reach the level of statistical significance. Although an Independent Samples T-test is robust in the face of violations of the normality assumption [1], it is important to note that the distribution did not meet the normality assumption.
DISCUSSION This study explored the extent to which message templates influenced the inclusion of affective statements. Jarvenpaa and her colleagues hypothesized that, for trust to develop, “members have to explicitly verbalize their commitment, excitement, and optimism” [7]. In the CAMOC-DIT model, the inclusion of these types of statements is seen as employing an affectivity strategy. Affectivity is just one of many communication strategies (e.g. contextualization, control by planning) identified in the CAMOC-DIT model as a candidate for computer support. Te'eni [15] hypothesized that “templates of appropriate affectivity and feedback on current messages” might be an effective way of supporting affectivity. The templates employed in this study encouraged users to express their commitment, excitement, and optimism.
A process for identifying and tallying the affective statements within each message was described in the results section. In the data analysis, each expression of commitment, excitement, or optimism was counted. A means analysis was then performed to detect any differences in the number of affective statements between the two groups. Although there was a difference in the group means in favor of the treatment group (Regular = 22.68 vs. Structured = 18.33), the difference did not reach a level of statistical significance (p < .05). Therefore, the hypothesis could not be supported. The small number of affective statements identified might be attributed to the nature of the problem. Jonassen [8] presented a “typology of problems, each type of which engages different cognitive, affective, and conative processes” (p. 63). Jonassen’s thesis when creating this taxonomy was that each type of problem necessitates a different instructional strategy. However, his taxonomy can assist in interpreting the results of this study by providing a framework for understanding how the characteristics of the problem might have influenced the participants’ level of affective involvement. The problem used in this study would be considered a decision-making type in Jonassen’s typology because the problem involved identifying benefits and limitations while weighing and selecting options. Continuing the use of the typology, the problem would be classified as being well structured, because the problem and its bounds were presented clearly to the participants. The complexity level would be classified as low to medium, because of the number of issues, functions, variables, the degree of connectivity among those properties; the type of functional relationship among those properties; and the stability among the properties of the problem over time. The problem was abstract in nature because it relied on domain-general strategies as opposed to strategies that only members of a select discipline might know. The fidelity of the problem was low because the problem was filtered and was not presented with all of the other problems/variables that would be in play in a more realistic scenario. Another possible explanation for the results is that the guidance offered in the template descriptions, along with the template labeling, made it easier for users to succeed. For the purpose of this study, templates were operationalized as template labels, structure, and descriptions. The descriptions associated with each template went beyond simply describing the label of the template by offering suggestions for content, behavior, and overall strategies. For example, in addition to suggesting content for the template, the description for the “Introduction” message template included the suggestion that “Expressing positive emotions throughout the project will help your team succeed. For example, ‘I am looking forward to working with you all’ would be good comment to include.’ These descriptions essentially assisted users in creating better and more appropriate messages, which could have led receivers to perceive senders as more competent. Evidence to support this interpretation was found when analyzing data to test the hypothesis related to the number of affective statements in messages. It was observed that there were several instances of template users including similar or identical phrases as suggested in the template descriptions. These phrases, or closely related ones, were identified by previous researchers [6] as ones that likely promoted trust development in virtual teams.
Implications The results of this study offer implications for information system designers and users. Designers should consider how they might incorporate features that scaffold users’ accomplishing tasks and relationship goals that they might not accomplish alone. Although the CAMOC-DIT [15] is a starting point for understanding what types and dimensions of communication scaffolds might be most appropriate for a given set of goals, little is known about this area, particularly in business settings. Exploring the impact of communication scaffolds on communication patterns and relationships is one of the areas discussed below in Suggestions for Future Research.
CONCLUSION This study built on previous studies examining how trust develops in virtual teams and Te'eni’s [15] CognitiveAffective Model of Organizational Communication for Designing IT to develop and implement a communication tool with the intent of influencing communication patterns of the users. Interestingly, and in contrast to what previous research suggested, users of the template-driven discussion board system did not author statistically more
affective statements than users of the regular discussion board. This might have been attributed to the type of problem that was used in this study. Selecting a more emotionally charged or complex problem might have changed these results.
REFERENCES [1] Ary, D., Jacobs,L., and Razavieh,A. Introduction to Research in Education. Belmont, CA:Wadsworth Publishing, 2005. [2] Beranek, P. The Impacts of Relational and Trust Development Training on Virtual Teams: An Exploratory Investigation, In Anonymous Proceedings of the 33rd Hawaii International Conference on System Sciences, Washington, DC, USA: IEEE Computer Society, 2000, pp. 1020. [3] Chi, M. Quantifying qualitative analyses of verbal data: A practical guide. The Journal of Learning Sciences, 6:3, 1997), 271-315. [4] Claus, L. Too much of a good thing? Negative effects of high trust and individual autonomy in self-managing teams. Academy of Management Journal, 47:3, 2004), 385-399. [5] David, P., and McDaniel,R. A field study of the effect of interpersonal trust on virtual collaborative relationship performance. MIS Quarterly, 28:2, (June, 2004), 183-227. [6] Jarvenpaa, S., Knoll,K., and Leidner,D. Is anybody out there? Antecedents of trust in global virtual teams. Journal of Management Information Systems, 14:4, 1998), 29-64. [7] Jarvenpaa, S.L., and Leidner,D.E. Communication and trust in global virtual terms. Organization Science, 10:6, (June, 1999), 791-815. [8] Jonassen, D.H. Toward a design theory of problem solving. Educational Technology Research & Development, 48:4, (December, 2000), 63-85. [9] Krackhardt, D., and Stern,R.N. Informal networks and organizational crises: An experimental simulation. Social Psychology Quarterly, 51:2, (June, 1988), 123-140. [10] Malone, T., Lai,K., and Fry,C. Experiments with oval: A radically tailorable tool for cooperative work. ACM Transactions on Information Systems, 13:2, (April, 1995), 177-205. [11] Mayer, R.C., Davis,J.H., and Schoorman,F.D. An integration model of organizational trust. Academy of Management Review, 20:3, (July, 1995), 709-734. [12] Nelson, K.M., and Cooprider,J.G. The contribution of shared knowledge to IS group performance. MIS Quarterly, 20:4, (December, 1996), 409-432. [13] Remidez, H., Stam,A., and Laffey,J. Web-based template-driven communication support systems: Using shadow netWorkspace to support trust development in virtual teams. International Journal of e-Collaboration, 3:1, (January-March, 2007), 65-83. [14] Tan, B.C.Y., Wei,K.K., Huang,W.W., and Ng,G. A dialogue technique to enhance electronic communication in virtual teams. IEEE Transactions on Professional Communication, 43:2, (June, 2000), 153-165. [15] Te'eni, D. Review: A cognitive-affective model of organizational communication for designing IT. MIS Quarterly, 25:2, (June, 2001), 251-312.
