Performance patterns in face-to-face and computer ... - IngentaConnect

2 downloads 0 Views 223KB Size Report
achieve that by comparing face-to-face (FTF) and computer-supported (CS) teams over a series of three sessions on three response variables: performance, ...
The current issue and full text archive of this journal is available at www.emeraldinsight.com/1352-7592.htm

Performance patterns in face-to-face and computer-supported teams Pilar Pazos Department of Engineering Management and Systems Engineering, Old Dominion University, Norfolk, Virginia, USA, and

Mario G. Beruvides

Performance patterns in teams

83 Received July 2010 Revised November 2010 Accepted December 2010

Industrial Engineering Department, Texas Tech University, Lubbock, Texas, USA Abstract Purpose – This paper presents a longitudinal experimental study on teams with the purpose of investigating the impact of communication media on decision-making teams. The authors aims to achieve that by comparing face-to-face (FTF) and computer-supported (CS) teams over a series of three sessions on three response variables: performance, cohesiveness, and synergy. Design/methodology/approach – A total of 24 teams, each of five students, participated in three separate decision-making sessions in which they solved a survival simulation scenario. Each team was randomly assigned to either face-to-face (FTF) or computer-supported (CS) communication condition. The analysis compared overall means and mean patterns over time on the three response variables across the two communication media. Findings – Results suggest that there were no differences in overall performance between CS and FTF teams and no differences in performance changes over time between the two media; there were no overall differences in overall synergy or synergy changes over time; and FTF teams reported higher average cohesiveness than CS teams, but cohesiveness improved at a faster rate in CS teams than in FTF teams. Overall these results suggest that the CS communication did not reduce the group’s ability to work together. Moreover, the higher increase in cohesiveness reported by CS teams suggests that the ability to build relationships can increase over time. Practical implications – Given the prominence of information technologies as a communication mechanism, the question of how team members in remote locations perform over time is of great theoretical and practical importance. Originality/value – This study provides some preliminary evidence that computer communication does not significantly reduce the group’s ability to perform over time for decision-making tasks. CS teams report lower overall levels of cohesiveness which could indicate that some communication barriers might still limit the group’s ability to build relationships. Keywords Team working, Group work, Decision making, Information media, Team performance, Feedback Paper type Research paper

Introduction There is broad agreement that teams are instrumental for companies’ success in today’s competitive environment. In particular, teams play a key role in decision-making processes. For years now, many companies have been using information technology to support collaborative tasks (Davidow and Malone, 1992;

Team Performance Management Vol. 17 No. 1/2, 2011 pp. 83-101 q Emerald Group Publishing Limited 1352-7592 DOI 10.1108/13527591111114729

TPM 17,1/2

84

Jarvenpaa and Ives, 1994; Martins et al., 2004) through the use of virtual teams. However, our understanding of how virtual teams use technology to communicate during decision making is very limited. For the purpose of this study, we define a virtual team as one whose members are working towards a common goal by crossing geographic boundaries (Martins et al., 2004) and using information technology (e.g. e-mail, videoconference) to communicate. For the remaining of the document, we will refer to computer-supported teams as a specific type of virtual team who uses a computer interface to support their communication. Given the prominence of information technologies as a communication mechanism, the question of how effective team members are in making decisions from remote locations is of great theoretical and practical importance. Our understanding of how virtual teams perform is limited by the fact that most studies evaluated those teams by comparing them with face-to-face teams at only one instance (e.g. Lea and Spears, 1991; Straus, 1997). Such studies did not account for the dynamic and complex nature of groups. Studies investigating group performance over time have just recently been the object of more attention, but there is still much we do not know about how computer supported-groups learn and perform over time. With this research, we attempt to shed some light into this area by comparing the performance patterns of computer-supported groups and face-to-face groups over time. Knowing how different media impacts group performance and interpersonal relationships is critical to team managers in today’s work environments. The theoretical foundation for this investigation was established through the analysis and integration of applicable theories and concepts that are presented next. We begin by providing a brief overview of relevant findings related to decision making and computer-supported groups. Next, we present the theoretical model and the research methodology that we used to test the hypotheses. We conclude by providing results and implications for team leaders and team members. Decision-making tasks A majority of experts in team research (Hackman and Morris, 1975; Marks et al., 2001; McGrath and Arrow, 2000) would agree that the performance of a group cannot be studied generically without specific regard to the task. This recommendation is based in the notion that performance is strongly influenced by the type and characteristic of the task on hand. As a result, the type of task has been reported as a critical variable to consider when studying team-related processes and outcomes (McGrath, 1984). The most accepted and cited typology of group tasks was developed by McGrath (1984), and it describes the main types of tasks that teams may encounter. Based on that typology, decision-making tasks are defined as follows (McGrath, 1984): . those for which a right answer can not be identified with certainty; and . those for which the group needs to select an answer by consensus. This study focused on teams solving decision-making tasks. Virtual teams and computer support The increasing presence of virtual teams has resulted in abundant research examining various aspects of virtual team adoption, use, and performance. Previous studies

investigating the impact of information technologies on group outcomes are not conclusive and often times contradictory. Our understanding of group performance and media use is limited by the fact that most research has been conducted using ad-hoc groups, that is, groups that met only once for the purpose of the research study with no prior history of working together. Moreover, data collected in those studies was frequently limited to one point in the life of the team. Such studies did not account for temporal changes inherent to the dynamic nature of groups. Only a small number of studies have addressed temporal issues in decision-making groups (Alge et al., 2003; Arrow and McGrath, 1993; Kelly and Karau, 1999; Marks et al., 2001; McGrath and O’Connor, 1996). Alge et al. (2003) examined temporal issues and identified type and amount of experience as important factors that may enable virtual groups to overcome the limitations of electronic media. Scholars have called for additional research examining temporal scope, to account for the dynamic nature of teams (Alge et al., 2003; Baltes et al., 2001; Bell and Kozlowsky, 2002; McGrath and Arrow, 2000; Marks et al., 2001). Most studies exploring the performance of groups when using computer-supported platforms based their results on groups lacking previous contact (zero-history groups). This research attempts to shed some light on the topic by using a longitudinal study that evaluates groups over a period of time. Understanding performance in computer-supported groups An abundance of research can be found in the area of computer-supported groups (Alge et al., 2003; Baltes et al., 2001; Burke and Chidambaram, 1995; Poole and DeSanctis, 1990). Several theories have attempted to explain the impact of computer support on groups. The three theories more relevant for the current study are: media richness theory (MRT), social information processing theory (SIP), and adaptive structuration theory (AST). These theories will be presented to describe the influence of computer-supported communication on team outcomes and will be utilized to support the research hypotheses. Media richness theory (MRT) (Daft and Lengel, 1986) describes the fit between specific types of communication media and the task on hand. MRT suggests that computer-mediated group communication has lower social presence and less task focus than face-to-face communication (Daft et al., 1987; Sproull and Kiesler, 1986). The supporters of media richness theory suggest that leaner media (e.g. instant message, e-mail) are better suited for non-equivocal tasks such as communicating information, whereas richer media (video conferencing) are considered more appropriate for more equivocal tasks (where different interpretations of the data are possible). Media richness suggests that limitations in lean media such as instant message or text-based communication are expected to persist over time. However, some studies have reported that media richness theory has not been able to successfully predict media choice in managers (Lee, 1994; Markus, 1994). Chidambaram (1996) outlines the limitations of media richness theory as a deterministic model that generally seeks to explain group performance through the inability of certain media (like computer-supported communication, e-mail, etc.) to convey visual and auditory cues. As a result, MRT does not fully account for the complexity of the interaction between teams and communication technology.

Performance patterns in teams

85

TPM 17,1/2

86

Social information processing (SIP) theory takes a different stance by arguing that media effects on group outcomes and processes are likely to be transitory or change as teams use the technology (Walther, 1992a, b, 1996). Rather than suggesting that certain media is constrained by static limitations, SIP theory suggests that the impact of media on group outcomes will likely evolve over time, allowing for social processes and group outcomes to be enhanced as users gain familiarity with each other and gain expertise with the media and their features. Social information processing (SIP) theory provides some explanation for how a medium may, over time, allow groups to engage in increased relational communication. The theory suggests that team members are driven to develop social relationships and that those relationships can also be built in computer-mediated groups (Walther, 1996, p. 10). Walther (1992a, b) suggests that, over time, users will adapt to the communicative cues that the media offers them; therefore, the role of time should not be ignored in the study of group communication and behavior. We attempt to address this gap by evaluating groups on a series of sessions. Using a similar argument, Chidambaram (1996) proposes that the users of computer media can adapt the medium to meet their relational needs. Such adaptation encompasses alternative uses of the media or innovative ways of overcoming inherent communication or structural barriers. A third theory describing the impact of communication technologies on team outcomes is adaptive structuration theory (AST). AST was first proposed by Poole and DeSanctis (1990) and suggests that teams communicating through technology develop structures and methods of communication through time, such that if given adequate time, they can reach a relational level similar to that of face-to-face groups. It describes the adjustment process when appropriation of a new technology occurs. In particular, the theory focuses on how actions of social members of a collective create the structures that enable and constrain future actions. To sum up, MRT is a deterministic theory suggesting that lean media such as chat or IM will lead to lower levels of performance for equivocal tasks such as decision-making tasks. The theory suggests that media effects and constraints remain static over time and disregards the dynamic and complex nature of teams and the adaptation to the environment and to the media. As a result, MRT offers limited insight about the actual media impact on group outcomes over time. This study will evaluate outcome patterns over time and explore a team’s adaptation to the media. Team cohesiveness Cohesiveness has been a variable of interest in the study of teams. Festinger (1950, p. 274) defined group cohesiveness as “the resultant forces which are acting on the members to stay in a group”. Prior studies revealed a significant relationship between group cohesiveness and group performance (Mullen and Copper, 1994; Zaccaro and Lowe, 1988). Zaccaro and Lowe (1988) also found that cohesiveness is directly related to task commitment (Zaccaro and Lowe, 1988). A study by Warkentin et al. (1997) using student teams indicated that face-to-face teams reported higher levels of cohesiveness than did VTs. A more recent study reported a positive relationship between task cohesiveness and team effectiveness for dispersed teams (Gonzalez et al., 2003). On the other hand, Aiello and Kolb (1995) found that higher levels of cohesiveness did not result in increased productivity for VTs when working on a simple task.

The relationship between cohesiveness, satisfaction, and performance has been pointed out as a critical one to understand in workgroups (Hackman and Morris, 1975; Zaccaro and Lowe, 1988). Burke et al. (1999) suggest that media characterized by less capacity limits the exchange of verbal and visual cues facilitating a type of interaction that has more task-focused and less relational-focus. Their results also indicate that although team cohesiveness can be lower for leaner media, computer-supported groups have the potential to increase their cohesiveness over time. Since cohesiveness is mostly a relational type of variable, we expect average cohesiveness to be lower in computer-supported groups than in face-to-face groups. However, we expect group cohesiveness to increase over time in both settings, with higher rate of increase for computer-supported groups as group members adapt to the technology and to each other.

Performance patterns in teams

87

Conceptual model Most real teams have a shared history; thus, their interaction and their previous performances will likely shape the team dynamics and future performance. This study uses a longitudinal approach that collected data on teams over three sessions, each of them taking place two weeks apart. The goal was to examine teams’ dynamic patterns of performance, synergy, and cohesiveness over the three sessions in two different settings, face-to-face (FTF) and computer supported (CS). This study incorporates an exploration into performance dynamics by evaluating teams over time using three separate sessions in which they solved independent tasks. Figure 1 depicts the proposed conceptual model with the relationships that this study seeks to investigate. The independent variables are communication mode (FTF versus CS) and session number (1, 2, 3). The right side represents the response variables: performance, cohesiveness, and synergy, which will be collected using a longitudinal study approach throughout three sessions. Research hypotheses Based on media richness theory and the reported limitations of computer-supported media to transmit information when compared to face-to-face media (Daft and Lengel, 1986), we hypothesize that FTF groups will initially surpass CS groups in the average value of all three outcome variables: performance, cohesiveness, and synergy (H1, H2 and H3). Nevertheless, we have presented supporting theories (AST and SIP) that indicated that over time, teams are likely to adapt to the use of computer-supported media and become more efficient at using it as they gain familiarity and experience. As

Figure 1. Proposed conceptual model

TPM 17,1/2

88

a result, we predict that the initial superiority in response variables for FTF teams will likely diminish over time. Based on our review of relevant theories and with support from SIP theory and AST, we hypothesize that CS groups will show different over time patterns in the response variables than FTF groups (H4, H5 and H6). Evidence suggests that over time users go through an adaptation to the communication media and, as they become more familiar with each other and comfortable with new media, are able to use it more effectively (Walther, 1992a, b, 1996). Based on the support from these findings, we believe that teams using computer support will adapt to using this medium to communicate with fellow team members, and this will result in a higher rate of increase in the three response variables for CS teams when compared to FTF teams (H4, H5 and H6). That is, we believe that even though CS groups might initially have lower levels of the response variables (due higher richness of FTF communication), they will improve at a higher rate over time as group members adapt to the CS media. Below are the hypotheses in the alternative form: H1. There are differences in overall mean performance between FTF and CS groups. H2. There are differences in overall mean cohesiveness between FTF and CS groups. H3. There are differences in overall mean synergy between FTF and CS groups. H4. Group performance in CS groups will increase at a higher rate than FTF groups. H5. Group cohesiveness in CS groups will increase at a higher rate than FTF groups. H6. Group synergy in CS groups will increase at a higher rate than FTF groups. Research methodology This study was conducted at a large public university in Texas. We used a controlled experiment methodology to isolate the impact of computer support on team outcomes. Sample selection was carefully performed to have a balanced sample across treatment (CS groups) and control (FTF groups) and including all the possible randomizations of the three tasks. The design required a sample multiple of six as there are six possible orderings of the three tasks. This protocol for determining sample size aligns with recommended practices of sample selection in controlled experiments (Montgomery, 2009). Participants We used a total of 120 subjects grouped in 24 teams of five that met three times. A total of 24 five-person teams were recruited to participate with the incentive of a prize for the highest-scoring group. The sample comprised 46 percent females and 54 percent males. Regarding class level, 34.5 percent of the participants were seniors, 31 percent juniors, 15.9 percent sophomores, and 17.7 percent freshmen. Additional background information was collected by means of a questionnaire. Participants were asked to assess their level of experience working in groups using a five-point Likert-type scale

anchored with “not experienced” to “very well experienced”. A total 72 percent of participants rated themselves as experienced or well-experienced working in groups, 16 percent as somewhat experienced, and 11 percent as very well experienced. Post hoc tests indicated no significant differences in demographic variables between the two conditions. Experimental design Each participant was randomly assigned to a five-person team, and each team was randomly assigned to one of two treatments, FTF or CS. Teams remained the same for the duration of the study. Every team went through an identical training session to introduce the communication interface, get group members acquainted with each other, and introduce the decision-making structure. The length of the training session was 20 minutes. After the training, each team went through the three experimental sessions. Each session was conducted two weeks apart, and no formal interactions occurred among team members between sessions. Computer teams used a “text-based” web interface (chat) to communicate from remote stations whereas face-to-face teams met around a table. Note that for the remaining section of this paper the term group and team will be used interchangeable. Each team attended three sessions. During each session, teams solved a randomly assigned task, that is, the order in which the three tasks were solved was random. The tasks portrayed three simulation scenarios (A, B, C). The following section describes the task scenarios. To ensure that differential task complexity did not account for differences in outcomes between FTF and CS teams, we used a balanced experimental design with a number of groups multiple of six to include all six possible orderings of the three tasks (ABC, ACB, BAC, BCA, CAB, CBA). This design provides a balanced representation of all the possible task orderings (randomizations) across experimental conditions for non-identical tasks. Each session consisted of a structured decision-making process including identification of objective, analysis of the situation, generation of alternative solutions, and selection of preferred solution. The process was facilitated using a pretested script identical in both conditions so that the facilitator would not have an impact on the team outcomes. The maximum time available for each session was 90 minutes for a total contact time of 270 minutes for all sessions. Each session was separated by two weeks. The data collection period took place over approximately three months. Decision-making task: survival simulations The three tasks chosen for the sessions portray survival simulation situations (Survival Simulation Series, 2002, 2003a, b). The tasks locate the group in hypothetical situations occurring in isolated areas. The group members must figure out a strategy and plan for survival that makes the best use of the resources available. The simulations occur in three different settings: a yacht that shipwrecks (reef simulation), a fire in the forest (bushfire simulation), and a helicopter crash (cascades simulation). The tasks required participants to make a decision about the best survival strategy and rank a set of 12 items according to their importance for survival. These types of tasks have been extensively used for research purposes as well as for team training (Silberman, 1990; Szumal, 2000).

Performance patterns in teams

89

TPM 17,1/2

90

Sessions protocol Both FTF and CS sessions followed identical protocols. A trained facilitator with extensive experience in team facilitation conducted all the meetings using a scripted protocol. The protocol guided all teams through a rational decision-making process consisting of analysis of the situation, identification of alternatives, evaluation of alternatives, and selection of preferred solution. Prior to the first session, teams received a training session addressing the following areas: ground rules and structure of the decision-making process, getting group members acquainted with each other, and introducing the communication media. During the training session, participants also filled out a questionnaire to collect their demographic information as well as data on their attitudes and experience with group decision making. During each of the three sessions, the groups learned about the simulation situation, and each individual was asked to make a decision about the best strategy to survive in the proposed scenario. Based on their strategy, they had to decide the relative importance of a set of 12 items in relation to their usefulness in surviving the described scenario. As a result, each participant created a ranked list of the 12 items. Once they had finished, they met as a group. During the group process, groups followed a structured decision-making process through the following phases: analyzing the situation, setting objectives, considering alternatives, analyzing consequences, and deciding on a final solution. Next, participants received feedback about their strategy selection during the decision-making process and how it compared to the experts’ suggested strategy. The specific stages used during the decision-making process are shown in Figure 2. The previous section described the general procedure followed in all sessions. The only difference between FTF and CS sessions was that FTF groups sat around a U-shaped table while CS groups sat at their individual workstations and communicated using a chat room. Relevant variables Two categories of variables will be described: independent and response variables. Since the main purpose of the study was to investigate group performance, the unit of analysis was the group. Table I summarizes the variables considered in the design.

Figure 2. Sequence of the decision-making process within each session

Communication platform, session number, and task scenario are the independent variables whereas group performance, cohesiveness, and synergy are the response variables. Type of task: decision-making tasks were selected based on the classification created by McGrath (1984). Decision-making tasks are defined as tasks that require reaching consensus on a preferred answer. The selected tasks can be scored against a recommended expert solution. In order to use comparable and consistent sessions, three tasks from the same simulation series were selected (Survival Simulation Series, 2002, 2003a, b). The tasks have very similar structure; however, they take place under different circumstances. They have been described as “content-free” situations that are likely to be outside the sphere of expertise of the group but that are designed to attract the attention to overall team problem-solving processes (Potter and Balthazard, 2002). Next we present the main response variables investigated in this study along with their operational definitions: . Individual performance was defined as the quality of the decision made by an individual in the group according to the expert criteria. Note that individuals decide their solution to the task before they meet as a group to decide on the group solution. The tasks chosen for the experiment have a pre-determined expert rating criteria based on the ranking of a list of items that assigns a score to any decision made (Survival Simulation Series, 2003a). Individual performance was measured using the same scale, ranging from 0 to 72. Lower values of the score corresponded to better individual performance. . Group performance was defined as the quality of the final decision made by the group according to expert criteria. Group performance was measured by calculating the difference between the group score at the end of the meeting and the expert score. The larger the score, the further it is from the expert ranking, thus reflecting the quality of the final decision. The range of values for group performance also ranged from 0 (perfect score) to 72 (worst score). . Cohesiveness is a widely used measure of group interaction. It has been defined as the degree of closeness that group members feel to one another and their attraction to the group (Miller, 1964). Although this measurement of cohesiveness is based on perceptions, it is not expected to reduce validity of the measure since this study is primarily focused on the actual feeling of each individual member towards others within the same group. A set of five items in the post-session questionnaire was used to assess perceived group cohesiveness (see the Appendix for items). These items, adapted from Seashore’s Index (Miller, 1964), measured perception of the group cohesiveness after each session. The instrument, originally created to be used in work teams, was later adapted and validated to be used by student groups resulting in high levels of reliability (Chidambaram et al., 1990). The scores from the five items were summed for each Independent variables

Response variables

Communication platform Session number Task scenario

Objective group performance Cohesiveness Group synergy

Performance patterns in teams

91

Table I. Main variables in the design

TPM 17,1/2

92

.

individual, resulting in the cohesiveness index for each subject. Finally, the cohesiveness index for the group was calculated as the average value of the individual average scores. The index ranges from 5 to 25, with the lower scores corresponding to more cohesive groups. Post-hoc analysis revealed that the instrument had a reliability estimate of a ¼ 0:84. We also conducted an exploratory factor analysis using Varimax rotation to evaluate the underlying factor structure in order to determine if in fact the five items were part of the same cohesiveness construct. The analysis was conducted using SPSS 16. Both the scree plot (see Figure 3) and the results of the analysis confirm the existence of only one factor that incorporates all five items. The total variance explained by the five factors was 61 percent. These results help us support the construct validity of the cohesiveness measure. Group synergy was defined as the potential ability of groups to perform better as a result of being part of a group as compared to average individual performance. For the purpose of this study, the term synergy refers to the improvement on the quality of the decision due to interaction between individuals. Synergy was investigated as the difference between the average individual score and the group score (Survival Simulation Series, 2003b).

Group size was held constant at five members per team along with facilitator effects that were also controlled. All sessions had a facilitator who assisted with the experiment but did not influence the content of the discussions. In both settings, the facilitator followed a pre-tested script. Post-hoc analysis of individual responses revealed no significant differences in facilitator role or impact between groups in the two treatment conditions. Data analysis procedure This study sought to investigate the differences in performance, synergy, and cohesiveness for groups that met repeatedly over time. The analysis was twofold: assessing the between-group effect by comparing average values between FTF and CS

Figure 3. Scree plot

and the within-group effect by comparing patterns over the three sessions for both media. The type of analysis chosen was based on the characteristics of the data collected. Data points in the response variables come from three observations, one per session, for each of the 24 groups. The analysis was conducted using the PROC MIXED procedure for repeated measures from SAS/STAT statistical package. This procedure provides a very flexible modeling environment for handling repeated measures problems that include a hierarchical structure such as the one in this study in which individuals are nested in groups as well as modeling of repeated measures type of data (Singer, 1998). Mixed-effects model is a term used for statistical models with fixed (e.g. treatment) and random effects, covariance pattern models, and combinations of them. Numerous authors have indicated mixed-effects approach has important advantages over traditional methods of repeated-measures analysis (Littell et al., 1987; Guerin and Stroup, 2000). Mixed models have several benefits including using all available data on each subject, reducing the impact from missing data and increased flexibility in modeling. This type of analysis allows for modeling data at the team and individual level in the same analysis, situation that is often relevant in team-level research. Data in this study come from individuals that interact with a team over a period of time, so data from individuals within the same team are likely to be correlated (such that students from the same team are more likely to show similarity in some variables). With these types of data, classical methods such as OLS regression would not produce correct standard errors. Hierarchical linear modeling (HLM) has been suggested as a better approach as it takes the issue of correlated errors into consideration and provides more realistic and conservative statistical testing. The PROC MIXED procedure in SAS/STAT provides an approach to modeling repeated measures data that combine both ANOVA and HLM (Wallace and Green, 2002). Before applying PROC MIXED, the normality assumption was evaluated using QQ plots, and data suggested the assumption was reasonably met. The analytical methods focused on comparing changes over time in both platforms (FTF vs CS). Three response variables were investigated: performance, cohesiveness, and synergy. We observed and analyzed the patterns in the values of the response variables over three sessions, under two different platforms (CS and FTF).

Results A summary of the results of the hypotheses tests are shown in Table II. Three separate analyses using PROC MIXED were conducted for each of the dependent variables. Table III shows the overall means and standard deviation of the response variables.

Hypothesis 1 Results of the analysis revealed that the main effect of platform on average performance was not significant (F 1;22 ¼ 0:95, p ¼ 0:3397). In other words, data are consistent with the hypothesis stating no overall difference in average performance between FTF and CS teams.

Performance patterns in teams

93

TPM 17,1/2 H1 H2

94

H3 H4 H5

Table II. Summary of hypotheses tests

Table III. Overall means and standard deviations per experimental condition

H6

Alternative hypothesis

Result

There are differences in average performance between FTF and CS groups There are differences in average cohesiveness between FTF and CS groups There are differences in average synergy between FTF and CS groups Group performance in CS groups will increase at a faster rate than FTF groups Group cohesiveness in CS groups will increase at a faster rate than FTF groups Group synergy in CS groups will increase at a faster rate than FTF groups

Not supported

Communication medium

Performancea Mean Std error

FTF CS

32.33 34.67

3.683 4.788

Supported Not supported Not supported Supported Synergy: not supported

Cohesivenessb Mean Std error 9.8625 11.0708

0.33935 0.34687

Mean 2.50 0.50

Synergy Std error 3.636 3.230

Notes: a Performance scores ranged from 0 to 72 with 0 being the best and 72 the worst possible score; b Cohesiveness scores range from 5 to 25 with 5 being the highest and 25 the lowest

Hypothesis 2 Results of the analysis revealed that the main effect of platform on cohesiveness was significant (F 1;22 ¼ 8:59, p ¼ 0:008). That is, CS teams reported significantly lower levels of cohesiveness than FTF teams. Average levels of cohesiveness for each communication media are shown in Table III. Hypothesis 3 Results of the analysis revealed that the main effect of platform on synergy was not significant (F 1;22 ¼ 0:05, p ¼ 0:819). In other words, we found no differences in average synergy between FTF and CS teams. Hypothesis 4 The difference on performance patterns between both FTF and CS is a key element of this study. In particular, such patterns will reveal information concerning groups’ use of the communication platform and how performance changes over time in both platforms. For this purpose, the effect of the interaction term platform*session on performance was investigated. The interaction was not significant (F 2;36 ¼ 1:24, p ¼ 0:3), suggesting that the alternative hypothesis H4 was not supported. Results indicate both platforms show similar performance profiles. Hypothesis 5 This hypothesis evaluated the cohesiveness profile over the three sessions. Results indicate a significant interaction between platform and session in the cohesiveness

variable (F 2;40 ¼ 3:39, p ¼ 0:0431). In other words, there were significant differences in the cohesiveness profiles of FTF and CS groups. Differences in profiles were further evaluated using the output of the LSMEANS statement in SAS/STAT. The analysis provided the mean cohesiveness estimates shown in Figure 4. Multiple comparisons of the different levels were conducted to identify where the significant differences existed. Results indicated that CS groups showed a higher increase in cohesiveness than FTF groups from session 2 to session 3 (p ¼ 0:01). The increase is represented in Figure 4 by the slope between the last two points in the FTF and CS cohesiveness lines.

Performance patterns in teams

95

Hypothesis 6 This hypothesis investigated group synergy profiles of both platforms over the three sessions. The statistical analysis revealed no significant differences in the platform *session interaction term for the synergy variable (F 2;40 ¼ 0:54, p ¼ 0:5852). In other words, synergy profiles did not significantly differ between FTF and CS groups. Discussion and implications Results from this study add to the body of knowledge on computer-supported teams through an exploration on how response variables change over time in FTF and CS. The findings revealed no significant differences in overall performance between both platforms and no differences in performance patterns over time. That is, CS groups performed at similar levels than FTF groups. Synergy changes over time did not differ across platforms indicating that CS groups were as effective building synergy as FTF groups. This is a significant result in itself because virtual teams facilitate savings in travel and accommodation for organizations, provide flexibility, and allow for a less limited pool of participants. Results also indicate that CS groups showed significantly lower levels of cohesiveness than FTF groups. Prior research suggests that high levels of cohesiveness can reduce communication barriers and be instrumental in promoting collaboration (Powell et al., 2004). This result can have implications for team leaders and virtual team members. Training methods focused on increasing cohesiveness levels in virtual teams during the team formation process might help enhance team communication and performance. Results also suggest a faster increase in cohesiveness in CS groups between session 2 and 3, which could indicate that CS

Figure 4. Cohesiveness profiles from the interaction term platform*session

TPM 17,1/2

96

teams might become more effective at establishing relationships as time passes. This result is consistent with social information processing theory, which suggests that the impact of media on group outcomes will likely evolve over time, allowing for social processes and group outcomes to be enhanced as users gain experience with the communication media and their features (Walther, 1992a, b, 1996). These data support the argument that the impact of computer support on team outcomes is not one of deterministic nature that consistently leads to lower level outcomes when compared to face-to-face teams. Conclusions In conclusion, the results obtained allow us to draw reasonable conclusions about the impact of computer support in group performance, synergy, and cohesiveness. To do so required an understanding of CS and FTF by comparing patterns over time in the response variables. Results suggest that the use of computer media to communicate did not negatively impact overall group performance for the type of tasks used in this study. However, we did find that FTF groups reported higher levels of cohesiveness than CS groups. Differences in media richness between FTF and CS communication could have contributed to those differences as participants reported feeling less attached to team members when their interaction was via computer. However, we also found that from session 2 to 3 cohesiveness increased at a higher rate in CS than in FTF teams. This significant increase in cohesiveness over time suggests that teams might be able to overcome some of the barriers of CS communication. This result is consistent with theory of adaptive structuration, which suggests that the users of computer media will adapt the medium to meet their relational needs (Walther, 1992a, b, Chidambaram, 1996). The particular characteristics of the population used may limit the potential for generalization of our findings. Participants belong to a generation characterized by their familiarity with technology and extensive experience communicating and establishing relationships via chat and other electronic communication methods. We expect that this familiarity with the platform will likely differ by generation, but results remain relevant to understand the future workforce and how different generations might interact and communicate. One limitation of this study is a side effect of its longitudinal nature. Studies using repeated measures designs are subject to potential undetected interactions between subjects outside the laboratory setting. However, subjects were asked not to discuss their experiences during sessions with other subjects, did not know each other ahead of time, and had no other formal opportunities to interact as a group. One factor that may limit the external validity of the results relates to team membership stability. Based on membership stability, teams can be classified as ad hoc (purposively assembled to participate in a project) or intact (members with a shared history of working together within an organization) (Salas et al., 2008). The objective of this study was to isolate the impact of communication technology in the outcome variables. A controlled experimental setting facilitated the accomplishment of that goal. However, the teams used in this study differ from traditional teams in that the later ones have a shared history as a result of having worked together in the past. This shared history has the potential to influence future team performance. Intact teams are more traditionally used in field study research and usually result in higher

external validity. However, the large number of confounding variables usually present in field research reduces internal validity. As with most experimental designs, this study resulted in higher internal validity than a field study to the expense of decreased external validity. Some general guidelines to increase the external validity of research using ad hoc groups were followed (Walther, 1996). First, group members were given a training session to help establish relationships before the sessions. The training also familiarized teams with the communication media and general information on their role as team members. They were told that they would meet in their intact groups together over several sessions. Second, following McGrath (1984), “concocted” alliances recommendations, teams were presented with a real incentive tied to the outcome of their task accomplishment. Accordingly, participants were informed that teams with higher levels of performance would be given a monetary award based on the quality of the decisions reached by their team. While the use of student subjects offers questionable generalization to professional teams, this study was intended as a longitudinal experiment to complement previous studies while contributing to our understanding of team decision-making (see also McGrath, 1984). We believed this study addressed gaps in the literature that ignored the longitudinal effect of communication media on team outcomes. By controlling for confounding factors, we were able to isolate the impact of media to better understand how teams adapt to the technology and how team performance changes over time as a result of that adaptation. Future research Results of the current study suggest the need to further investigate how both platforms compare when teams are engaged in more complex tasks or for longer periods of time. In addition, an important question that remains is how individual perceptions about the communication platform and about the other group members change over time and what factors might trigger those changes. We also suggest further studies exploring the dynamics of team performance using intact teams to complement the results obtained in the present study. It is known that the current generation of students, also known as the millennial generation, is more familiar with computer communication than previous generations. Future research can investigate the generational differences and their impact on virtual team performance and adoption of information technologies. References Aiello, J.R. and Kolb, K.J. (1995), “Electronic performance monitoring and social context: impact on productivity and stress”, Journal of Applied Psychology, Vol. 80, pp. 339-53. Alge, B.J., Wiethoff, C. and Klein, H.J. (2003), “When does medium matter? Knowledge-building experiences and opportunities in decision making teams”, Organizational Behavior and Human Decision Processes, Vol. 91, pp. 26-37. Arrow, H. and McGrath, J.A. (1993), “Membership matters: how membership change and continuity affect small group structure, process and performance”, Small Group Research, Vol. 24 No. 3, pp. 334-61.

Performance patterns in teams

97

TPM 17,1/2

Baltes, B.B., Dickson, M.W., Sherman, M.P., Bauer, C.C. and LaGanke, J.S. (2001), “Computer-mediated communication and groups decision making: a meta-analysis”, Organizational Behavior and Human Decision Processes, Vol. 87 No. 1, pp. 156-79. Bell, B.S. and Kozlowsky, S.W. (2002), “A typology of virtual teams: implications for effective leadership”, Group and Organization Management, Vol. 27, pp. 14-49.

98

Burke, K. and Chidambaram, L. (1995), “Developmental differences between distributed and face-to-face groups in electronically supported meeting environments: an exploratory investigation”, Group Decision and Negotiation, Vol. 4, pp. 213-33. Burke, K., Aytes, K., Chidambaram, L. and Johnson, J. (1999), “A study of partially distributed work groups: the impact of media, location, and time on perceptions and performance”, Small Group Research, Vol. 30 No. 4, pp. 453-90. Chidambaram, L. (1996), “Relational development in computer supported groups”, MIS Quarterly, Vol. 20 No. 2, pp. 143-65. Chidambaram, L., Bostrom, R.P. and Wynne, B.E. (1990), “A longitudinal study of the impact of group decision support systems on group development”, Journal of Management Information Systems, Vol. 7 No. 3, pp. 7-25. Daft, R.L. and Lengel, R.H. (1986), “Organizational information requirements, media richness and structural design”, Management Science, Vol. 32 No. 5, pp. 554-71. Daft, R.L., Lengel, R.H. and Trevino, L.K. (1987), “Message equivoquality, media selection, and manager performance: implications for information systems”, MIS Quarterly, Vol. 11 No. 3, pp. 355-66. Davidow, W.H. and Malone, M.S. (1992), The Virtual Corporation, HarperCollins, New York, NY. Festinger, L. (1950), “Informal social communication”, Psychological Review, Vol. 57, pp. 271-82. Gonzalez, M.G., Burke, M.J., Santuzzi, A.M. and Bradley, J.C. (2003), “The impact of group process variables on the effectiveness of distance collaboration groups”, Computers in Human Behavior, Vol. 19, pp. 629-48. Guerin, L. and Stroup, W.W. (2000), “A simulation study to evaluate PROC MIXED analysis of repeated measures data”, Proceedings of the 12th Kansas State University Conference on Applied Statistics in Agriculture, April 30-May 2, 2000, Manhattan, KS. Hackman, J.R. and Morris, C.G. (1975), “Group tasks, group interaction process, and group performance effectiveness: a review and proposed integration”, in Berkowitz, L. (Ed.), Advances in Experimental Social Psychology, Vol. 8, Academic Press, New York, NY, pp. 45-99. Jarvenpaa, S.L. and Ives, B. (1994), “The global network organization of the future: information management opportunities and challenges”, Journal of Management Information Systems, Vol. 10 No. 4, pp. 25-57. Kelly, J.R. and Karau, S.J. (1999), “Group decision making: the effects of initial preferences and time pressure”, Personality and Social Psychology Bulletin, Vol. 25, pp. 1342-54. Lea, M. and Spears, R. (1991), “Computer-mediated communication, de-individuation and group decision making”, International Journal of Man-machine Studies, Vol. 34 No. 2, pp. 283-301. Lee, A.S. (1994), “Electronic mail as a medium for rich communication: an empirical investigation using Hermeneutic interpretation”, MIS Quarterly, Vol. 18 No. 2, pp. 143-57. Littell, R.C., Milliken, G.A., Stroup, W.W. and Wolfinger, R.D. (1987), SAS System for Mixed Models, SAS Institute, Cary, NC.

McGrath, J.E. (1984), Groups: Interaction and Performance, Prentice-Hall, Englewood Cliffs, NJ. McGrath, J.E. and Arrow, H. (2000), “The study of groups: past, present, and future”, Personality and Social Psychology Review, Vol. 4 No. 1, pp. 95-105.

Performance patterns in teams

McGrath, J.E. and O’Connor, K.M. (1996), “Temporal issues in work groups”, in West, M.A. (Ed.), Handbook of Work Group Psychology, John Wiley & Sons, Chichester, pp. 25-52. Marks, M.A., Mathieu, J.E. and Zaccaro, S.J. (2001), “A temporally based framework and taxonomy of team processes”, Management Review, Vol. 26 No. 3, pp. 356-76. Markus, M.L. (1994), “Electronic mail as the medium of managerial choice”, Organization Science, Vol. 5 No. 4, pp. 502-27. Martins, L.L., Gilson, L.L. and Maynard, M.T. (2004), “Virtual teams: what do we know and where do we go from here?”, Journal of Management, Vol. 30 No. 6, pp. 805-36. Miller, D.C. (1964), Handbook of Research Design and Social Measurement, Davic Mckay Company, New York, NY. Montgomery, D.C. (2009), Design and Analysis of Experiments, 7th ed., Wiley, New York, NY. Mullen, B. and Copper, C. (1994), “The relation between group cohesiveness and performance: an integration”, Psychological Bulletin, Vol. 115 No. 2, pp. 210-27. Poole, M.S. and DeSanctis, G. (1990), “Understanding the use of group decision support systems”, in Steinfield, C. and Fulk, J. (Eds), Organizations and Communication Technology, Sage, Newbury Park, CA. Potter, R.E. and Balthazard, P.A. (2002), “Understanding human interaction and performance in the virtual team”, The Journal of Information Technology Theory and Application, Vol. 4 No. 1, pp. 1-25. Powell, A., Piccoli, G. and Ives, B. (2004), “Virtual teams: a review of current literature and directions for future research”, Database for Advances in Information Systems, Vol. 35 No. 1, pp. 6-36. Salas, E., DiazGranados, D., Klein, C., Burke, C.S., Stagl, K.C., Goodwin, G.F. and Halpin, S.M. (2008), “Does team training improve team performance?”, Human Factors, Vol. 50, pp. 903-33. Silberman, M.L. (1990), Active Training: A Handbook of Techniques, Designs, Case Examples, and Tips, McGraw Hill, New York, NY. Singer, J.D. (1998), “Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models”, Journal of Educational and Behavioral Statistics, Vol. 23, pp. 323-55. Sproull, L.S. and Kiesler, S.B. (1986), “Reducing social context cues: electronic mail in organizational communication”, Management Science, Vol. 32 No. 11, pp. 1492-512. Straus, S.G. (1997), “Technology, group process, and group outcomes: testing the connections in computer-mediated and face-to-face communication”, Human-computer Interaction, Vol. 12, pp. 227-66. Survival Simulation Series (2002), Cascades Survival Simulation, Human Synergistics, Douglas. Survival Simulation Series (2003a), Bushfire Survival Simulation, Human Synergistics, Douglas. Survival Simulation Series (2003b), Reef Survival Simulation, Human Synergistics, Douglas. Szumal, J.L. (2000), “How to use group problem solving simulations to improve teamwork”, in Silberman, M. and Philips, P. (Eds), The 2000 Team and Organization Development Sourcebook, McGraw Hill, New York, NY.

99

TPM 17,1/2

100

Wallace, D. and Green, S.B. (2002), “Analysis of repeated measures designs with linear mixed models”, in Moskowitz, D.S. and Hershberger, S.L. (Eds), Modeling Intraindividual Variability with Repeated Measures Data, Erlbaum, Mahwah, NJ, pp. 103-34. Walther, J.B. (1992a), “Interpersonal effects in computer-mediated interaction: a relational perspective”, Communications Research, Vol. 19 No. 1, pp. 52-90. Walther, J.B. (1992b), “A longitudinal experiment on relational tone in computer-mediated and face-to-face interaction”, paper presented at the Hawaii International Conference on System Sciences, Los Alamitos, CA. Walther, J.B. (1996), “Computer-mediated communication: impersonal, interpersonal, and hyperpersonal interaction”, Communications Research, Vol. 23 No. 1, pp. 3-43. Warkentin, M.E., Sayeed, L. and Hightower, R. (1997), “Virtual teams versus face-to-face teams: an exploratory study of a web-based conference system”, Decision Sciences, Vol. 28 No. 4, pp. 975-96. Zaccaro, S.J. and Lowe, C.A. (1988), “Cohesiveness and performance on an additive task: evidence for multidimensionality”, The Journal of Social Psychology, Vol. 128 No. 4, pp. 547-58. Further reading Burke, K., Aytes, K. and Chidambaram, L. (2001), “Media effects on the development of cohesion and process satisfaction in computer-supported workgroups”, Information Technology and People, Vol. 14, pp. 122-41. McGrath, J.E. (1991), “Time, interaction and performance (TIP): a theory of groups”, Small Group Research, Vol. 22, pp. 147-74. Pazos, P., Beruvides, M., Jian, J., Canto, A., Sandoval, A. and Taraban, R. (2007), “Structuring group decision making in a web-based environment by using the nominal group technique”, Computers and Industrial Engineering, Vol. 52 No. 2, pp. 277-95. Seashore, S.E. (1954), Group Cohesiveness in the Industrial Workplace, University of Michigan, Ann Arbor, MI. Zack, M.H. and McKenney, J.L. (1997), “Social context and interaction in ongoing computer-supported management groups”, Organization Science, Vol. 6 No. 4, pp. 394-422.

Appendix

Performance patterns in teams

101

Figure A1. Questionnaire items assessing cohesiveness

About the authors Pilar Pazos is an Assistant Professor in the Department of Engineering Management and Systems Engineering at Old Dominion University. Before joining Old Dominion she worked in the areas of quality control, team learning and consulting. Most recently, she was a Research Associate at Northwestern University with a joint position for the VaNTH Engineering Research Center and the Searle Center for Teaching Excellence. Her research interests include: knowledge management, organizational learning, collaborative learning, group decision making and performance, virtual teams and team dynamics. Pilar Pazos holds a BSc in Industrial Engineering from the University of Vigo, Spain, an MS in Systems and Engineering Management from Texas Tech University and a PhD from Texas Tech University (2005) in Industrial Engineering with a focus on engineering management and a minor in Applied Statistics from the Rawls College of Business. Pilar Pazos is the corresponding author and can be contacted at: [email protected] Mario G. Beruvides is AT&T Professor of Industrial Engineering and Director of the Laboratory for Systems Solutions at the Department of Industrial Engineering in Texas Tech University. He is a Registered Professional Engineer from the state of Texas. Mario G. Beruvides holds a BS in Mechanical Engineering and an MS in Industrial Engineering from the University of Miami and a PhD in Industrial Engineering from Virginia Polytechnic Institute.

To purchase reprints of this article please e-mail: [email protected] Or visit our web site for further details: www.emeraldinsight.com/reprints