Improving Cybersecurity Incident Response Team ...

SECURITY& PRIVACY

IEEE

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page

M q M q

M q

MqM q THE WORLD’S NEWSSTAND®

MULTIDISCIPLINARY SECURITY

Improving Cybersecurity Incident Response Team Effectiveness Using Teams-Based Research

Julie Steinke, Balca Bolunmez, Laura Fletcher, Vicki Wang, Alan J. Tomassetti, Kristin M. Repchick, Stephen J. Zaccaro, Reeshad S. Dalal, and Lois E. Tetrick | George Mason University

Examining other emergency response teams’ methods of adaptation, communication, problem solving, trust building, and developing shared knowledge among team members can improve cybersecurity incident response teams’ performance.

M

ore often than not, cybersecurity incident response involves collaboration among multiple analysts. Cybersecurity incident response teams (CSIRTs) exist in a dynamic and constantly changing environment in which they must effectively engage in information management and problem solving while adapting to complex circumstances.1 For example, multiple analysts might collaborate to determine the extent to which an attack has penetrated a system, while others might coordinate announcements and alerts to additional teams in the organization or even outside agencies. A critical factor in CSIRT members’ success is their ability to function well within a team. Although many people believe their effectiveness depends largely on technological aspects (for example, system infrastructure and cyberanalysts’ technical skills), organizations and researchers are also incorporating behavioral research on individual and team characteristics that drive successful team performance.1,2 Several characteristics define CSIRTs’ core operating and performance environments.1 CSIRTs must

20

July/August 2015

SECURITY& PRIVACY

IEEE

effectively manage information to operate in complex environments and must adapt to changing performance requirements while still monitoring their networks and responding to threats. Other types of response teams share several of these characteristics despite operating in distinctly different domains. In this article, we examine three types of response teams and consider how their strategies can benefit CSIRTs.

Team Deﬁnitions and Characteristics CSIRTs are composed of two or more individuals who interact with one another as well as with IT infrastructure, other IT personnel, end users, management, and other CSIRTs to prepare for and respond to computer security incidents.3 Most notably, CSIRTs are responsible for protecting networks from threats (for example, exfiltration of data and denial-of-service attacks) and maintaining a secure operating environment for a defined constituency (for example, a firm or a nation). The typical CSIRT operating environment is complex—high in information load, information

Copublished by the IEEE Computer and Reliability Societies

1540-7993/15/$31.00 © 2015 IEEE


M q M q

M q


SECURITY& PRIVACY

IEEE


THE WORLD’S NEWSSTAND®

diversity, and degree of uncertainty (for example, the rate of information change).4 Further, the structure of the environment in which CSIRTs function can change quickly and continually, requiring analysts to collaborate adaptively as they monitor networks and respond to threats. Teams that function in complex environments face unique circumstances that potentially influence their effectiveness. Behavioral scientists, such as organizational psychologists, study these teams’ characteristics and processes to develop strategies for improving team functioning in such environments. However, CSIRTs remain relatively unexamined and could benefit from developmental strategies known to enhance the effectiveness of other response teams. Based on overlapping team characteristics (for example, effectively managing information, problem solving, and functioning adaptively in complex environments) and the convergence of physical and cybersecurity components, we identified three teams that are conceptually similar to CSIRTs: emergency medical systems (EMS), military response (MR), and nuclear power plant operating (NPPO) teams. All these teams work in complex, time-constrained environments in which failure to adapt to the changing environment can result in catastrophic performance failures.

Emergency Medical Systems Teams EMS teams engage in services ranging from advising patients on emergency phone lines to providing acute care to patients in emergency departments.5 These team members are involved in prehospital emergency response and include emergency dispatchers, paramedics, and ambulance drivers as well as hospital-based emergency responders, such as nurses and physicians. EMS team members must adapt to patients’ changing health needs, often under high time constraints, in incidents ranging from small-scale situations, such as car accidents, to highly complex mass casualties, for instance, resulting from natural disasters. For effective teamwork to exist in EMS teams, team members must collaborate effectively to handle coordination, risk communication, initial patient care, patient handoff, and treatment documentation.5

Military Response Teams MR teams include rapid-response teams often specializing in particular events such as medical outbreaks, natural disasters, and other events requiring a timely response to a developing situation as well as incident response teams; fleet antiterrorism security teams; marine expeditionary units; and navy sea, air, and land teams. MR teams need to effectively communicate with internal and external agencies, adapt to changing missions, manage large amounts of information, and meet time-sensitive

Adaptation

Shared knowledge of expertise

Communication Team effectiveness

Trust

Collective problem solving

Figure 1. Model of team effectiveness. Five factors that can improve emergency response teams’ success.

demands while responding to complex emergency situations.6 Furthermore, MR team members must develop a shared understanding of the situation, instantly and effectively work together in intricate time-critical events, and engage in continuous learning by performing learned procedures under new conditions.7

Nuclear Power Plant Operating Teams NPPO teams are the “‘brains’ of the complex nuclear systems they manage.”8 They make complex decisions that involve interpreting vast amounts of information while managing numerous systems. In addition, NPPO teams make these decisions under difficult circumstances that often include sensitive time constraints and heavy workloads.8 After abnormalities are detected, NPPO teams determine the cause, take corrective action, and record their actions in operating data logs.9 NPPO team members benefit from skills, such as communication and adaptability, that help them perform these functions effectively.8

Recommendations to Improve CSIRT Effectiveness Figure 1 presents aspects of team functioning that research we describe later has demonstrated to enhance EMS, MR, and NPPO teams’ effectiveness. We focus on five factors: adaptability, problem solving, communication, trust building, and creating shared team knowledge among team members. 21

www.computer.org/security

SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q


M q M q

M q


SECURITY& PRIVACY

IEEE


M q M q

M q



Improve CSIRT Adaptation CSIRTs operate in environments in which their tools and networks are continually changing and increasing in complexity. Moreover, their adversaries are continually learning, growing, and improving. These factors require CSIRTs to change their core performance strategy to protect their networks and constituencies. In other words, CSIRTs must be able to adapt to new and unexpected events. Team-level adaptation entails shifts in team performance that reflect necessary reactions to changing circumstances.10 Teams use information from their environment to adjust their performance strategies, thereby adapting to maintain their ability to perform effectively. Consultants who have worked with NPPOs have stated, “standardized procedures might not be safe in unexpected and unknown situations that have not been anticipated while designing the procedures.”11 This statement applies equally to the CSIRT domain. A strategy that has been shown to enhance team adaptation in military contexts is perturbation training, which aims to improve a team’s ability to adapt by counteracting habituation and procedural rigidity associated with routine team interactions.12 During perturbation training, teams go through realistic, simulated situations that closely represent what they would actually do on the job. In early trials, teams tackle the incident with no interruptions. In later trials, the standard coordination procedures are disrupted—for example, by disabling technology that is critical to mission success—thus forcing teams to coordinate in new ways. In one study, perturbation training enabled teams trained in this method to significantly outperform other teams not similarly trained (by 13 percent) in two of three critical training missions.12 The same tactic of disabling a relied-upon resource could be applied to any CSIRT training involving multiple trial periods to encourage adaptation of procedures that will transfer to novel, real-life situations. When teams are forced to adapt, they will likely experience stress. Stress can often lead to narrowing of team perspectives such that team members under stress place more focus on their individual needs than on team needs.13 Another training protocol that might benefit CSIRTs is one that will prepare individuals to maintain effective performance levels while resolving high-stress, time-critical incidents. One such protocol that has been used with military teams is stress exposure training (SET).14 SET’s primary objectives are to increase teams’ familiarity with environments, teach skills that help individuals successfully maintain task performance, and prepare teams to maintain effective functioning in stressful situations.14 According to research in this area, there are three stages of SET training:14 22

IEEE Security & Privacy

SECURITY& PRIVACY

IEEE

■ providing individuals with information regarding why stress training is important and identifying specific stressors in the environment (for example, recapping past events in which stress significantly affected the outcome), ■ providing individuals with specific cognitive and behavioral skills to help them cope with stressors while maintaining effective performance, and ■ having individuals apply and practice skills under increasingly representative conditions of the actual environment (for example, increasing time pressure over multiple training phases). In one study conducted with military personnel, SET training resulted in decreased perceptions of stress and an 18 percent performance improvement.15 These benefits were retained among study participants who performed novel tasks in the presence of new stressors. CSIRTs could realize similar gains if they utilized SET. Other research on military personnel found that preparatory activities such as contingency planning, “war gaming” (simulations between opposing forces that represent potentially real conflict scenarios), and frame switching during mission planning were instrumental in fostering adaptive mindsets. Thinking through a more complete set of attack strategies can lead to adaptation by fostering knowledge of additional potential attack mitigation solutions. Such thinking can also reflect a level of cognitive flexibility that is necessary for adaptive performance.16 In CSIRTs, this technique is sometimes called penetration testing or red teaming.17 In such exercises, team members evaluate their vulnerabilities by attempting to infiltrate their own system. Despite its potential to increase CSIRT effectiveness, a survey indicated only 17 percent of CSIRTs used penetration testing.18 CSIRTs can utilize information from their own attacks to guide understanding of how they should act to prevent future attacks. Beyond attacking teams’ own systems, red teaming can involve playing devil’s advocate during discussions of approaches to problems, searching for flaws in technology, bringing in others who act out hacking attempts in a safe environment, and anticipating innovative technological advances by adversaries.19 Many CSIRTs use these strategies; however, typical uses address only technical skills and lack a focus on the development of teamwork skills.19

Enhance CSIRT Problem Solving A major part of CSIRTs’ work is problem solving in ambiguous domains where a clear solution isn’t always readily apparent. To help team members interact effectively when solving complex problems, researchers studying other types of teams have investigated several team-training strategies. One such strategy is the “Think July/August 2015


M q M q

M q


SECURITY& PRIVACY

IEEE


M q M q

M q


Like a Commander” tactical adaptive thinking training actually communicated using fewer, less complex intermethod utilized in MR teams.20 This method’s over- actions. There was a 93 percent decrease in team memarching purpose is to increase the number of domain- bers involved in communication patterns and 88 percent specific thought habits team members use automatically fewer actor switches (more actor switches indicate in a crisis.20 Thought habits are cognitive behaviors that two-way exchanges of information whereas fewer actor can be trained to the point of automaticity so that cog- switches indicate more one-way communication).24 nitive resources can be used for more complex probWhen events arise, team members need to be able lems.20 Each of these “habits” guides decision makers’ to effectively communicate all information associated behaviors and should be reviewed with team members with those events throughout the team. In EMS teams, in a group discussion. Doing so increases the amount poor communication during handoffs (for instance, of critical information that team members are capable the transfer of patient care from one EMS team memof pinpointing in tactical ber to another) accounts scenarios.21 for nearly 70 perCSIRTs can bencent of mistakes efit from such trainthat occur.25 On Simulation tasks can be altered to represent ing by making clear the other hand, aspects of uncertainty CSIRT members the informational when handled cormight face during incident response. cues team memrectly, handoffs can bers must recognize improve patient across incidents. safety by facilitatIdentifying and discussing ing information sharing these cues might reduce the likelihood that team mem- and decision making across team personnel shifts.26 In bers miss vital information when making a decision or CSIRTs, many cyber–incident handling errors occur handing off incidents. during handoffs.27 The “Team Coordination Model” (TCM) is another EMS teams have developed several methods to strategy designed to help military teams who work in improve the quality of handoffs. Foremost among these dynamic environments.22 TCM involves identifying a are the use of mnemonics designed to prompt the two defined mission comprising a network of tasks. Individ- parties to exchange all appropriate information clearly ual team members are assigned separate tasks that must and concisely.28 For example, the mnemonics SBAR be coordinated for mission success, thereby developing (situation, background, assessment, and recommenshared mental models across team members—that is, dation) and SHARED (situation, history, assessment, individuals formulate shared understandings about top- risks, events, and documentation) are used in emergency ics such as their own roles and responsibilities within situations due to their effectiveness in conveying relthe team as well as those of other team members.22 evant information.28,29 Research has shown that mediTCM simulations are structured so that team members cal teamwork training that emphasized the use of such learn how, when, and to whom to communicate infor- mnemonics reduced the number of negative patient outmation during decision-making scenarios. These types comes (for instance, medication and transfusion errors) of simulations can provide a way to analyze team col- due to communication problems by 65 percent.30 Other laboration and subsequently team performance.22 This researchers have also found that medical teamwork method could prove useful for CSIRTs because simula- training shortened patient handoff times by 12 percent tion tasks can be altered to represent aspects of uncer- on average while slightly improving patient outcomes.29 tainty CSIRT members might face during incident Several types of handoffs occur in CSIRTs durresponse, thus reflecting their work’s dynamic structure. ing the response process. Specifically, the sending and receiving of information can occur between person and Enhance Communication person, person and technology, person and team, and team and team, creating multiple places for errors and among CSIRT Members To handle events, CSIRT members must communicate mistakes.31 Although there are several descriptions for effectively. Successful communication involves send- how handoffs occur, there are few prescriptions for how ing messages that are accurate, relevant, and timely and to make them effective. As such, checklists created by that target the right persons.23 A study on NPPO teams cybersecurity analysts (ideally from different levels) suggests that it’s not the amount or complexity of infor- and mnemonics adapted to the CSIRT context offer mation but rather the communicated information’s rich- realistic, actionable, and promising options for CSIRTs ness that’s vital in effective crisis management.24 In this to improve the quality of communication during handstudy, high-performing NPPO teams in crisis situations offs. In addition, when preparing for incidents, CSIRTs 23


SECURITY& PRIVACY

IEEE


M q M q

M q


SECURITY& PRIVACY

IEEE


M q M q

M q



can engage in structured team strategy discussions to as they occur, leading to fewer opportunities for team better understand team goals and how to achieve them. members to develop initial trust. Thus, virtual or ad hoc Communication briefings held before teams begin teams might take longer to share information and coopa mission or problem can be instrumental in subse- erate effectively and face the potential for higher conquent team performance. Such briefings can reduce flict; they could benefit from strategies that promote the communication breakdowns by creating shared mental initial development of trust. In medical teams, nurses will seek information or models (shared expectations about what information should be given and received), improving team mem- advice from help providers they perceive as trustworbers’ understanding of upcoming tasks, and enhancing thy. Such trust-based behaviors reduce medical errors, coordination and cohesion.32 As we noted, these brief- and the effects are even stronger under conditions of ings can also foster team adaptation by creating oppor- greater uncertainty.37 Various actions such as buildtunities for teams to discuss ing team members’ reputacontingency plans tions through word of in the case of unexmouth, finding simiShared mental models are important pected events.33 larities with others, to team functioning because they help In medical conacknowledging team texts, one- to fourmembers’ expertise, team members interpret information and minute structured ensuring good perforsituational contingencies in similar ways. briefings involving a mance, and being satchecklist of operaisfied with the group tions and communication as a whole have all been points identified by nurses, surgeons, trainees, and suggested as important factors for developing long-term anesthesiologists reduced communication failures (for trust in military teams.38 Leaders can encourage trust-building behaviors that example, failing to provide, or delaying the provision of, useful information) by 64 percent.34 Such results sug- could eventually develop into team norms. Team trustgest that CSIRTs could benefit from structured brief- based norms help develop a climate of psychological ings that emphasize team communication breakdown safety—the mutually held “belief that the team is safe for interpersonal risk taking.”39 Coaching team leaders points prior to incident response and during handoffs. Military teams that conducted a 10-minute team and minimizing subordinate concerns about power difstrategy meeting prior to a simulated military opera- ferences among medical operating-room teams were tion task demonstrated 33 percent more developed found to significantly increase trust and “speakingshared mental models than teams that simply talked to up” behaviors, which led to improved learning.40 This each other prior to the simulation.7 Further, processes research has shown that in medical settings, leaders among teams involved in strategy meetings (for exam- can downplay power differences by inviting and actple, coordination activities and leadership) improved ing on others’ suggestions, communicating that other 7 percent compared to teams not involved in strategy roles are just as critical to success as the surgeon’s role, meetings.7 Again, methods that have worked well to and encouraging others to be aware of their own limenhance communication among EMS and MR teams its. Indeed, research with medical teams in which memcould benefit CSIRTs. Researchers should examine var- bers were encouraged to speak up about their mistakes ious types of mnemonics, checklists, and briefing styles detected approximately twice as many errors compared to determine those most effective in CSIRT communi- to teams that didn’t exhibit these behaviors.41 Such cation development. activities enabled these team members to learn from each other’s failures and improve their performance, Develop Trust among CSIRT Members which in turn increased psychological safety among Trust among team members—that is, team members’ members and created greater trust in team leaders. belief that they can rely on each other in risky situaMethods to create initial levels of trust among team tions—is one of the most important factors associated members before they begin to work closely with each with team success.35 Trust increases information shar- other can be crucial for information sharing during ing and cooperation and reduces conflict, all of which the initial stages of operations. In pre-mission stages of are important for team effectiveness.36 Both initial lev- military operations, this swift trust was shown to be 7 els of trust and long-term trust in teams are important percent higher when team members had a shared idenfor team effectiveness. However, members of the same tity (for example, belonged to the same regiment).42 team don’t always work in close proximity to each other, Even if teams come together on an ad hoc basis to solve and ad hoc teams are often formed to handle events an incident, trust expectations are important. For ad 24


SECURITY& PRIVACY

IEEE

July/August 2015


M q M q

M q


SECURITY& PRIVACY

IEEE



hoc or newly formed teams, developing a shared identity among team members by acts as simple as creating a team name or motto could promote higher levels of initial trust. Identifying shared associations and experiences (for example, involvement in similar training programs or participation in similar military branches) could help CSIRT members initially rely on one another.

Develop Shared Team Knowledge among CSIRTs CSIRTs—along with EMS, MR, and NPPO teams— must integrate vast amounts of information in their work. Therefore, team members’ ability to effectively manage and share information can greatly influence team effectiveness. Organizational psychologists focus on two forms of knowledge in teams: shared mental models43 and transactive memory—shared knowledge of who knows what.44 Shared mental models are important to team functioning and team member interaction because they help team members interpret and understand information and situational contingencies in similar ways.45 Shared mental models also allow team members to understand how their roles and expertise fit together. With this shared understanding, team members can successfully coordinate actions during complex tasks because each team member knows what behaviors must occur in coordination and synchrony for effective task completion.44 Of course, it’s conceivable that team members share inaccurate mental models.46 Thus, developmental strategies must improve the extent to which accurate mental models are shared. Several such methods to foster effective shared mental models, such as cross-training, have been developed and empirically validated. These methods can be used to increase mental model accuracy—that is, the team members’ understanding of who should do what and when.47 Team members who received cross-training scored 33 percent higher than those who didn’t in their impression of how well they understood other team members’ roles and what was expected of them during tasks.48 Teams already possessing shared mental models (that is, seasoned teams) communicated and performed better over three cross-training sessions than newly formed teams lacking in shared mental models.48 Various cross-training methods can be used. Team members can hear about other members’ specific task responsibilities (for example, through information presentations), watch them do what they do, or actually perform these specific tasks normally completed by other members.47 Some initial short-term costs are associated with cross-training because experienced individuals move into roles in which they’re less

experienced. Nonetheless, over time, cross-training is quite beneficial to teams. Similarly, guided team self-correction training fosters shared mental models.49 This training method entails providing guidance to team members on topics that can help them identify team performance problems.49 In a recent study, prescriptive expert models of teamwork were provided to military teams who were instructed to base their self-critique, feedback, and planning activities on the examples in the provided models. In this study, teams that debriefed using the expert model demonstrated more accurate teamwork mental models, resulting in a 38 percent increase in teamwork processes and a 110 percent increase in performance.49 This approach has been used successfully during combat system teams’ simulation exercises, in nuclear power industry accident investigations, and elsewhere as a tool to enhance onthe-job performance.49 Role identification behaviors include sharing information related to experiences, skills, and abilities with team members. These simple interpersonal interactions help team members gain greater understanding of their teammates’ capabilities and expertise. CSIRTs can easily integrate role identification practices into their training exercises to develop shared knowledge. The Think Like a Commander method has also been shown to aid in the development of shared team knowledge, especially in training situations with ambiguous, incomplete, or inconsistent information.20,50 After-action reviews (AARs) and debriefings are learning opportunities in which discussions are held to review recent performance events.51 Pointing out where communication broke down, where interactions didn’t occur when they should have, and where actions weren’t properly coordinated helps team members build and refine a common understanding of how they need to better interact and coordinate on future problems and missions. To gain such an effect, AARs need to be structured carefully. Several evidence-based best practices for AARs have been identified from research on military teams:52 ■ focus on a few critical performance issues to maximize limited time and emphasize the most important outcomes, ■ shorten the delay between the incident and feedback so participants’ recall of the incident is better, ■ provide both individual and team-oriented feedback so team members and the team as a whole know what to do, ■ emphasize process feedback rather than outcome feedback so team members can focus on improving coordination rather than on merely attaining good outcomes, 25


SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q


M q M q

M q


SECURITY& PRIVACY

IEEE


M q M q

M q



■ ensure that people feel comfortable during the AAR so they don’t hesitate to speak frankly, and ■ take notes on lessons learned to preserve institutional memory. In one study of six military training exercise sessions, a 10-minute AAR that utilized a set of these practices improved performance by 50 percent, efficacy by 22 percent, openness of communication by 11 percent, and cohesion by 15 percent in subsequent exercises.51 A variation of AARs can be used as a training exercise wherein managers have subject matter experts provide their ideal solution to a training scenario before reviewing what the trainees actually did. This method provides an easy way to compare experts’ and novices’ processes. This technique could work in CSIRTs. Together, subject matter experts and junior analysts could review recordings and notes; discuss findings; and identify where inappropriate or unnecessary information was used, how communication can be made more effective, and unnecessary steps in the solution process. In addition to having a shared knowledge of expected interactions and tasks throughout a team, team members must possess knowledge of other team members’ unique areas of expertise. This knowledge makes communication and information sharing faster because it allows team members to know who they can contact for help or who might need additional information during an event.53 For example, when a CSIRT analyst faces an unfamiliar incident, knowing that another team member has previously solved similar incidents is useful information because it facilitates advice seeking and reduces incident resolution time. However, not all areas of expertise might be known because analysts continually develop additional skills to keep up with the everchanging nature of their jobs. CSIRTs can develop shared knowledge of unique expertise among team members by using several simple strategies that have been found to significantly improve team performance in other team types. These strategies work by providing role clarification among team members and educating members on the unique expertise that exists in their teams. For example, crosstraining enhances team members’ knowledge of other team roles, thus providing greater shared role clarification. To use this method in CSIRTs, managers can have team members give presentations to the team, providing information and insights into their unique areas of expertise.54 In addition, analysts who are less experienced in one role can job-shadow experienced team members to learn about other roles and how they solve problems. Knowledge maps that display individual team members’ roles and expertise (for example, team directories) 26


SECURITY& PRIVACY

IEEE

can also enhance shared knowledge and provide a resource for members to use when incident-handling advice is needed. In addition, regular discussions in which team members share topics they’re currently working on can facilitate the development of each person’s knowledge of expertise throughout the team, ultimately allowing for more efficient team activities. These tactics have been used in various team types in addition to EMS, MR, and NPPO teams and have consistently enhanced the development of shared knowledge among team members.

S

ubstantial research identifies drivers of team effectiveness in EMS, MR, and NPPO teams; CSIRTs can use these strategies to improve incident response. However, CSIRT managers should also consider various costs associated with these strategies. For instance, as we mentioned, cross-training involves short-term time (and perhaps performance) costs because in intensive cross-training, experienced individuals move into roles in which they possess less experience and knowledge. Other potential developmental strategies might involve similar costs associated with time or other resources (for example, staff time spent in training and hiring knowledgeable trainers). However, the amount of upfront time invested in these strategies could lower future incident response times by limiting teamworkrelated problems. Thus, we recommend that managers identify the most critical needs for their own CSIRTs and then select the developmental strategies that provide the highest benefit-to-cost ratio based on their unique circumstances. Our recommendations come with some limitations. The ever-changing nature of cybersecurity work means that no two CSIRTs are alike. Thus, strategies that work for one CSIRT might not work as well for others. These differences could be due to many factors including CSIRT size, type (for example, those housed in private, government, or coordinating organizations), team composition, the impact of cultural differences given the increasingly global boundaries of cybersecurity, or even the type of attacks CSIRTs face. In this respect, future research on how these kinds of factors might change the development and implementation of strategies that impact CSIRT effectiveness is necessary. These caveats aside, evidenced-based practices used to enhance teams’ abilities to be adaptive, solve problems, communicate, build trust, and develop shared knowledge among team members have been shown to improve the effectiveness of EMS, MR, and NPPO teams. Such research and the best practices validated by this research offer a starting point for improving CSIRT effectiveness. July/August 2015


M q M q

M q


SECURITY& PRIVACY

IEEE



Acknowledgments This material is based on research sponsored by the US Department of Homeland Security (DHS) Science and Technology Directorate, Homeland Security Advanced Research Projects Agency, Cyber Security Division (DHS S&T/HSARPA/ CSD), BAA 11-02, and the Air Force Research Laboratory Information Directorate under agreement FA8750-12-20258. The US government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation thereon. We acknowledge the contribution of team members on this research effort who are not authors of this article.

References 1. S.J. Zaccaro et al., “A Taxonomic Classification of Cyber Security Incident Response Performance,” Psychosocial Dynamics of Cybersecurity, S.J. Zaccaro et al., eds., Routledge, to be published in 2015. 2. S.L. Pfleeger and D.D. Caputo, “Leveraging Behavioral Science to Mitigate Cyber Security Risk,” Computers and Security, vol. 31, no. 4, 2012, pp. 597–611. 3. T.R. Chen et al., “An Organizational Psychology Perspective to Examining Computer Security Incident Response Teams,” IEEE Security & Privacy, vol. 12, no. 5, 2014, pp. 61–67. 4. H.M. Schroder, M.J. Driver, and S. Streufert, Human Information Processing, Holt, Reinhart & Winston, 1967. 5. D. Schottke, First Responder: Your First Response in Emergency Care, Jones Bartlett, 2007. 6. L.A. Zimmerman et al., “Methods and Tools for Training Crisis Response,” research note 2012-07, US Army Research Inst. Behavioral and Social Sciences, 2012; www.dtic.mil/dtic/tr/fulltext/u2/a564316.pdf. 7. S. Dalenberg, A.L. Vogelaar, and B. Beersma, “The Effect of a Team Strategy Discussion on Military Team Performance,” Military Psychology, vol. 21, no. 2, 2009, pp. S31–S46. 8. M.J. Waller, N. Gupta, and R.C. Giambatista, “Effects of Adaptive Behaviors and Shared Mental Models on Control Crew Performance,” Management Science, vol. 50, no. 11, 2004, p. 1534. 9. “Occupational Outlook Handbook—Power Plant Operators, Distributors, and Dispatchers,” Bureau of Labor Statistics, US Dept. Labor, 2014; www.bls.gov/ooh /production/power-plant-operators-distributors-and ________________________________ -dispatchers.htm. _________ 10. C.S. Burke et al., “Understanding Team Adaptation: A Conceptual Analysis and Model,” J. Applied Psychology, vol. 91, no. 6, 2006, pp. 1189–1207. 11. J. Brüngger et al., “PUMA-Development and Application Tool for Supporting Nuclear Power Plant Operating Teams in Unexpected and Unknown Situations,” 5th Int’l Conf. Applied Human Factors and Ergonomics, 2014, p. 178. 12. J.C. Gorman, N.J. Cooke, and P.G. Amazeen, “Training Adaptive Teams,” J. Human Factors, vol. 52, no. 2, 2010, pp. 1–13.

13. J.E. Driskell, E. Salas, and J. Johnston, “Does Stress Lead to a Loss of Team Perspective?,” Group Dynamics: Theory, Research, and Practice, vol. 3, no. 4, 1999, pp. 291–302. 14. J.E. Driskell, E. Salas, and J. Johnston, “Stress Exposure Training,” Making Decisions under Stress: Implications for Individual and Team Training, J.A. Cannon-Bowers and E. Salas, eds., Am. Psychological Assoc., 1998, pp. 191–217. 15. J.E. Driskell, J.H. Johnston, and E. Salas, “Does Stress Training Generalize to Novel Settings?,” J. Human Factors, vol. 43, no. 1, 2001, pp. 99–110. 16. J.K. Nelson, S.J. Zaccaro, and J.L. Herman, “Strategic Information Provision and Experiential Variety as Tools for Developing Adaptive Leadership Skills,” Consulting Psychology J.: Practice and Research, vol. 62, no. 2, 2010, pp. 131–142. 17. C. Peake, “Red Teaming: The Art of Ethical Hacking,” SANS Inst., 16 July 2003; www.sans.org/reading -room/whitepapers/auditing/red-teaming-art-ethical ________________________________ -hacking-1272. ________ 18. G. Killcrece et al., State of the Practice of Computer Security Incident Response Teams (CSIRTs), tech. report CMU/ SEI-2003-TR-001, Software Eng. Inst., Carnegie Mellon Univ., 2003. 19. D.F. Longbine, “Red Teaming: Past and Present,” School of Advanced Military Studies, Army Command and General Staff College, 2008; www.dtic.mil/cgi-bin/Get TRDoc?AD=ADA485514. ______________ 20. S.B. Shadrick and I.W. Lussier, “Training Complex Cognitive Skills: A Theme-Based Approach to the Development of Battlefield Skills,” Development of Professional Expertise, K.A. Ericsson, ed., Cambridge Univ. Press, 2009, p. 287. 21. J.W. Lussier and S.B. Shadrick, “Adaptive Thinking Training for Tactical Leaders,” Symp. Advanced Technology for Military Training, 2003; www.dtic.mil/get-tr-doc /pdf?AD=ADA428347. _____________ 22. R. Giachetti and J.A. Rojas-Villafane, Simulation Modeling and Statistical Network Tools for Improving Collaboration in Military Logistics, tech. report AFRLRH-WP-TR-2009-0110, Air Force Research Laboratory, 2008; www.dtic.mil/dtic/tr/fulltext/u2/a510072.pdf. 23. T. Rench et al., “Accelerating Unit Adaptability: A Principle-Based Approach to Unit Communication,” Interservice/Industry Training, Simulation, and Education Conf., 2014. 24. A.A. Stachowski, S.A. Kaplan, and M.J. Waller, “The Benefits of Flexible Team Interaction during Crises,” J. Applied Psychology, vol. 94, no. 6, 2009, pp. 1536–1543. 25. K.M. Sutcliffe, E. Lewton, and M.M. Rosenthal, “Communication Failures: An Insidious Contributor to Medical Mishaps,” Academic Medicine, vol. 79, no. 2, 2004, pp. 186–194. 26. K. Wood et al., “Clinical Handovers between Prehospital and Hospital Staff: Literature Review,” Emergency Medical J., 2014; doi:10.1136/emermed-2013-203165. 27


SECURITY& PRIVACY

IEEE

M q M q

M q

MqM q


M q M q

M q


SECURITY& PRIVACY

IEEE


M q M q

M q



27. Growing the Security Analyst: Hiring, Training, and Retention, tech. report 4AA5-3982ENN, Hewlett-Packard Development, Aug. 2014; www8.hp.com/h20195/v2 /getpdf.aspx/4AA5-3982ENN.pdf?ver=1.0. _________________________ 28. D. Siassakos et al., “Clinical Efficiency in a Simulated Emergency and Relationship to Team Behaviours: A Multisite Cross-Sectional Study,” BJOG: Int’l J. Obstetrics & Gynaecology, vol. 118, no. 5, 2011, pp. 596–607. 29. L.A. Riesenberg, J. Leitzsch, and B.W. Little, “Systematic Review of Handoff Mnemonics Literature,” Am. J. Medical Quality, vol. 24, no. 9, 2009, pp. 196–204. 30. S. Deering et al., “On the Front Lines of Patient Safety: Implementation and Evaluation of Team Training in Iraq,” Joint Commission J. Quality and Patient Safety, vol. 37, no. 8, 2011, pp. 350–356. 31. C. Alberts et al., Defining Incident Management Processes for CSIRTs: A Work in Progress, tech. report CMU/SEI2004-TR-015, Software Eng. Inst., Carnegie Mellon Univ., 2004. 32. Y. Xiao, S. Henrickson Parker, and T. Manser, “Teamwork and Collaboration,” Rev. Human Factors and Ergonomics, vol. 8, no. 1, 2013, pp. 55–102. 33. J. Tucker and K.M. Gunther, “The Application of a Model of Adaptive Performance to Army Leader Behaviors,” Military Psychology, vol. 21, no. 3, 2009, pp. 315–333. 34. L. Lingard et al., “Evaluation of a Preoperative Checklist and Team Briefing among Surgeons, Nurses, and Anesthesiologists to Reduce Failures in Communication,” Archives Surgery, vol. 143, no. 1, 2008, pp. 12–17. 35. R.C. Mayer, J.H. Davis, and F.D. Schoorman, “An Integrative Model of Organizational Trust,” Academy Management Rev., vol. 20, no. 3, 1995, pp. 709–734. 36. J.L. Wildman et al., “Trust Development in Swift Starting Action Teams: A Multilevel Framework,” Group Organization Management, vol. 37, no. 2, 2012, pp. 138–170. 37. D.A. Hofmann, Z. Lei, and A.M. Grant, “Seeking Help in the Shadow of Doubt: The Sensemaking Processes Underlying How Nurses Decide Whom to Ask for Advice,” J. Applied Psychology, vol. 94, no. 5, 2009, pp. 1261–1274. 38. A.I. Blair and J.B. Hanna, Trust and Partnering with the Joint Team, tech. report AU/AFF/003/2009-04, Air Command and Staff College, Air Univ. Maxwell Air Force Base, Apr. 2009; www.dtic.mil/dtic/tr/fulltext/u2/a53 9800.pdf. _____ 39. A.C. Edmondson, “Psychological Safety and Learning Behavior in Work Teams,” Administrative Science Q., vol. 44, no. 2, 1999, p. 354. 40. A.C. Edmondson, “Speaking Up in the Operating Room: How Team Leaders Promote Learning in Interdisciplinary Action Teams,” J. Management Studies, vol. 40, no. 6, 2003, pp. 1419–1452. 41. A.C. Edmondson, “Learning from Failure in Health Care: Frequent Opportunities, Pervasive Barriers,” Quality Safety Health Care, vol. 13, no. 2, 2004, pp. ii3–ii9. 28


SECURITY& PRIVACY

IEEE

42. B.D. Adams et al., “Swift Trust in Distributed Ad Hoc Teams,” tech. report CR 2007-139, Defence Research and Development, Toronto, Canada, Oct. 2007; www.dtic .mil/cgi-bin/GetTRDoc?AD=ADA477148. _________________________ 43. J.A. Cannon-Bowers, E. Salas, and S.A. Converse, “Cognitive Psychology and Team Training: Shared Mental Models in Complex Systems,” Human Factors Society Bull., vol. 33, 1990, pp. 1–4. 44. D.M. Wegner, T. Giuliano, and P. Hertel, “Cognitive Interdependence in Close Relationships,” Compatible and Incompatible Relationships, W.J. Ickes, ed., Springer, 1985, pp. 253–276. 45. J. A. Cannon-Bowers, E. Salas, and S. Converse, “Shared Mental Models in Expert Team Decision Making,” Individual and Group Decision Making, N.J. Castellan Jr., ed., Lawrence Erlbaum, 1993. 46. P.J. Ellis and M.J. Pearsall, “Reducing the Negative Effects of Stress in Teams through Cross-Training: A Job Demands-Resources Model,” Group Dynamics: Theory, Research, and Practice, vol. 15, no. 1, 2011, pp. 16–31. 47. J.A. Cannon-Bowers et al., “The Impact of Cross-Training and Workload on Team Functioning: A Replication and Extension of Initial Findings,” J. Human Factors, vol. 40, no. 1, 1998, pp. 92–101. 48. R. Espevik, B.H. Johnsen, and J. Eid, “Outcomes of Shared Mental Models of Team Members in Cross Training and High-Intensity Simulations,” J. Cognitive Eng. Decision Making, vol. 5, no. 4, 2011, pp. 352–377. 49. K.A. Smith-Jentsch et al., “Guided Team Self-Correction: Impacts on Team Mental Models, Processes, and Effectiveness,” Small Group Research, vol. 39, no. 3, 2008, pp. 303–327. 50. R. Woltjer et al., “Role-Playing Exercises to Strengthen the Resilience of Command and Control Systems,” Proc. 13th European Conf. Cognitive Ergonomics, 2006, pp. 71–78. 51. A.J. Villado and W. Arthur Jr., “The Comparative Effect of Subjective and Objective After-Action Reviews on Team Performance on a Complex Task,” J. Applied Psychology, vol. 98, no. 3, 2013, pp. 514–528. 52. E. Salas et al., “Debriefing Medical Teams: 12 EvidenceBased Best Practices and Tips,” Joint Commission J. Quality Patient Safety, vol. 34, no. 9, 2008, pp. 518–527. 53. A.R. Kim et al., “Analysis of Team Communication Characteristics Using SNA Technique,” Trans. Korean Nuclear Society Autumn Meeting, Oct. 2011. 54. E. Blickensderfer, J.A. Cannon-Bowers, and E. Salas, “Cross Training and Team Performance,” Decision Making under Stress: Implications for Training and Simulation, J.A. Cannon-Bowers and E. Salas, eds., Am. Psychological Assoc., 1998, pp. 299-312. Julie Steinke is a postdoctoral research fellow in George

Mason University’s Industrial/Organizational Psychology program. Her research interests include July/August 2015


M q M q

M q


SECURITY& PRIVACY

IEEE


M q M q

M q


teams, competition and conflict, performance under stress and adversity, and resilience. Steinke received a PhD in industrial and organizational psychology from Wright State University. Contact her at [email protected]. ____________

chology program. Her research interests include team processes, CSIRT effectiveness, and multiteam systems. Repchick received an MA in industrial and organizational psychology from George Mason University. Contact her at [email protected]. ____________

Balca Bolunmez is a doctoral student in George Mason

Stephen J. Zaccaro is a professor of industrial and orga-

University’s Industrial/Organizational Psychology program. Her research interests include judgment and decision making, job performance, organizational diversity, and research methods. Bolunmez received an MS in textile engineering from Dokuz Eylul University and an MBA in management of technology from New Jersey Institute of Technology. Contact her at [email protected]. _____________

nizational psychology at George Mason University. His research interests include group dynamics, team performance, multiteam systems, leadership, leader development, and work attitudes. Zaccaro received a PhD in social psychology from the University of Connecticut. Contact him at [email protected]. ____________

Laura Fletcher is a graduate student in George Mason

University’s Industrial/Organizational Psychology program. Her research interests include teams, multiteam systems, networks, and creativity. Contact her at [email protected]. ___________ Vicki Wang is a graduate student in George Mason Uni-

versity’s Industrial/Organizational Psychology program. Contact her at [email protected]. ___________

Reeshad S. Dalal is chair of the Psychology Department

and an associate professor of industrial and organizational psychology at George Mason University. His research interests include employee performance, work motivation, decision making, personality and its interactions with the work situation, job attitudes, and research methods. Dalal received a PhD in industrial and organizational psychology from the University of Illinois at Urbana–Champaign. Contact him at rdalal @gmu.edu. __________ Lois E. Tetrick is a professor in George Mason Univer-

Alan J. Tomassetti is a doctoral candidate in George

Mason University’s Industrial/Organizational Psychology program. His research interests include judgment and decision making, policy capturing, leadership, team effectiveness, and situational strength. Tomassetti received an MA in industrial and organizational psychology from George Mason University. Contact him at ____________ [email protected]. Kristin M. Repchick is a doctoral candidate in George

Mason University’s Industrial/Organizational Psy-

sity’s Industrial/Organizational Psychology program. Her research interests include occupational health and safety, occupational stress, the work–family interface, psychological contracts, social exchange theory and reciprocity, organizational commitment, and organizational change and development. Tetrick received a PhD in industrial and organizational psychology from Georgia Tech. Contact her at ___________ [email protected]. Selected CS articles and columns are also available for free at ___________________ http://ComputingNow.computer.org.

Subscribe today! IEEE Computer Society’s newest magazine tackles the emerging technology of cloud computing.

computer.org/ cloudcomputing _________________

29

www.computer.org/security ____________

SECURITY& PRIVACY

IEEE


M q M q

M q
