on approaches to building theories: process, variance and systems

ON APPROACHES TO BUILDING THEORIES: PROCESS, VARIANCE AND SYSTEMS

Andrew Burton-Jones Management Information Systems Division Sauder School of Business University of British Columbia [email protected] Ephraim R. McLean Department of Computer Information Systems J. Mack Robinson School of Business Georgia State University [email protected] Emmanuel Monod Management Information Systems Paris Dauphine University [email protected]

Working paper, Sauder School of Business, UBC

February, 2011

Acknowledgments We thank Omar El Sawy, Allen Lee, Stefan Lukits, Aazadeh Madani, and Frantz Rowe for their comments on earlier versions of this paper, as well as participants in a research seminar at UBC. Support was provided by the Social Sciences and Humanities Research Council of Canada to the first author.

1

ON APPROACHES TO BUILDING THEORIES: PROCESS, VARIANCE AND SYSTEMS

ABSTRACT There has been growing interest in theory building in information systems. We extend this literature by examining theory building approaches. We define an approach as a researcher’s choice of the types of concepts and relationships used to construct the theory, and we examine three approaches: process, variance, and systems. Although each one has been used in past literature, discussions of them show some confusion. For instance, some researchers suggest that there are only two approaches (process and variance). Others imply that one’s epistemological orientation (such as positivist or interpretive) or goal (such as understanding or prediction) determines one’s approach. Finally, others suggest that theories should be developed using one approach only (such as a pure variance or pure process approach). In contrast to these views, we explain why there is no one-to-one correspondence between one’s approach and one’s methodology, epistemology, or theoretical goal, and we explain why researchers can often benefit from combining approaches. We also suggest different ways of combining approaches and illustrate how they can be used to improve research on information systems success. Overall, our paper contributes by (1) clarifying the approaches that researchers can use to build theory, (2) freeing researchers from strictures that they may perceive when building theories, and (3) illustrating the feasibility of our suggestions for an important research domain.

Keywords: theory, epistemology, process, variance, system, explanation, understanding, prediction, IS success.

2

ON APPROACHES TO BUILDING THEORIES: PROCESS, VARIANCE AND SYSTEMS

INTRODUCTION According to Glaser and Strauss (1967), the highest rewards in science go to those who generate an important theory. Many researchers also consider a paper’s theoretical contribution to be the main measure of its quality (Straub et al. 1994; Daft 1995; Sutton and Staw 1995). It is widely agreed, however, that the information systems (IS) discipline is at an early stage of theory building (Webster and Watson 2002). This is partly a product of our history. In the early years, we were encouraged to adopt theories from other disciplines rather than develop our own (Keen 1980). During the 1990’s, our field placed great emphasis on research methods (Lee 1989; Straub 1989; Klein and Myers 1999; Boudreau et al. 2001), but the tradition of borrowing theories from other fields remained. Only recently has concerted attention been placed on adapting and extending theory (rather than simply borrowing theory) from other fields (Truex et al. 2006) and on building our own theories (Markus and Saunders 2007; Grover et al. 2008).1 To help support theory building efforts, researchers have recently proposed ways to evaluate theory (Weber 2003), described the goals that different theories may have (such as analyzing, explaining, predicting, and prescribing) (Gregor 2006), and outlined the elements required of theories of system design (Gregor and Jones 2007). Our aim is to complement these works by describing the basic building blocks of theory and the approaches that researchers can use when assembling these building blocks to form a theory. We focus on two building blocks – concepts and relationships among concepts – and we focus on three general approaches for 1

In addition to history, there are likely other reasons for the lack of theory in our field, such as the difficulty of theory building or the lack of training on it in doctoral programs.

3

assembling these building blocks to form a theory – process, variance, and systems. Although these approaches have been described in prior research, discussions of each one show a degree of confusion. As a result, researchers may be unaware of the many ways they can go about building theory. In this essay, we (1) clarify the approaches that researchers can use to build theory, (2) explain how these approaches can be used, either alone or in combination, freeing researchers from strictures they may have perceived based on past research, and (3) demonstrate how these ideas can be used to improve theory in an important research domain (information systems success). Overall, our paper is not about building specific theories, but rather about choosing how to build theory. Our message is ultimately pragmatic. Because our discipline studies very complex phenomena, the principle of requisite variety (Ashby 1958) reminds us that we need an equally rich variety of approaches to build theories to account for these phenomena. The intended contribution of our paper lies in helping researchers (and reviewers of research) to understand the range of approaches available and use these approaches astutely.

THEORETICAL APPROACHES IN IS RESEARCH “Theory” is notoriously difficult to define (Freese 1980; Sutton and Staw 1995; Lee 2004). We use the following working definition, which is consistent with previous definitions (Weber 2003 p. iv; Gregor 2006 p. 616): a theory is an account of some empirical phenomenon. Although researchers can construct a theoretical account for different reasons (such as to help them explain, predict, or understand some phenomenon) and from different epistemological persuasions (Orlikowski and Baroudi 1991), all theoretical accounts will consist of at least two elements: “concepts” and “relationships among concepts.” In Table 1, we provide statements from a wide variety of sources that support this view. We recognize that they are not the only elements of theory. For example, to some, complete theories must contain boundaries (Dubin, 4

1978; Weber, 2003), modalities (Kant, 1781; Giddens, 1984), moral context, and voice (Pentland, 1999). Theories must also be expressed in some means of representation (Gregor 2006 p. 620). However, the focus of our paper is on concepts and relationships among concepts.

Table 1: Elements of Theory: Concepts and Relationships Supporting Statements “Any theory has two components: the concepts or categories that the theory employs, and the relationships…among these concepts.” “… ‘theory’ means in all empirical sciences, the explicit formulation of determinate relations between a set of variables.…” “Theory is about the connections among phenomena.” “There are, then, theoretical terms, theoretical laws, and theories; each may be analyzed by reference to the other two.” “Theorizing is how we think about the relationships among the elements in the world that occupy our research attention.” “A theory is a set of statements about the relationship(s) between two or more concepts or constructs.”

Reference Maxwell 1992 p. 291 Schutz 1973 p. 51-52 Sutton and Staw 1995 p. 378 Kaplan 1964/1998 p. 297 Van Maanen et al. 2007 p. 1147 Jaccard and Jacoby 2010 p. 28

We define an approach to building theory as a researcher’s choice of the types of concepts and types of relationships that they use to construct their theory. Consider the technology acceptance model (TAM) (Davis 1989). TAM consists of concepts such as “ease-of-use,” “usefulness,” and “intentions to use an IT” and relationships such as there being a positive effect of ease-of-use on usefulness, and a positive effect of ease of use and usefulness on intentions. At the level of an “approach,” TAM consists of certain types of concepts (properties of things) and certain types of relationships (one-way, seemingly deterministic relations) that some would characterize as a “variance” approach, because variations in the properties is what drives the relationships posed. This paper reviews three approaches that researchers can use: variance, process, and systems. It is possible that other approaches could exist (such as “chaos theory” approaches), but we believe these three can account for most theories that IS researchers construct. Table 2 summarizes each one; more detailed definitions are offered later. 5

Table 2: Three Theoretical Approaches as Espoused in IS Research Approach

Types of Concepts

Types of Relationships

Relevant References

Variance

Properties of entities that have varying values

Variation among the values of properties

Blalock 1969; Dubin 1978; Mohr 1982; Bacharach 1996

Process

Entities that participate in Sequences among events or are affected by events (typically probabilistic)

Mohr 1982; Abell 1987; Abbott 1988; Monge 1990

System

Wholes, parts, and emergent properties

Churchman 1968; Forrester 1968; Checkland 1999

Interactions and parts and reciprocal relationships

To further clarify the focus of our article, Figure 1 depicts a representation of research inspired from classical epistemology. According to Kant (1781), research implies at least: (1) human and social phenomena (such as users and IT systems); (2) perception of these phenomena (such as perceptions of users’ beliefs about IT); and (3) a priori concepts and categories used to construct theory (such as constructs and notions of causality). Since Kant, epistemology has focused on (4) the influence of historical and socio-economical context on researchers (Foucault 1972; Kuhn 1996); (5) researchers’ goals (such as understanding, explanation, and predictions); and (6) forms of theoretical expression. We merely add one’s theoretical approach as an additional element to indicate the types of concepts and relationships that one uses when one perceives and theorizes. One’s theoretical approach does not determine what one chooses to perceive or theorize about (e.g., natural attitudes, rational choices, or political pressures). Rather, it influences how one perceives and theorizes about such phenomena (e.g., as variation among properties, sequences of events, or reciprocal systemic processes). Sometimes one’s approach is so present in one’s mind that it could be called a “worldview.” We use “approach” rather than “worldview,” because “worldview” is used in phenomenology (Heidegger 1953) with various meanings that may be misleading in our context. Moreover, the word “approach” implies a sense of pragmatism that we believe is important. We recognize that for socio-historical and 6

cultural reasons, researchers in a particular community may have common theoretical goals, approaches, and forms of expression (Kuhn, 1996). Nonetheless, researchers should always be willing to choose whatever approach, or combination of approaches, they deem most useful.

Figure 1: Distinguishing an “Approach” from other Aspects of Theory Building

The focus of our article—approaches to building theory—is not something new. After all, any effort to create or extend theory will involve some approach. However, there are few explicit sources describing these approaches. Moreover, as we will show, the few sources that exist are only helpful up to a point. This might not have been so problematic in the past, because IS researchers typically borrowed theory. However, as IS researchers are called more to build theory, they will inevitably face the choice of what approach to use and how to use it. An explicit description of these approaches and how to use them could help these researchers. 7

Some History and Claims about Theoretical Approaches in IS Despite the long history of the systems approach in IS research (Churchman 1968), a review of IS journals may lead one to believe that there are only two theoretical approaches in IS: process and variance. Coined by Mohr (1982), the process/variance distinction has enjoyed a wide uptake in organization science (Pentland 1999; Poole et al. 2000) and IS research (Shaw and Jarvenpaa 1997). Many IS doctoral students are introduced to the distinction via Markus and Robey’s (1988) seminal article. The importance of the distinction is also emphasized in premier journals. For example, Webster’s and Watson’s (2002 p. xix) editorial explains to authors that “conceptual models are generally derived from variance (factor) or process theories….”2 If an approach is defined by the type of concepts and relationships that researchers use when they theorize about some phenomena, then clearly it is something quite fundamental. In fact, identifying the generic types of concepts and relationships that humans use to theorize about the world is an extremely old, and hotly contested topic, spanning the disciplines of linguistics (Lackoff 1987), sociology (Dumont and Wilson 1967; Drysdale 1996), psychology (Bruner 1986; Medin et al. 2000; Markman and Gentner 2001), and ontology (Bunge 1977; Rescher 1996). Researchers should be aware that Mohr (1982) was not drawing on these fields when he coined the process/variance distinction. Nor did he base his distinctions in classical epistemology. Mohr’s emphasis was pragmatic (pp. 4-5): he simply believed that different approaches were suited to different types of research and he wanted to promote awareness of the process approach. Like Mohr, we will not attempt to identify the elemental concepts that drive human thought. We will, however, go a step further than Mohr, and review core elements of the process, variance, and system approaches in light of past and current research. Our underlying 2

Likewise, in organization science, Chiles (2003 p. 288) writes that the process and variance approach are the “two fundamental types of theory in social science research.”

8

objective is to clarify what we perceive to be some confusion regarding these approaches in extant literature. Moreover, we are concerned that many researchers seem to assume that Mohr’s distinctions are grounded in an accepted and agreed-upon philosophical basis. For example, Table 3 includes statements by senior scholars that give great weight to Mohr’s distinctions. As we explain later, we believe there is value in taking a broader perspective on these issues.

Table 3: Strong Claims about the Process/Variance Distinction in IS Research Strong Claim

Example

IS theories are generally one of two forms: process or variance

Webster and Watson 2002 p. xix

Variance theories are causal; process theories are not

DeLone and McLean 2003 p. 15

Variance theories are positivist; process theories are interpretive

Walsham 1995 p. 388; Wheeler 2002 p. 140

Variance theories and process theories should not be combined

Markus and Robey 1988; Seddon 1997

Laws in variance theory provide prediction; laws in process theory provide understanding

Wheeler 2002 p. 135

THREE ESPOUSED APPROACHES: VARIANCE, PROCESS, & SYSTEMS When one describes a theoretical approach, one can discuss how it is espoused or how it is used. This section describes how each approach is espoused in the literature. In later sections, we will explain why researchers need not always use these approaches in the way that they are espoused, sometimes for good reason, but it is useful to describe how they are espoused first. We are not aware of any paper that has described all three approaches in this paper. Rather, different researchers tend to describe different approaches. Moreover, as Table 2 indicated, multiple researchers have described each one. Rather than describe each approach as espoused by one researcher only, we attempt in this section to summarize the main points agreed to by most researchers who espouse that approach. We take some pains to do so completely because all three approaches are quite rich and no single source does justice to any one of them. 9

Table 4 summarizes the key characteristics of each approach. Rows 1 and 3 were noted earlier in Table 2 and concern the types of concepts and relationships in a theory. The remaining rows consider two issues associated with concepts and relationships—time and causality—that prior works have found helpful in differentiating approaches (Poole et al. 2000). In the next subsections, we describe each approach and clarify the differences highlighted in Table 4.

Table 4: Espoused Differences among the Process, Variance, and System Approaches Dimension 1. Type of concepts (also in Table 2) 2. Change in concepts over time

Variance Approach

Process Approach

Systems Approach

Properties of entities that have varying values

Entities that participate in or are affected by events

Wholes (comprising parts) that have emergent properties

Properties do not change over time (only their values change)

Entities change over time

Wholes, their parts, and their properties can change over time.

Variation among values of properties

Sequences among events (typically probabilistic)

Interactions among parts and reciprocal relationships

3. Types of relationships (also in Table 2) 4. Time ordering in the relationships among concepts

Time ordering among Time ordering of events independent variables is important (properties) is immaterial

Time ordering of events and properties are important

5. ‘Causal’ logic in the relationships among concepts

Causal logic based on necessary, sufficient, and efficient causality

Causal logic based on necessary, final, formal, and efficient causality

Causal logic based on material, final, efficient, and reciprocal causality

The Variance Approach Mohr coined the term “variance” to describe the way that researchers view the world when they see it comprised of independent and dependent variables. Different versions of this approach have been described in social science (Blalock 1969; Dubin 1978; Bacharach 1996) and it is a very popular approach because of the widespread statistical machinery available to test theories created with this approach. For example, in a recent survey of IT impact research, about 80% of articles in leading IS journals were found to have used a variance approach (Pare et al. 2008). In terms of theoretical concepts, the variance approach focuses on properties of entities, 10

often called variables or factors. It is assumed that these properties can have different values even though the property itself has a fixed meaning. For example, an IT system might have the property “system quality.” The meaning of system quality remains fixed over time even though the values for any given system could change over time (e.g., from high to low) and different systems could have different values at any point in time. According to Dubin (1978), the fact that the meaning of properties remains constant over time is crucial for theories to be long lived, because it allows different researchers to study the same properties in independent research projects, thus leading to an accumulating body of knowledge about a phenomenon over time. In terms of theoretical relationships, the variance approach focuses on variation among the values of properties. The properties and their associations with other properties are assumed to remain constant over time (perhaps reflect an underlying ‘law,’ Weber 2003), allowing researchers to assume continuity of effect (Mohr 1982; Poole et al. 2000 pp. 32-33). For example, consider a researcher who predicts that system quality (X1) and availability of resources (X2) explain users’ intention to use a system (Y). According to Poole et al. (2000 p. 34), a variance approach would consider the temporal order of these X variables to be immaterial, because each one is assumed to have an independent and continuous effect on Y. Finally, in relation to causal logic, the variance approach is said to assume necessary, sufficient, and efficient causality (Mohr 1982 p. 38). Each type can be explained using the prior example of system quality, availability of resources, and intention to use a system. As espoused in the literature, a variance approach would suggest that users’ intention to use a system will not change unless there is a change in system quality or availability of resources (i.e., a change in the antecedents is necessary). In addition, if the quality of a system changes or if the availability of resources changes, the variance approach assumes that there will be a change in users’ intentions (i.e., a change in the antecedents is sufficient). Finally, it is assumed that changes in system 11

quality and availability of resources influence users’ intentions directly, without the need for additional factors or events (i.e., they are efficient causes for the outcome).3 The concepts and relationships in the variance approach can be assembled in a wide range of ways. For example, Dubin (1978 p. 78) distinguishes four different types of properties: enumerative properties, which are properties an entity always has (e.g., a person’s age), associative properties, which are properties an entity may have (e.g., a person’s income), relational properties, which are properties an entity has in relation to other entities (e.g., a person’s centrality in a group), and statistical properties, which describe an entity’s range of values on a property (e.g., a person’s average monthly income). Shoemaker et al. (2004 p. 59) give a similarly detailed treatment of different types of relationships. Researchers can also differ in the emphasis they give to the different dimensions of causality, for example, emphasizing sufficiency and thus searching for a complete set of independent variables, or emphasizing efficiency and thus searching for mediating variables that explain how a predictor’s effect works. A full examination of the range of ways that the variance approach can be used would merit its own paper, but suffice to say, the variance approach is quite flexible. The Process Approach Despite the flexibility of the variance approach, Mohr felt it was ill suited to studying organizational change. He advocated the process approach. Just like the variance approach, the process approach has a long history independent from Mohr (e.g., Abbot 1983; Abell 1984). Since Markus and Robey (1988) introduced this approach to IS, it has been used in a range of studies (e.g., Newman and Robey 1992; Crowston 2000; Montealegre and Keil 2000), but it is still used

3

The notion of efficient causality stems from Aristotle. Poole et al. (2000, p. 42) explain “Aristotle distinguished four causes…material, formal, efficient, and final. Respectively they indicate that from which something was made (material cause), the pattern by which something was made (formal cause), that from which comes the immediate origin of movement or rest (efficient cause), and the end for which it is made (final cause)….”

12

much less than the variance approach. For example, in their survey of IT impact research, Pare et al. (2008) found that only 20% of articles in leading IS journals used a process approach and this 20% was almost entirely found in just one journal (Information & Organization). Similar observations have been noted in other disciplines (Rescher 1996; Emirbayer 1997). In terms of theoretical concepts, the process approach focuses on entities participating in events. If the entities can act, they are often referred to as focal actors (Pentland 1999; Ramiller and Pentland 2009).4 For example, the Coping Model of User Adaptation (Beaudry and Pinsonneault 2005) explains how users adapt to IT events in their organizations. In this theory, the focal actors are the users, and the events are the introduction of new systems or the modifications of old systems. As Table 4 showed, the process approach assumes that entities, or focal actors, change over time. For example, the introduction of a new system might make a user concerned about his job security. This might lead the user to react differently to other events (such as performance reviews) than he would have reacted in the absence of the new system. In terms of theoretical relationships, the process approach focuses on accounting for an outcome by reference to a sequence of events involving the focal actors. This sequence is typically assumed to be probabilistic (Markus and Robey 1988, Mohr 1982). For example, in Beaudry’s and Pinsonneault’s (2005) theory, one of the outcomes is exiting the company. They explain that “exit” occurs as a result of the following probabilistic sequence of events: (i) the user becomes aware of an IT event, (ii) perceives it to be a threat, (iii) perceives that they have little control over it, and (iv) engages in self-preservation, by exiting the company. The sequence is probabilistic rather than deterministic because it is possible that a different sequence of events might occur. For example, Beaudry and Pinsonneault explain that when users perceive IT events

4

Some process researchers take a more extreme view, assuming no entities or focal actors (i.e., the whole world is processual), but this view is rarer than the view discussed here (Rescher 1996).

13

to be threats, many outcomes are possible, exit being just one. Finally, in relation to causal logic, the process approach is said to use necessary, final, formal, and efficient causality. This differs from the variance approach in two ways. First, no single event in a chain is considered sufficient to determine a subsequent event (Mohr 1982). Second, events can be determined by the goals of focal actors (final causality) and/or their plans (formal causality) (Poole et al. 2000 pp. 42-43). These four types of causality can all be seen in Beaudry’s and Pinsonneault’s (2005) study. For example, users have no need to exit the organization unless an event threatens them (necessary causality); the outcomes that occur are driven partly by users’ goals to maintain their well-being (final causality) and their strategies to adapt to events (formal causality); and each outcome has an immediate precursor, for example, exit occurs as a result of a user engaging in self-preservation (efficient causality). As Table 4 showed, time is an important element in the causal logic too. For example, users appraise IT events after the events occur, and take actions after they have appraised a situation. As with the variance approach, the concepts and relationships in the process approach can be assembled in many ways. For example, researchers can view entities as things that can influence events, such as organizations that act, or as things constituted by events, such as organizations constituted by patterns of actions (Langley 2009). Likewise, researchers can distinguish routine events from events that start or end processes (Newman and Robey 1992) and between events that can be examined in isolation and events that can only be understood as part of a series (Peterson 1998). Different researchers can also emphasize different elements of causality, for example, by placing more or less emphasis on final and formal causality depending on the extent to which actors have power in the context being studied. Once again, our aim here is not to identify every way that the process approach can be implemented, but rather to highlight that researchers can use the approach in a wide variety of ways. 14

The Systems Approach The emphasis given to the process/variance distinction in IS research (Markus and Robey 1988; Shaw and Jarvenpaa 1997; Webster and Watson 2002) may suggest that they are the only approaches. However, another is the systems approach. It derives from a conviction that the world is comprised of wholes and interacting parts, not merely entities, properties, and events (Boulding 1956; von Bertalanffy 1968). Mattesich (1978) explains: “…the systems approach is based on the insight that the interrelations of certain components may result in an entity (system) with its very own properties. Hence, this approach looks at systems holistically, emphasizing the interrelations of the system’s components…, the properties and boundaries of the system vis-à-vis its environment…., [and its] function or purpose…”

The systems approach can be traced back to the debate between holism and reductionism in Greek philosophy (Klir 1991 p. 24), but it advanced most rapidly in the two decades after the second world war (Dubin 1978) when it had a strong impact on many academic fields. In fact, many of our field’s forefathers were systems theorists (Churchman 1968; Forrester 1968; Trist 1981; Checkland 1999). Interest in the systems approach dissipated in the 1980’s and 1990’s, and its use became rare, at least in North American IS research (Lee 2004). As one systems theorist opined, IS researchers “continue to believe that there are such things as unilateral causation, independent and dependent variables, origins, and terminations” (Abdel-Hamid 1988 pp. 397398). A similar trend occurred in organization science, as multilevel researchers noted: “[D]espite the historical tradition and contemporary relevance of organizational systems theory, its influence is merely metaphorical. The system is sliced into organization, group, and individual levels, each level the province of different disciplines, theories, and approaches” (Kozlowski and Klein 2000).

In the last decade, however, there has been renewed interest in the systems approach (Anderson et al. 1999; Porra 1999; Clark et al. 2007; Rivard and Lapointe 2010). 15

In terms of theoretical concepts, the systems approach focuses on wholes, parts, and emergent properties that arise from interactions among parts. The fact that properties “emerge” means that entities change and thus time is a key part of one’s theory. Multilevel research is a good example. Multilevel researchers often create constructs to reflect emergent properties of collectives, such as a group’s memory, by examining patterns of interaction among members of a collective. Such researchers also study how properties emerge and entities change over time: “The structure of a collective construct refers to the actions and interactions among individuals that generate the collective phenomenon that a collective construct [reflects].” “…collective properties [tend] to emerge and change more gradually than individual ones …. For example, the emergence and change of collective usage is likely to be gradual because changes in collective usage require coordination among individuals, dyads, groups, and so on” (Burton-Jones and Gallivan 2007 pp. 661, 672) .

In terms of theoretical relationships, the systems approach focuses on the interactions among parts of the system. Reciprocal relationships, known as feedback, are also typical. For example, Clark et al. (2007) offer a causal loop to explain how executives respond to perceived IT gaps by increasing commitment to systems. This increased commitment leads to reductions in IT gaps, which then leads to reduced commitment to systems. This is known as “negative” feedback because it leads to equilibrium. “Positive” feedback can also be posed to explain how vicious or virtuous cycles arise (Garud and Kumaraswamy 2005). When specifying relationships in the systems approach, time is also an important factor, as multilevel researchers have noted: “Although researchers often assume that the effect of independent variables on dependent variables is instantaneous, this may not be the case; especially in collectives, the relationship between predictor and outcome variables may take time (e.g., days, months or years) to emerge” (Burton-Jones and Gallivan 2007 p. 471).

Finally, in relation to causal logic, the systems literature does not contain much discussion of necessary or sufficient causality, unlike the process and variance literatures. However, systems 16

researchers often do invoke material, final, efficient, and reciprocal causality. Material causality is invoked when a researcher explains how the “whole” comes into being. For example, BurtonJones and Gallivan (2007) argued a group uses an IT only when its members use it in an interdependent fashion. Final causality is invoked when a researcher accounts for system-level phenomenon by reference to the goals or aims of those in a system (Poole et al. 2000). For example, Morgeson and Hofmann (1999) describe how collectives often organize themselves into different configurations depending on the goal they wish to achieve. Efficient causality is invoked when a researcher studies the mechanism by which a system-level outcome occurs, for example, when a researcher concludes that a group’s use of a system enables the group to perform well by helping it make better decisions (Sarker 2006). Finally, it is also typical for a systems researcher to propose feedback effects, such as group members reflecting on their group’s performance and revising their use of an IT so that the group can perform more effectively (reciprocal causality). Just like the variance and process approaches, the systems approach can be implemented in a wide range of ways. At the risk of oversimplifying distinctions among different traditions, one can think of a continuum ranging from approaches that assume that systems are hard, mechanistic, closed, and relatively predictable, to those that assume that systems are soft, organic, open, and inherently unpredictable (Burns and Stalker 1994; Checkland 1999). As a result, researchers can assemble systems theories in many ways, while still holding true to the general characteristics of the systems approach that we have described.

RECONSIDERING THEORETICAL APPROACHES The prior section showed how the variance, process, and systems approaches can be described and distinguished using a fairly small set of dimensions: by the types of concepts and relationships they use, and the closely associated issues of time and causation. This is a useful 17

discovery because it highlights the essential similarity underlying each approach. We believe their similarity has important implications for how researchers treat and use the approaches. Theoretical Approaches are not Methodological Approaches Although research requires a close interplay between theory and method, these two aspects of research are distinct. However, when one reads discussions of process, variance, and systems approaches, one often finds that the discussions mix theoretical issues with methodological ones. For example, consider the widely held view, noted earlier in Table 4, that the time-ordering of independent variables is immaterial in the variance approach. This stems from the argument that the variance approach can be likened to a regression model (Abbott 1988). Although the timeordering of independent variables is indeed immaterial in a regression model, this is a statement about a statistical technique, not a theoretical approach. We see no reason why researchers could not use a variance approach to develop a theory in which there are temporal distinctions among antecedent factors (such as relationships among a property at time 1, a second property at time 2, and a third property at time 3) and then test it with a structural equation model rather than a regression model, as Kim (2009) did. Likewise, there is no reason why a researcher could not use a variance approach to develop a theory in which a construct has a discontinuous effect on another construct over time (Shoemaker et al. 2004 p. 59). In short, researchers should not equate theoretical approaches with methodological approaches because this may lead them to underestimate the capabilities of the theoretical approaches they are examining. Theoretical Approaches are not Meta-theoretical Perspectives IS researchers often distinguish among meta-theoretical perspectives, such as positivist, interpretive, and critical perspectives (Orlikowski and Baroudi 1991) and human and natural science perspectives (Hirschheim 1985). Some researchers suggest that these perspectives are associated with specific theoretical approaches, for example, that variance theories are positivist 18

and process theories are interpretive (Walsham 1995; Wheeler 2002). Much like our discussion of methods, we believe that theoretical approaches are distinct from metatheoretical positions. For example, a key aim of interpretive research is to understand a social setting from the actors’ point of view (Lee 1991). Just as researchers use scientific concepts and theories to understand the world, actors in day-to-day life use lay theories and concepts (Markman and Gentner 2001 p. 232). It would seem prudent not to assume that actors will only adopt one specific type of concepts when thinking about the world, such as the type considered in the process approach. Although people in their day-to-day lives do often think in terms of actors, events, and processes (Bruner 1991), they also think in terms of relationships among properties of things, and among wholes and parts (Zacks and Tversky 2001; Medin and Atran 2004). Therefore, being open to multiple theoretical approaches could only help researchers, providing them with more conceptual tools with which to understand and describe the way that actors themselves understand and describe their social settings. A similar logic can be used for researchers who adopt a positivist or critical perspective; all can benefit by being open to multiple theoretical approaches. Theoretical Approaches are not Rules The approaches we have outlined are categories or classes. That is, we can use them to classify existing theory just as we can use them to guide the creation of new theory (Shaw and Jarvenpaa 1997 pp. 72-73). Research on classification tells us that things can always be classified in a multitude of ways and that individuals can be instances of a class even though they lack properties of that class (Lackoff 1987). This has two key implications. First, as noted earlier, we cannot say that the three approaches we outlined are the only approaches. Each one has examples in IS research and each one has roots as far back as the ancient Greeks (Klir 1991; Rescher 1996). This long history is probably a good indication that they are useful in some way. However, just as “theory is a way of looking at the facts, of organizing and representing them” (Kaplan 1964/1998 19

p. 309), the approaches we have outlined are simply ways of looking at theories, and how they are organized and represented. Researchers could propose other ways of looking at theories and thereby identify different approaches. The second implication is that there is no reason why any single theory must exhibit all characteristics of any one of the approaches we have described (just as there is no reason why all animals classified as birds must fly). It is quite possible that a good theory will lack an element of one of the approaches, or combine elements of more than one approach. In this light, consider the following widely espoused views (Mohr 1982; Poole et al. 2000): 

the variance approach involves sufficient causality; the process approach does not



the process approach involves final and formal causality; the variance approach does not



the variance approach and the process approach should not be combined

We believe that all of these views can and should be superseded. For example, assume that a researcher constructs a theory that follows a process approach in every respect except that it includes an event that is sufficient to cause a distal outcome, much like one domino falling can be sufficient for many dominoes to fall. In contrast to past views, we would still say that this theory followed a process approach (just as we would say that a non-flying bird such as a penguin is still a bird). More importantly, we could not deny a theory’s legitimacy simply because it did not follow some element of the process approach. For the same reason, researchers should feel free to use a variance approach to develop a theory that includes elements of final causality, for example, by having a construct to represent the influence of an actor’s goals. Researchers should also feel free to construct theories that combine elements of different approaches, much like nature has animals with characteristics of both birds and mammals (e.g., the Australian platypus). Like all classifications, the approaches we outlined simply tell us what typically ‘goes together’ in theories. Knowing what typically goes together is useful, but it should not be viewed as a rule. 20

A Theoretical Approach is not Determined by the Researcher’s Goal Several IS researchers have reviewed the goals of theory (Gregor 2006; Hovorka et al. 2008). We will briefly review three goals often mentioned in research and explain why each approach can support each goal. The basic reason in each case is that each goal depends on researchers specifying relationships among concepts, and this is done in each approach. Explanation. Explanations say why something occurs (Salmon, 1998). The “why” in theories is explained by the relationships among concepts in the theory (Kaplan 1964/1998 pp. 333, 346). The strength of a theory’s explanation derives from the accuracy of these relationships.5 Because each approach we have outlined includes relationships, each one can provide a basis for an explanation. The details of the explanation will differ in each case because the nature of the relationships differs, e.g., a researcher might explain an outcome by referring to lawful relation among properties (a variance approach), a sequence of events (a process approach), or interactions among parts (a systems approach). The degree to which any of these explanations is good depends on how well the relationships specified reflect the phenomenon being studied. Prediction. Predictions foretell the state of a property or event (Dubin 1978), i.e., they say what state or event will occur. Like explanations, they depend on a theory’s relationships. For this reason, some argue that explanations and predictions are simply reverse logical operations (Suppe 1977). Intuitively, however, researchers can make predictions without being able to explain the mechanisms involved—the intervening processes or intervening variables. Nevertheless, because all three theoretical approaches contain relationships, all three can provide a basis for prediction. Again, the strength of the prediction depends on how accurately the relationships specified in the theory match the phenomenon being studied.

5

The strength may also depend on additional components in a theory (such as boundaries) (Dubin, 1978; Weber, 2003). As noted at the outset of our paper, however, these additional components are outside our paper’s scope.

21

Understanding. There are two common meanings of understanding in research: scientific understanding and verstehen. Scientific understanding refers to explanation (Salmon 1998; Hovorka et al. 2008). For the reasons noted above, each approach can support scientific understanding. Verstehen (a German term for understanding) is used by researchers who follow phenomenological approaches (Schwandt 1997). According to past research, researchers interested in verstehen tend to develop theories that differ from those developed by researchers interested in scientific understanding in three ways: (1) they tend to use theories as sensitizing devices rather than as objects to falsify (Klein and Myers, 1999), (2) they tend to ground their research in ideographic rather than nomothetic details (Klein and Myers, 1999), and (3) they seek to understand relationships among concepts but they do not assume strong lawlike relationships (Maxwell 1992; Schwandt 1997). This third difference is the only difference that relates to one’s theoretical approach; the preceding differences relate instead to how one uses an approach. Moreover, as we note below, none of the theoretical approaches require researchers to specify strong, lawlike relationships. Thus, researchers can use any one of the approaches to support understanding. As with explanation and prediction, the extent of understanding gained in a given study depends on how well the researcher’s theory fits the setting studied.6 Theoretical Approaches do not Determine Precision Theoretical approaches are often distinguished in terms of determinism. For example: “By their very structure, variance theories posit an invariant relationship between antecedents and outcomes. This assumption may simply be too stringent for social phenomena. … As Sutherland put it, “not all real-world phenomena will ultimately become deterministic if we spend enough time analyzing them.” …In circumstances like these, process theories may [be attractive alternatives]” (Markus and Robey 1988 p. 592)

6

As with theories that support explanation, theories to support verstehen may also need additional elements to support understanding, e.g., modalities and moral context (Giddens 1984; Pentland 1999). These are not central to the distinction among variance, process, and systems approaches, however, so are not considered here.

22

The view expressed in this statement stems from the notion that the variance approach involves necessary and sufficient causality, whereas the process approach involves necessary causality only. Much like our previous arguments, we believe that this view can be broadened. A broader view is useful because it allows us to see that the process approach also results in explanations that are deterministic, just in a different way. As Mohr (1982 p. 59) explains: “To say that X is necessary for Y is to say that Y is sufficient for X: If Y, then X.” In other words, for researchers following a process approach, if the outcome of interest occurred, it can be determined, without any doubt, that the prior event occurred, even if it was not observed. In other words, the causal logic of necessity invokes its own kind of determinism. For this reason, the presence of determinism per se is not the best way to differentiate among theoretical approaches. A different way of thinking about this issue is to assess the precision of relationships in a theory. We use the word precision according to its dictionary meaning of “exactness.” In all three theoretical approaches, researchers can specify relationships in more or less precise ways. For example, a researcher following a variance approach may use the logic of necessary and sufficient causality as a heuristic device when thinking of antecedent factors but, in fact, have no expectation that the antecedents are truly necessary and sufficient. Likewise, a researcher following a process approach may use the logic of necessary causality when thinking of precursor events to some outcome but may be completely open to the possibility that the outcome might occur without the precursor events specified in the theory. In both cases, we would simply say that the relationships specified in the theory have low precision. More precision may be gained over time as research in an area progresses. For example, researchers may identify additional mediating variables (in the variance approach), triggering events (in the process approach), and important interactions (in the systems approach). Ultimately, however,

23

philosophers remind us that relationships specified in social science will always be imprecise, whatever one’s approach to theory building (Kaplan 1964/1998 pp. 351-355). Revisiting Claims about Theoretical Approaches Having reviewed and clarified each approach, we now briefly revisit some prior claims about theoretical approaches. By increasing researchers’ awareness and understanding of theoretical approaches, the authors listed in Table 5 contributed significantly to research. With increasing calls on researchers to build and extend theory (Weber 2003; Markus and Saunders 2007), we think now is an important time to build upon their work and adopt a broader conception of theoretical approaches. This broader conception would recognize that there is no one-to-one relationship among research elements—such as researchers’ goals, metatheoretical positions, theoretical approaches, and empirical approaches. Moreover, it would recognize that theoretical approaches are guides, not rules that must be conformed to. Overall, the view we have advanced is motivated by the fact that building good theory is difficult and researchers need all the flexibility they can get.

Table 5: Broadening Conceptions of Theoretical Approaches Conception

Citation

A Broader View

IS theories are generally one of two forms: process or variance

Webster and Watson 2002 p. xix

At least three approaches to theory building are used in IS research (variance, process, and systems). Researchers can also combine them. There is no reason why researchers should limit themselves to the process or variance approaches alone.

Variance theories are causal; process theories are not

DeLone and McLean 2003 p. 15

Each approach to theory building offers causal logic, but different dimensions of causality are addressed in each one. While some dimensions of causality are common to each approach (e.g., efficient causality), others are unique (e.g., material causality, which is unique to the systems approach).

Variance theories are positivist; process theories are interpretive

Walsham 1995 There is no one-to-one relationship between one’s theoretical p. 388; approach and one’s meta-theoretical position. Each approach Wheeler 2002 can be used with each position. p. 140

24

Conception

Citation

A Broader View

Variance theories and process theories should not be combined

Markus and Robey 1988; Seddon 1997

Researchers can combine theoretical approaches. A theory can also be built that follows one approach predominantly but fails to follow certain characteristics of the approach. A theory should not be judged by its conformance to an approach but by its ability to help a researcher account for some phenomenon.

Laws in variance theory provide prediction; laws in process theory provide understanding

Wheeler 2002 p. 135

Researchers’ ability to explain, predict, or understand something depends on the precision of the relationships specified in the theory and the extent to which these relationships reflect the phenomenon being studied. All three approaches allow researchers to specify relationships, with various degrees of precision. Thus, there is no one-to-one relationship between a theoretical approach and any particular goal.

Considering Two Possible Counterarguments Several criticisms could be made regarding our arguments. Two would be particularly fatal if true. First, researchers might claim that our arguments are misguided, that is, they might argue that researchers should not comply with our advice. Second, and in contrast, researchers might claim that our arguments are not new, that is, they might argue that researchers are well aware of the three approaches and use them fully. We believe that neither of these arguments is true. To obtain evidence for our position, we reviewed all of the papers given “Best Paper” awards by MIS Quarterly since 2000. Because MIS Quarterly weights the theoretical contribution of a paper heavily, we could be assured that these papers provide reasonable examples of theory in IS research. We then hired two independent and qualified coders to assess the theories developed in these papers against a detailed set of coding criteria. The aim was to determine if these papers: (1) contained at least some characteristics that were consistent with our arguments (because this would refute the first criticism above) and/or (2) showed a complete understanding of the three approaches (because this would support the second criticism). We omit the detailed results of this review for space reasons, but they can be obtained from the authors upon request. 25

The results of the review supported our position. For example, the coders found several papers that used a systems approach (Lamb and Kling 2003), not just papers using a variance approach (Dennis et al. 2008) or a process approach (Majchrzak et al. 2000). In addition, they found that some papers used elements of more than one approach, e.g., the coders found that Majchrzak et al. (2000) primarily used a systems approach but also used elements of the process approach. These results suggest that our arguments do not contradict good research practice. At the same time, the coders found evidence that our arguments are not completely understood by all researchers. For example, none of the papers explicitly discussed all three approaches and explained their reasons for using the approach that they used. Moreover, there were even papers that explicitly stated that they adopted one approach but in practice appeared to have adopted a different approach, e.g., Burton-Jones and Gallivan (2007, p. 659) stated that they used a variance approach but the coders disagreed, finding that this article primarily used a systems approach. Overall, it appeared from our review that our arguments are consistent with good practice in the field but are, nevertheless, not simply a restatement of what researchers already know.

GUIDANCE FOR COMBINING THEORETICAL APPROACHES In addition to providing a description and analysis of theoretical approaches, we also wish to provide guidance for how researchers can improve their use of theoretical approaches. Based on our earlier analysis in Table 5, there are several ways we could do this. We decided to focus on providing guidance for one of these areas: combining theoretical approaches. We chose this because we believe it offers a particularly significant opportunity to improve theory in IS research in light of the results in Pare et al. 2008 that researchers very rarely combine approaches. As noted earlier, Mohr (1982) advised against combining variance and process approaches and several IS researchers have restated his view (Markus and Robey 1988; Seddon 1997). Our view 26

is more in line with those who point out benefits of combining approaches, e.g.: I would argue that the insistence on exclusion of variables from process research unnecessarily limits the variety of theories constructed. It may be important to understand the effect of events on the state of an entity (a variable) or to identify the effect of a contextual variable on the evolution of events (Langley 1999 p. 693).

Despite researchers such as Langley (1999) and Shaw and Jarvenpaa (1997) touting the benefits of combining theoretical approaches over a decade ago, we are not aware of any guidance on how to go about doing so. Instead, guidance on using theoretical approaches has focused on each approach in isolation (see, e.g., Van de Ven 2007). Perhaps this is one reason why so few papers explicitly combine approaches. For example, in their analysis of ‘IT impact’ research from 1991-2005, Pare et al. (2008) found only one article in their entire sample of 161 that combined approaches. The article they found combined the process and variance approach. However, other combinations are possible. To highlight these opportunities for researchers, Table 6 highlights 12 different ways that combinations can be developed, and the benefits that can be obtained from each one. Two of these combinations (#1 and #6) correspond to the combinations referred to in Langley’s quote above, but all 12 combinations offer opportunities for research. Table 6: Benefits of Combining Theoretical Approaches Original approach Pure Variance

Benefits that researchers can obtain by combining the original approach with: Process Approach: 1. Improving understanding of concepts: Understanding whether the state of an entity is affected by events or processes 2. Improving understanding of relationships: Understanding the process by which a relationship among properties occurs Systems Approach: 3. Improving understanding of concepts: Understanding whether the state of a component (lower-level) property is affected by a higher-level property of the system 4. Improving understanding of relationships: Understanding whether a relationship among properties is affected by a higher-level property of the system

27

Original approach Pure Process

Benefits that researchers can obtain by combining the original approach with: Variance Approach: 5. Improving understanding of concepts: Understand whether the occurrence of an event is affected by the state of a property 6. Improving understanding of relationships: Understand whether the influence of an event in a process depends on the state of some property Systems Approach: 7. Improving understanding of concepts: Understand whether the emergence of an entity or the occurrence of an event hinges on a higher-level property of the system 8. Improving understanding of relationships: Understand the process by which a system emerges or has effects

Pure Systems

Variance Approach: 9. Improving understanding of concepts: Understand whether an emergent property of a system is affected by a lower-level property of the system 10. Improving understanding of relationships: Understand whether interactions among parts of a system depend on properties of the parts Process Approach: 11. Improving understanding of concepts: Understand whether the existence of a system or emergent property hinges on particular events or processes 12. Improving understanding of relationships: Understand whether interactions among parts of a system follows a particular process

In addition to identifying specific combinations that researchers can seek and the benefits that can be obtained from each one, we also suggest two strategies that researchers can use to combine approaches (see Table 7). Each of these approaches has associated benefits and risks, which we describe in turn below. The key for authors using either of these strategies is to maximize the benefits while minimizing or accounting for the associated risks.

Table 7: Strategies for Combining Theoretical Approaches Strategy

Description

Benefits

Risk

Independent

Theorize about the phenomena using two or more approaches independently

Corroboration Insight

Redundancy Uncertainty

Hybrid

Theorize about the phenomena once using a hybrid approach

Completeness Ecological validity

Reduced parsimony Complexity/error

28

The first strategy in Table 7 is the independent strategy. This is based on Kaplan and Duchon’s (1988) advice for multimethod research. In this strategy, a researcher builds a theory from two or more approaches independently (e.g., process, variance, and/or systems). The theories are then evaluated by conducting independent inquiries of propositions emanating from each approach. If the results from each inquiry corroborate each other, more confidence can be gained regarding one’s ability to understand, explain, and/or predict the phenomenon in question. More importantly, this approach might lead to novel insights if different findings arise (Davis 1971; Kaplan and Duchon 1988). Given that the independent strategy offers two strong benefits, it is not clear why it is used so rarely. To our knowledge, only Sabherwal and Robey (1995) have used this strategy explicitly. They conducted separate analyses of IS development practices from a process view and a variance view and then corroborated the two sets of results. Admittedly, they concentrated more on methods than theory, and only considered process and variance approaches (not the systems approach), but it is still an exemplar. Perhaps the reason why the independent strategy is used rarely is the effort that it entails. If the propositions and results are the same between approaches, a researcher may consider this extra work to have been redundant. Alternatively, if the propositions and results differ, the researcher may remain uncertain about his/her findings and need to perform further research before submitting the research for review and publication. The second strategy for combining approaches is the hybrid strategy. Several researchers have recommended it (Shaw and Jarvenpaa 1997; Webster and Watson 2002), but it has not been clear exactly what types of hybrids can be constructed and what benefits they offer. To help address this problem, we highlighted several ways that hybrids can be constructed in Table 6 and two benefits that be obtained in Table 7. The two benefits are related because theories are only partial accounts of the world. The world is not limited to entities and properties, processes and 29

events, or parts and wholes; rather, it contains all these things. Consequently, a hybrid strategy enables researchers to obtain a more complete account of the part of the world that is of interest to them. In addition to advancing theory, this also improves researchers’ ability to generalize findings to practice. This is because practitioners operate in a world that contains entities and properties, processes and events, and parts and wholes, not just one or two of these elements. Thus, a hybrid strategy can enable researchers to build theories that offer more realistic (ecologically valid) insights to practitioners. As Table 7 shows, a major risk with the hybrid strategy is that theories constructing using this strategy might lack parsimony. Thus, researchers must use a hybrid approach only if the increase in understanding afforded by the more complex theory outweighs the loss of parsimony that results. Another risk is that researchers may make errors in applying the hybrid strategy. Two common errors are that researchers (a) apply a theoretical approach that is not applicable to the setting studied (e.g., applying a systems approach when the phenomenon is not systemic) (Morgeson and Hofmann 1999) or (b) transform one approach mistakenly into another approach, thereby failing to obtain the benefit of the first approach (e.g., when a researcher attempts to use a process approach but transforms the elements of the process into variables that he or she is used to working with) (Poole et al. 2000). Neither of these errors is a necessary outcome of using a hybrid strategy. Nevertheless, errors are perhaps more likely with this strategy given that it is the most complex and there are few exemplars to learn from. The independent strategy can be implemented by a research team made up of experts in each approach alone, but the hybrid strategy requires researchers who are experts in combining approaches (a rare skill).

30

USING APPROACHES TO IMPROVE THEORY: AN EXAMPLE In addition to providing general guidance for improving theory, we would like to provide guidance for improving specific theories in IS. For two reasons, we use DeLone and McLean’s (1992; 2003) IS Success Models as a case illustration (see Figure 2). First, determining IS success remains an ongoing concern in practice and research. Although the field has several accepted theories regarding IS adoption, there is less in the way of solid theory regarding performance outcomes from using IS. Thus, this is an area in need of theoretical attention. Second, D&M’s success model is one of the most well cited models in IS research, but has been criticized for using a hybrid approach (Seddon 1997). Also, it was originally proposed merely as a ‘model,’ not a theory. Moving it from being a model to a theory is a significant opportunity for our field.

System Quality

Use Individual Impact

Information Quality

Organizational Impact

User Satisfaction

Reprinted by permission, W. DeLone and E. McLean, Information Systems Success: The Quest for the Dependent Variable. Information Systems Research, 3(1), 1992, pp. 60-95. Copyright 1992, The Institute of Management Science (INFORMS), 901 Elkridge Landing Road, Suite 400, Linthicum, MD, 21090 USA.

Figure 2: D&M (1992) IS Success Model: Original Form

The objective of the D&M model is to define IS Success. The model has two underlying propositions: (1) the success of an IS depends on what dimension of success one examines, and (2) the dimensions of success are related. Figure 2 illustrates each one. The six dimensions in the model illustrate the first proposition, while the arrows between the dimensions illustrate the 31

second. In the sections below, we examine what the D&M model would be (and could be) if the model used the variance approach, the process approach, or the systems approach.7 We then describe what theoretical approach we believe the D&M model actually utilizes and discuss how this approach and others could be investigated more fully to improve theories of IS success. Overall, our aim is not to criticize the D&M model. On the contrary, our analysis is motivated by an opportunity that has always existed with their work—discovering what it would take to extend their “model” into a “theory”—and identify ways that researchers could proceed in this endeavor. The D&M IS Success Model from a Variance, Process, and Systems Perspective In a variance approach, a model’s concepts are properties of things that vary. Figure 3 shows what the D&M model would become if it followed a pure variance approach. It is similar to the original model, except that we explicated the relationships among the properties and we excluded the link to ‘organizational impact’ because this link seems to imply a different level of a social (organizational) system. Translating the D&M model into a pure variance form is useful because it reveals how its underlying theory needs to be improved. We highlight several improvements that should be made along these lines in Table 8, focusing particularly on clarifying the concepts in the theory and the relationships among the concepts. To date, many researchers have adopted the D&M model uncritically. Seddon (1997) undertook a notable extension to the model from a pure variance perspective, but as Table 8 shows, much more theoretical work is needed.

7

To conserve space, we rely primarily on diagrammatic representations of the D&M models. However, it should be clear that the conclusions would be the same irrespective of the formalism, whether diagram, narrative, or formulae.

32

System Quality

Use Individual Impact

Information Quality

User Satisfaction

Figure 3: D&M (1992) IS Success Model: Variance Approach

A process approach requires that a model’s concepts be events that follow a probabilistic sequence. Figure 4 shows what D&M’s model would become, according to DeLone and McLean (2003 p. 16), if it followed a pure process approach. Explicating this process is useful because it highlights the extremely simple process assumed by the D&M model. Moreover, it reveals opportunities for improving the rigor with which the concepts and the relationships among the concepts could be specified. We summarize several improvements that can be made along these lines in Table 8. In short, translating the D&M model into a process form once again reveals how its underlying theory can and should be improved.

Create system s

Use system s

Consequences of system use

Figure 4: D&M (1992) IS Success Model: Process Approach

Finally, a systems approach requires that a model’s concepts involve interacting parts and emergent properties. Figure 5 shows what D&M’s model would become if it followed a systems approach. The figure shows organizations (wholes) consisting of information systems and individual users (parts). These parts interact through individuals using systems. These interactions can then lead to changes in attributes of these parts, e.g., users become more or less 33

satisfied with a system over time based on their interaction with it. Moreover, out of these interactions can emerge a change in an organizational-level property, i.e., organizational impact. Once again, transforming the D&M model into a systems form helps reveal improvements that can be made to it. We summarize these in Table 8. DeLone and McLean (2003) acknowledged the importance of feedback effects, but with the exception of Kanungo (2003), few have examined IS success from a systems approach. This offers significant opportunities for research. Organizational system Properties:  Organizational impact of information system

Individual user

Information system Properties:  System quality  Information quality

Emergent effect

Interaction between IS and user (i.e., usage)

Properties:  Experience with using IS  Satisfaction with IS  Impact from using IS

Figure 5: D&M (1992) IS Success Model: Systems Approach

Table 8: Theoretical Improvements to be made to the D&M Model Approach Dimension

Required Improvements

Variance

It is not clear what “impact” or “use” mean, e.g., whether they refer to amounts of use and impact, or to specific types of use (e.g., effective use) and impacts (e.g., performance)

Concepts

Relationships

Necessary and sufficient causality is unclear because: (a) the model does not explain the direction (i.e., positive or negative) of the relationships between concepts (b) the model does not explain why the relationships among all concepts are mediated and linear, i.e., why there are no unmediated relationships or moderated relationships

Process

Concepts

The scope of each event in time is not clear, e.g., what “create system” includes/excludes It is not clear what “consequences” mean, e.g., whether it refers to one or many events

34

Approach Dimension Relationships

Required Improvements Final causality is unclear because the model’s final outcome (consequences) is ill defined Formal causality is unclear because the model does not theorize how events are planned Efficient causality is unclear because the model does not theorize how soon events occur after one another

Systems

Concepts

It is not clear what individual and organizational impact refer to (e.g., whether to performance or something else) It is not clear why “organizational impact” is the only concept at a higher level (i.e., why there are no other emergent properties)

Relationships

Formal and efficient causality are unclear (as noted also for the process approach above) Material and emergent causality are unclear because the model does not explain exactly how impacts emerge at the organizational (whole) level from the individual (parts) level Reciprocal causality is unclear because the only feedback in the model is between properties of an individual user (i.e., usage and satisfaction, shown in Figure 3). Presumably, the organizational impact of a system will have feedback effects on individual systems and users, but these are not shown.

What approach does the D&M IS Success Model use? According to DeLone and McLean (2003), their model used the process and variance approaches. However, our analysis demonstrates that it uses elements of all three approaches: variance, process, and systems. The variance approach is evident in its use of properties such as system quality, user satisfaction, and so on, rather than events. The process approach is evident in its conceptualization of three general phases of success: creation, use, and consequences. The systems approach is evident in its conceptualization of multiple levels of an organizational system. Nevertheless, as Table 8 shows, the use of each theoretical approach in the D&M model could be improved. In a ten-year review of their model, DeLone and McLean (2003) noted that many researchers have adopted the model uncritically. Our analysis suggests that this strategy is unwise because the D&M model needs to be refined and improved, as DeLone and McLean (1992, p. 88) noted when they first proposed it. 35

Seddon (1997, p. 242) argued that by combining several theoretical approaches, the D&M model created “a level of muddled thinking that is likely to be counter-productive to future IS research.” Seddon suggested that the D&M model should use a pure variance approach. Although useful contributions can be made from a pure variance perspective, we do not believe that this is an inherent problem of the D&M model. As we have noted already, elements of different theoretical approaches can and should be combined if it is useful to understand the phenomenon of interest. In the case of IS success, it would seem that multiple theoretical approaches would be useful because it is a very complex phenomenon. Properties such as system quality and user satisfaction can reflect IS success, but the levels of these properties and their interrelationships depend heavily on the timing of critical events (e.g., whether an IS is implemented on time, whether users are trained before phasing out an old IS, and so on). Moreover, IS success can differ in important ways across levels of an organization and links across levels can be complex and reciprocal (Harris, 1995). For all of these reasons, we suggest that each theoretical approach adds a layer of meaning to the nature of IS success and should improve researchers’ ability to understand, explain, and/or predict it. In contrast to Seddon (1997), therefore, our analysis suggests that the problem with D&M’s IS Success Model is not that it combines theoretical approaches per se; instead the problem (and opportunity) is that the particular combination they used could be clarified and refined and other combinations could be sought. Pursuing such research would enable their work to move from being a model of IS success to being a theory, which would be of substantial value to both research and practice.

CONCLUSION Theorizing serves an important role in any discipline. To create or extend a theory, a researcher must use a particular approach. This paper examines three archetypal approaches: 36

variance, process, and systems. It also clarifies these approaches in light of past research. Finally, it suggests how researchers could use these approaches in new ways to improve the field’s ability to understand, explain, and/or predict an important IS phenomenon. Research on theory building tends to be undertaken in two basic ways: in the field of philosophy, researchers debate the logical and philosophical bases of theory (e.g., Suppe 1977); and in applied fields, researchers try to analyze and improve theory building practices in their own discipline (e.g., Mohr in management and Markus and Robey in IS). Our work is an example of the second approach. Many years have passed since Mohr’s and Markus and Robey’s seminal papers on theory building. Over that time, more research has come to light regarding the nature of the process and variance approaches and research on the systems approach has once again captured attention (Sawyer 2005). This makes it an opportune time to reexamine the theoretical approaches available to researchers and understand how these approaches can be used. Theories are just one part of research, and we focused on just one aspect of theories. Moreover, to some, our focus on theories may seem misplaced (Greenwald et al. 1986). Nonetheless, whatever one’s research interest or epistemological orientation, all researchers want to improve their ability to understand, explain, or predict empirical phenomena. A more sophisticated understanding of theoretical approaches can assist this process. It can help researchers who wish to build new theories, by helping them understand the types of concepts and relationships available to them; it can help researchers who wish to extend theories, by enabling them to see additional types of concepts and relationships that may complement those in the existing theory; and finally, it can help researchers in their reviewing roles, by enabling them to see ways in which authors can clarify the concepts and relationships in a theory and improve their justification.

37

REFERENCES Abbot, A. "Sequences of Social Events," Historical Methods (16:4) 1983, pp 129-147. Abbott, A. "Transcending General Linear Reality," Sociological Theory (6) 1988, pp 169-186. Abdel-Hamid, T.K. "The Economics of Software Quality Assurance: A Simulation-Based Case Study," MIS Quarterly (12:3), Sept 1988, pp 395-411. Abell, P. "Comparative Narratives," Journal for the Theory of Social Behavior (14) 1984, pp 309-331. Abell, P. The Syntax of Social Life: The Theory and Method of Comparative Narratives Clarendon Press, Oxford, 1987. Anderson, P., Meyer, A., Eisenhardt, K., Carley, K., and Pettigrew, A. "Introduction to the Special Issue: Applicatons of Complexity Theory to Organization Science," Organization Science (10:3), May-June 1999, pp 233-236. Ashby, R. "Requisite Variety and Implications for Control of Complex Systems," Cybernetica (1) 1958, pp 83-99. Bacharach, S.B. "Organizational Theories: Some Criteria for Evaluation," Academy of Management Review (14:4) 1996, pp 496-515. Beaudry, A., and Pinsonneault, A. "Understanding User Responses to Information Technology: A Coping Model of User Adaptation," MIS Quarterly (29:3) 2005, pp 493-524. Blalock, H.M. Theory Construction: From Verbal to Mathematical Formulations Prentice-Hall, Englewood Cliffs, NJ, 1969. Boudreau, M.-C., Gefen, D., and Straub, D.W. "Validation in Information Systems Research: A State-of-the-Art Assessment," MIS Quarterly (25:1), March 2001, pp 1-16. Boulding, K.E. "General Systems Theory: The Skeleton of Science," General Systems (1) 1956, pp 1-17. Bruner, J. Actual Minds, Possible Worlds Harvard University Press, Cambridge, Massachusetts, 1986. Bruner, J. "The Narrative Construction of Reality," Critical Inquiry (18:1), Autumn 1991, pp 121. Bunge, M. Treatise on Basic Philosophy: Volume 3: Ontology I: The Furniture of the World Reidel, Boston, 1977. Burns, T., and Stalker, G.M. The Management of Innovation Tavistock, London, UK, 1994. Burton-Jones, A., and Gallivan, M.J. "Towards a Deeper Understanding of System Usage in Organizations: A Multilevel Perspective," MIS Quarterly (31:4) 2007, pp 657-679. Checkland, P. Systems Thinking, Systems Practice John Wiley & Sons Ltd, Chichester: UK, 1999. Chiles, T.H. "Process Theorizing: Too Important to Ignore in a Kaleidic World," Academy of Management Learning and Education (2:3) 2003, pp 288-291. Churchman, C.W. The Systems Approach Dell Publishing Co., New York, 1968. Clark, T.D., Jones, M.C., and Armstrong, C.P. "The Dynamic Structure of Management Support Systems: Theory Development, Research Focus, and Direction," MIS Quarterly (31:3), Sept 2007, pp 579-615. Crowston, K. "Process as Theory in Information Systems Research," IFIP 8.2 International Working Conference on the Social and Organizational Perspective on Research and Practice in Information Technology, IFIP Conference Proceedings 169 Kluwer, Aalborg, Denmark, 2000, pp. 149-166.

38

Daft, R.L. "Why I Recommend That Your Manuscript Be Rejected and What You Can Do About It," in: Publishing in the Organizational Sciences, L.L. Cummings and P.J. Frost (eds.), Sage Publications Inc, Thousand Oaks: CA, 1995, pp. 164-182. Davis, F. "Perceived Usefulness, Perceived Ease of Use, and End User Acceptance of Information Technology," MIS Quarterly (13:3), September 1989, pp 318-339. Davis, M.S. "That's Interesting! Towards a Phenomenology of Sociology and a Sociology of Phenomenology," Philosophy of the Social Sciences (1) 1971, pp 309-344. DeLone, W.H., and McLean, E.R. "Information Systems Success: The Quest for the Dependent Variable," Information Systems Research (3:1), March 1992, pp 60-95. DeLone, W.H., and McLean, E.R. "The DeLone and McLean Model of Information Systems Success: A Ten-Year Review," Journal of Management Information Systems (19:4), Spring 2003, pp 9-30. Dennis, A.R., Fuller, R.M., and Valacich, J.S. "Media, Tasks, and Communication Processes: A Theory of Media Synchronicity," MIS Quarterly (32:3), Sep 2008, pp 575-600. Drysdale, J. "How are Social-Scientific Concepts Formed? A Reconstruction of Max Weber's Theory of Concept Formation," Sociological Theory (14:1), March 1996, pp 71-88. Dubin, R. Theory Building (Revised Edition) The Free Press, New York, 1978. Dumont, R.G., and Wilson, W.J. "Aspects of Concept Formation, Explication, and Theory Construction in Sociology," American Sociological Review (32:6), 1967, pp 985-995. Emirbayer, M. "Manifesto for a Relational Sociology," American Journal of Sociology (103:2), Sept. 1997, pp 281-317. Forrester, J.W. Principles of Systems, (Second Preliminary Edition ed.) MIT Press, Cambridge, Massachusetts, 1968. Foucault, M. The Archaeology of Knowledge Pantheon Books, New York, 1972. Freese, L. "Formal Theorizing," Annual Review of Sociology (6) 1980, pp 187-212. Garud, R., and Kumaraswamy, A. "Vicious and Virtuous Circles in the Management of Knowledge: The Case of Infosys Technologies," MIS Quarterly (29:1), March 2005, pp 9-33. Giddens, A. The Constitution of Society: Outline of the Theory of Structuration University of California Press, Berkeley, CA, 1984. Glaser, B.G., and Strauss, A.L. The Discovery of Grounded Theory Aldine, Chicago, 1967. Greenwald, A.G., Pratkanis, A.R., Leippe, M.R., and Baumgardner, M.H. "Under What Conditions Does Theory Obstruct Research Progress?," Psychological Review (93) 1986, pp 216-229. Gregor, S. "The Nature of Theory in Information Systems," MIS Quarterly (30:3), Sept 2006, pp 611-642. Gregor, S., and Jones, D. "The Anatomy of a Design Theory," Journal of the Association for Information Systems (8:5), May 2007, pp 312-335. Grover, V., Lyytinen, K., Srinivasan, A., and Tan, B.C.Y. "Contributing to Rigorous and Forward Thinking Explanatory Theory," Journal of the Association for Information Systems (9:2), Feb 2008, pp 40-47. Harris, D.H. (ed.) Organizational Linkages: Understanding the Productivity Paradox. National Academy Press, Washington, D.C., 1994. Heidegger, M. Being and Time State University of New York Press, Albany, 1953.

39

Hirschheim, R. "Information Systems Epistemology: An Historical Perspective," in: Research Methods in Information Sysfems, E. Mumford, R. Hirschheim and R. Fitzgerald (eds.), North-Holland, Amsterdam, 1985, pp. 13-38. Hovorka, D.S., Germonprez, M., and Larsen, K.R.T. "Explanation in Information Systems," Information Systems Journal (18:1) 2008, pp 23-43. Jaccard, J. and Jacoby, J. Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists, Guilford Press, New York, USA, 2010. Kant, I. The Critique of Pure Reason Macmillan, trans. N. Kemp Smith, 1933, London, 1781. Kaplan, A. The Conduct of Inquiry: Methodology for Behavioral Science Transaction edition 1998 (Originally published in 1964), Transaction Publishers, Piscataway, New Jersey, 1964/1998. Kaplan, B., and Duchon, D. "Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study," MIS Quarterly (12:4), December 1988, pp 571-586. Keen, P.G.W. "MIS Research: Reference Disciplines and a Cumulative Tradition," Proceedings of the First Conference on Information Systems) 1980, pp 9-18. Kim, S.S. "The Integrative Framework of Technology Use: An Extension and Test," MIS Quarterly (33:3) 2009, pp 513-537. Klein, H.K., and Myers, M.D. "A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems," MIS Quarterly (23:1) 1999, pp 67-93. Klir, G.J. Facets of Systems Science Plenum Press, New York, 1991. Kozlowski, S.W.J., and Klein, K.J. "A Multilevel Approach to Theory and Research in Organizations," in: Multilevel Theory, Research, and Methods in Organizations, K.J. Klein and S.W.J. Kozlowski (eds.), Jossey-Bass, California, 2000, pp. 3-90. Kuhn, T.S. The Structure of Scientific Revolutions, (Third ed.) University of Chicago Press, Chicago, USA, 1996. Lackoff, G. Women, Fire, and Dangerous Things: What Categories Reveal About the Mind Chicago University Press, Chicago, 1987. Lamb, R., and Kling, R. "Reconceptualizing Users as Social Actors in Information Systems Research," MIS Quarterly (27:2) 2003, pp 197-235. Langley, A. "Strategies for Theorizing from Process Data," Academy of Management Review (24:4) 1999, pp 691-710. Langley, A. "Studying Processes In and Around Organizations," in: The Sage Handbook of Organizational Research Methods, D.A. Buchanan and A. Bryman (eds.), Sage, London, 2009, pp. pp. 409-429. Lee, A.S. "A Scientific Methodology for MIS Case Studies," MIS Quarterly (13) 1989, pp 33-50. Lee, A.S. "Integrating Positivist and Interpretive Approaches to Organizational Research," Organization Science (2) 1991, pp 342-365. Lee, A.S. "Thinking about Social Theory and Philosophy for Information Systems," in: Social Theory and Philosophy for Information Systems, J. Mingers and L. Willcocks (eds.), John Wiley & Sons, Chichester, UK, 2004, pp. 1-26. Majchrzak, A., Rice, R.E., Malhorta, A., King, N., and Ba, S. "Technology Adaptation: The Case of a Computer-Supported Inter-Organizational Virtual Team," MIS Quarterly (24:4) 2000, pp 569-600. Markman, A.B., and Gentner, D. "Thinking," Annual Review of Psychology (52) 2001, pp 223247.

40

Markus, M.L., and Robey, D. "Information Technology and Organizational Change: Causal Structure in Theory and Research," Management Science (34:5), May 1988, pp 583-598. Markus, M.L., and Saunders, C. "Looking for a Few Good Concepts...and Theories...for the Information Systems Field," MIS Quarterly (31:1), Mar 2007, pp iii-vi. Mattessich, R. Instrumental Reasoning and Systems Methodology: An Epistemology of the Applied and Social Sciences D. Reidel Publishing Company, Dordrecht: Holland, 1978. Maxwell, J.A. "Understanding Validity in Qualitative Research," Harvard Educational Review (62:3) 1992, pp 279-299. Medin, D.L., and Atran, S. "The Native Mind: Biological Categorization and Reasoning in Development and Across Cultures," Psychological Review (111:4) 2004, pp 960-983. Medin, D.L., Lynch, E.B., and Solomon, K.O. "Are There Kinds of Concepts?," Annual Review of Psychology (51) 2000, pp 121-147. Mohr, L.B. Explaining Organizational Behavior Jossey-Bass, San Francisco, 1982. Monge, P.R. "Theoretical and analytical issues in studying organizational processes," Organization Science (1:4), November 1990, pp 406-430. Montealegre, R., and Keil, M. "De-Escalating Information Technology Projects: Lessons from the Denver International Airport," MIS Quarterly (24) 2000, pp 417-447. Morgeson, F.P., and Hofmann, D.A. "The Structure and Function of Collective Constructs: Implications for Multilevel Research and Theory Development," Academy of Management Review (24:2), April 1999, pp 249-265. Newman, M., and Robey, D. "A Social Process Model of User-Analyst Relationships," MIS Quarterly (16:2), June 1992, pp 249-266. Orlikowski, W.J., and Baroudi, J.J. "Studying Information Technology in Organizations: Research Approaches and Assumptions," Information Systems Research (2) 1991, pp 128. Pare, G., Bourdeau, S., Marsan, J., Nach, H., and Shuraida, S. "Re-examining the Causal Structure of Information Technology Impact Research," European Journal of Information Systems (17) 2008, pp 403-416. Pentland, B.T. "Building Process Theory with Narrative: From Description to Explanation," Academy of Management Review (24:4) 1999, pp 711-724. Peterson, M.F. "Embedded Organizational Events: The Units of Process in Organization Science," Organization Science (9:1), Jan-Feb 1998, pp 16-33. Poole, M.S., Van de Ven, A.H., Dooley, K., and Holmes, M.E. Organizational Change and Innovation Processes Oxford University Press, New York, 2000. Porra, J. "Colonial Systems," Information Systems Research (10:1), March 1999, pp 38-69. Ramiller, N.C., and Pentland, B.T. "Management Implications in Information Systems Research: The Untold Story," Journal of the Association for Information Systems (10:6), 2009, pp 474-494. Rescher, N. Process Metaphysics: An Introduction to Process Philosophy State University of New York, Albany, NY, 1996. Rivard, S., and Lapointe, L. "A Cybernetic Theory of the Impact of Implementors' Actions on User Resistance to Information Technology Implementation," Proceedings of the 43rd Hawaii International Conference on Information Systems, IEEE, Hawaii, 2010, pp. 1-10. Sabherwal, R., and Robey, D. "Reconciling Variance and Process Strategies for Studying Information Systems Development," Information Systems Research (6:4), 1995, pp 303327. 41

Salmon, W.C. Causality and Explanation Oxford University Press, New York, USA., 1998. Sarker, S. "Technology Adoption by Groups: A Test of Twin Predictions Based on Social Structure and Technology Characteristics," Annual Workshop of the Special Interest Group on Adoption and Diffusion of Information Technologies (SIG ADIT), Milwaukee, Wisconsin, 2006, pp. 1-25. Sawyer, R.K. Social Emergence: Societies as Complex Systems Cambridge University Press, Cambridge, 2005. Schutz, A. "Concept and Theory Formation in the Social Sciences," in: Collected Papers, M. Natanson (ed.), Martinus Nijhoff, The Hague, 1973, pp. 48-66. Schwandt, T.A. Qualitative Inquiry: A Dictionary of Terms Sage, 1997. Seddon, P.B. "A Respecification and Extension of the DeLone and McLean Model of IS Success," Information Systems Research (8:3), September 1997, pp 240-253. Shaw, T., and Jarvenpaa, S.L. "Process Models in Information Systems," in: Information Systems and Qualitative Research, J.L. A.S. Lee, & J.L. DeGross, eds. Information Systems and Qualitative Research (ed.), Chapman and Hall, London, 1997, pp. 70-100. Shoemaker, P.J., Tankard, J.W., and Lasorsa, D.L. How to Build Social Science Theories Sage Publications, Thousand Oaks, CA, 2004. Straub, D.W. "Validating Instruments in MIS Research," MIS Quarterly (13:2) 1989, pp 147169. Straub, D.W., Ang, S., and Evaristo, R. "Normative Standards for IS Research," DataBase for Advances in Information Systems (25:1) 1994, pp 21-34. Suppe, F. "The Search for Philosophical Understanding of Scientific Theories," in: The Structure of Scientific Theories, F. Suppe (ed.), University of Illinois Press, Urbana, 1977, pp. 3241. Sutton, R.I., and Staw, B.M. "What Theory is Not," Administrative Science Quarterly (40) 1995, pp 371-384. Trist, E.L. "The Evolution of Socio-Technical Systems," in: Perspectives on Organization Design and Behavior, A.H. Van de Ven and W.F. Joyce (eds.), Wiley, New York, 1981. Truex, D., Holmstrom, J., and Keil, M. "Theorizing in Information Systems Research: A Reflexive Analysis of the Adaptation of Theory in Information Systems Research," Journal of the Association for Information Systems (7:12), Dec 2006, pp 797-821. Van de Ven, A. Engaged Scholarship: A Guide for Organizational and Social Research Oxford University Press, Oxford, UK, 2007. Van Maanen, J., Sorensen, J.B., and Mitchell, T.R. "The Interplay Between Theory and Method," Academy of Management Review (32:4), Oct 2007, pp 1145-1154. von Bertalanffy, L. General Systems Theory Braziller, New York, 1968. Walsham, G. "The Emergence of Intepretivism in IS Research," Information Systems Research (4) 1995, pp 376-394. Weber, R. "Editor's Comments: Theoretically Speaking," MIS Quarterly (27:3), 2003, pp iii-xii. Webster, J., and Watson, R.T. "Analyzing the Past to Prepare for the Future: Writing a Literature Review," MIS Quarterly (26:2), June 2002, pp xiii-xxiii. Wheeler, B.C. "NEBIC: A Dynamic Capabilities Theory for Assessing Net-Enablement," Information Systems Research (13:2) 2002, pp 125-146. Zacks, J.M., and Tversky, B. "Event Structure in Perception and Conception," Psychological Bulletin (127:1) 2001, pp 3-21.

42