Design, Research, and Design Research: Synergies ...

3 downloads 11902 Views 1MB Size Report
educational technology in comparison to other design fields can .... education, as opposed to a scientific approach that uses a ..... Tech Trends, 41(5), 22-27.
Design, Research, and Design Research: Synergies and Contradictions ,;I,)

Wayne A. Nelson

, ;,. ,,

I

Notions of design, research, and design research in the field of educational technology are quite different from conceptualizations held by other design fields. Examining the ways that research is conducted and used in educational technology in comparison to other design fields can provide novel insights into how research and design practice can be better integrated. These insights may extend the conceptualization of design research beyond the typical model of scholarly research, and suggest implications for instructional design curriculum, pedagogy, and future research.

An Identity Crisis in Instructional Design Research For the past decade or more, significant changes in the study of learning and instruction have occurred. The development of the Learning Sciences and related advances in educational inquiry and research foci (Rowland, 2007) have produced an identity crisis in the field of instructional design. Traditional instructional design processes are no longer seen as viable by many (Gordon & Zemke, 2000; Sims, 2006), as a variety of alternative approaches to the design of learning and instruction are being pursued. Attention is being paid in many disciplines to how designers think and work, and how they employ research activities to help them engage with design situations. But there are significant differences between the ways that research is defined, taught, and conducted in the field of instructional design as compared to other design fields. This is due to the different purposes for which research is utilized in practice as opposed to academic settings. As Stappers (2007) notes:

Wayne A. Nelson currently is Professor and Faculty Fellow for Online Learning and Assessment at Southern illinois University Edwardsville. His research interests have included investigations of the characteristics of learning with various interactive media, and the design processes and expertise employed by designers to create learning activities and environments (e-mail: wnelson@ siue.edu).

A lot of debate has been devoted to the relation between design and research, and a consensus outcome has not been established ... One problem in the debate is that ... [the debate] is often carried on at a level of abstraction which tends to confuse rather than enlighten, because generic terms such as 'research' and 'design' carry more implicit connotations than explicit denotations (p. 81 ).

In part, our identity crisis is revealed by the confusing and sometimes contradictory definitions, terminology, and knowledge-based descriptions that have been suggested for instructional design. So before proceeding with descriptions of new forms of inquiry that represent the synergies between design and research, it seems helpful to define terms in order to examine the various ways that design research can be employed by instructional designers. In essence, research activities are undertaken to discover and utilize new knowledge, following studious processes in order to discover and interpret facts, revise theories, or apply new knowledge to practice. Various forms of research have been outlined, including basic research, applied research, and developmental research (National Science Foundation, 2012). These definitions can produce many nuances in terms of the various purposes for which research activities are undertaken, but it appears that most research in instructional design is applied or developmental. Even with all its nuances, it seems easier to formulate a definition of research than a definition of design. To design (verb) might mean to contrive, create, devise, fashion, execute, or construct, while a design (noun) might be defined as a mental scheme, a preliminary sketch, or an outline. These definitions can be a source of confusion (Hjelm, 2005), because the noun form of design involves a representation or a plan, while the verb form refers to human activity that results in a representation or plan. Generally, people in academic careers use "scientific research" approaches to develop and test theories. For years, luminaries have led instructional design research by employing forms of basic research to develop a science of instruction. Many scholars believe that instructional design is a scientific field (Reigeluth, Gunderson, & Merrill, 1994; Richey, Klein, & Tracey, 2011) and that research should provide empirical evidence in order to develop sound educational theories. The research that has resulted from this interpretation explains theories of instruction, not instructional design. Many beli€ve that there cannot really be a science of instructional design (e.g., Kember & Murpby, 1995). Reliance on empirical research to study design activities in order to validate theoretical models for instruction is surely appropriate, but it may never produce a general theory of or model for instructional design. One reason is related to the long debate about the nature of the field of instructional design. Do instructional designers use scientific principles and contribute to the development of theory as they engineer instructional products intended

EDUCATIONAL TECHNOLOGY /January-February 2013

3

to produce learning by students? Or, do instructional design"ers employ "designerly thinking" (Cross, 2001; Rowland, 2007) to solve "wicked problems" (Rittel & Webber, 1973) that can't be approached using scientific thinking, yet produce the same kind of products and learning outcomes for students? Design thinking is not scientific thinking, even though both can be creative. Design is concerned with "making," while science is concerned with "finding" (Owen, 2007). Design iS''Il form of problem solving Uonassen, 2008; Nelson, Magliaro, & Sherman, 1988) that requires extensive effort to understand the problem to be solved and to identify constraints that guide solution possibilities (Schqn't 1983). Designers work by synthesizing ideas within r,eal-world situations that involve creating artifacts and managing the environment, while scientists think analytically within an abstract, symbolic world. As Owen (1998) has noted: "Design is not science, and it is not art-or any other discipline. It has its own purposes, values, measures, and procedures .... ln short, there is little to point to as a theoretical knowledge base for design ... Knowledge is generated and accumulated through action. Doing something and judging the results is the general model" (pp. 1 0-11). With similar conceptions about what the human endeavor of design involves, other design fields are coming to the conclusion that design research should work to develop theories on design and theories for design in order to better understand the nature of design activity and to improve design practices (Zimmerman, Stolterman, & Forlizzi, 201 0). While some argue that both "normal" and design sciences are necessary in education (Sloane, 2006), it may be more useful to view instructional design as a design field so that we may more effectively study the actual practice of designers, and therefore gain a better understanding of what designers do and how they do it (Gibbons, 2003; Rowland, 1993). This article joins other scholars (e.g., Reeves, 2000) in promoting an alternative view 9f instructional design research that focuses on applied and development research activities in order to develop grounded theories based in design practice. It is necessary to note at this point that the term design thinking used in this article refers to the types of thinking and problem solving in which designers are engaged (i.e., designerly thinking, as described by Cross, 2001 ). This is different from the use of the term in the design studies literature, or more recently in educational circles. In these cases, design thinking refers to an approach where, initially, design processes were used as a means to address social conditions and sustainability (Moggridge, 2008). While this orientation to design has made a significant impact in businesses that need to promote and sustain innovation, and may eventually make great inroads in education (d.schooi-Stanford University, 2012), it is not the central focus of this article.

4

So What Is Design Research? Discussions of the nature of research within instructional design are not unique to this article. The evolution of various types of design research has been described using many labels, including action research, design experiments, developmental research, development research (van den Akker, 1999), design and development research (Richey & Klein, 2011 ), and engineering research (Edelson, 2006). An early attempt to review developmental research within instructional design (Richey & Nelson, 1995) suggested several purposes for research activities, and hinted at some of the different categories described in this article. More recently, development research was compared with other approaches typically used in the field of instructional design (van den Akker, 1999), suggesting that while methods might be similar, the nature of the knowledge gained from development research is in the form of design principles and heuristics derived through formative evaluation of "successive approximations of interventions in interaction with practitioners" (p. 8). Edelson (2006) also noted different kinds of thinking and research goals that are inherent in an engineering approach to design research in education, as opposed to a scientific approach that uses a theory-testing paradigm. · These developments happened as the larger field of educational research moved toward accepting design research paradigms (Bannan-Ritland, 2003; Barab & Squire, 2004). Debate about the nature, conduct, and validity of design research in education centered on methodological validity, whether conclusions could be generalized, and whether "science" could be advanced using design research (Collins, Joseph, & Bielaczyc, 2004; Shavelson, Phillips, Towne, & Feuer, 2003). Currently, the debate has largely ended, and design research is now generally accepted, albeit with the new label of "designbased research." An alternative conceptualization of the kinds of design research that are possible within the instructional design field is proposed in this article, using a classification of research during design, research about design, and research through design. The proposed categories exemplify different reasons for conducting research within a wide range of design situations, from pure theory to pure practice. Research during design happens as, part of an integral part of all design processes, where research activities are utilized in support of design practices in a particular context (Stapleton, 2005). Research about design is undertaken in order to understand, inform, and improve design practices. This form of design research generates knowledge about the effectiveness of design models, methods, and tools, as well as how designers employ various design processes and develop expertise as designers. Research through design is focused on meta-level questions with the larger objective of creating theoretical

EDUCATIONAL TECHNOLOGY/january-February 2013

knowledge, rather than creating a solution for a particular situation (Frankel & Racine, 201 0). The categories roughly correspond to the "mindsets" of designers suggested by Sanders (2008), and to the initial classification of research for, about, and through design proposed by Frayling (1993). Though not wholly precise, the descriptions in the figures which follow are suggested to help clarify the potential purposes of the various forms of design research that are possible. : ,;,~., This proposed classification may better meet what has been envisioned for design research in instructional design (Reeves, Herrington, & Oliver, 2005). It. can provide a better focus to develop new tools and m,t~ods that will facilitate better designs for learning (Mor & Winters, 2007), as well as exploring and refining theorie? of learning and curriculum (Nelson, Ketelhut, Clarke, Bowman, & Dede, 2005). The classification is also better aligned with design research in other fields, where there is a deep connection between design and research (Stapleton, 2005), and skills in each area are necessary for successful innovation (Stappers, 2007). A focus on the various small words that can connect design and research (i.e., "during," "about," and "through") will bring clarity to our conceptualization and communication about design research. It will also give designers a larger array of choices for design research in practice, as well as developing critical understanding of what designers do and the impact on learning that their products create.

Research During Design Research activities that are an integral part of any design process as practiced should be viewed as design research. Further, design research occurs in all phases of a project, not just in one particular part of a design and development process labeled as "design" (see Bichelmeyer's [2005] critique of the ADDIE model). These assumptions require a broader definition of research as a systematic search or investigation to produce new knowledge, rather than "scientific" r~search as practiced by academic scholars. Instructional designers employ techniques for needs analysis, content analysis, learner analysis, etc., but usually don't consider these activities as design research, even though research methods are employed (Boling, 2005). Some of the areas of design and development research described by Richey and Klein (2011) are similar to research during design, in particular, product development research that occurs during the design and development of an instructional product or program. But a major difference between their description and that proposed in this article is that research during design would not be published. The findings of research during design are internal to the design process (see Figure 1), and inform the design decisions being made about project needs, requirements, conceptualizations, testing, etc. Results from such research may exist in some form of documentation to analyze what happened

naturalistic

Design

\. (')

Figure 1. Research during design.

and why, but the results would never be publicly shared after the completion of the project. This is not to say that designers do not engage in research activities that codify and share "lessons learned." They definitely utilize reflection in action (Schon, 1983) and post hoc reflection about their design activities. The point is that publishing results based on "lessons learned" in a particular project or with a particular tool is better considered research about design (as discussed below) rather than research during design, if for no other reason than for clarity in our discussions of the various types of design research. One look at the contents of Brenda Laurel's book entitled Design Research (Laurel, 2003) reveals many ways that research activities are integrated into the design process in other design fields. Specific tactics and design research methods (not models, and not research methods) that are commonly employed in user experience design, product design, interactive design, etc., are described. Recommendations are included about how to formulate design research plans for moving the design process forward, from initial "up front" considerations through design conceptualizations, prototype testing and, finally, evaluation. The sampling of methods shown in Table 1 focuses on some of the methods that are used to examine existing products or situations in order to guide inquiry, but the methods themselves are processes. Settings for these kinds of research during design can be in labs, remote locations, or by using "guerilla research" tactics (Maier, 2011 ). The choice of design research method and location can also depend upon the type of data sources, research method (qualitative vs. quantitative), and context of use (Rohrer, 2008). Some of these methods come from usability research and are becoming an important

EDUCATIONAL TECHNOLOGY/January-February 2013

5

Table 1. Some design research methods and tactics (adapted from Bruseberg & Mcdonagh-Philp, 2000;

analytic

cumuta~.:

"lie

Laurel, 2003; Roschuni, 2009).

Design "Phase"

Methods or Tactics

"Up front" analysis

Product evaluations, usability testing (existing product), stakeholder interviews, f6cus group interviews, user interviews focused on existing situation/product, expert interviews, literature reylews, taxonomies, card sortlng, observation, docume~i analysis

Figure 2. Research about design. Requirements definition

Conceptual design

Personas, design workshops, dramatic/theatrical/narrative performance or document (scenarios), design games, experience cards, activity/task analysis, camera journal, empathy probes Ideation: Design probes, use cases, layered elaboration, design-incontext Communication of ideas: storyboards, "focus troupes," prototype trials, experience models and maps, conceptual design templates, content mapping, collaborative authoring (wikis)

activity for research during design in the practice of instructional design as well (Baek, Cagiltay, Boling, & Frick, 2008). The examples shown in Table 1 are research during design because the research is conducted with the intent to gather information, identify opportunities and constraints, understand the context, and make design, production, and evaluation decisions. It is research completed as part of the design process.

Research About Design In the field of instructional design, research about design has explored and val idated the effectiveness of design processes, developed new design tools, and examined the characteristics and utility of various artifacts produced as part of design activities. The goal of this kind of research is to "develop a detailed and unified understanding of the human activity of design or design related activities" (Forlizzi eta!., 2009, p. 2892). It is a "form of design research that is exploratory of the process and the materials, it is through making ... that new ideas can

6

be tested, to develop a critical understanding of what designers do and the objects that are created by design" (Burdick, 2003, p. 82). Research about design involves the analysis of design products and activities, and it operates from outside of design practice (see Figure 2), where observations that keep the subjects and objects of design at a distance can be made in order to produce "universally verifiable findings" (Schneider, 2007, p. 214). Designed products are studied with research about design in terms of the efficacy of methods, models, or tools used to design and produce the products. For example, various representations or "design sketches," written documentation, and other artifacts produced during design can be studied to determine the utility of the artifacts as design tools, as well as to help determine the nature and effectiveness of the design process and designer cognition (Baek et a/., 2008; Purcell & Gero, 1998). Research about design has studied ways that a visual language for instructional design might be developed to express design elements as well as to document and communicate design features (Waters & Gibbons, 2004).With a process focus, models and tools as well as the activities of individual designers or collaborative design groups have been studied. Considerable research regarding instructional design automation with tools and with learning objects has been completed, and more recently, with tools for learning design (Conole, 201 0) and the use of design patterns (Frizell & Hubscher, 2002). New design models to guide learning experience design activities have been proposed (Chen, 201 0; Cilesiz, 201 0; Park, 2008), while design culture and design communication processes have also been studied (l