University of Southern Indiana Evansville, Indiana ...

International Journal for the Scholarship of Teaching and Learning http://www.georgiasouthern.edu/ijsotl Vol. 5, No. 2 (July 2011) ISSN 1931-4744 @ Georgia Southern University

Assessing the Pedagogical Impact of the VaNTH Engineering Research Center on Faculty and Postdoctoral Professionals Monica Cox Purdue University West Lafayette, Indiana, USA [email protected] James Cawthorne Purdue University West Lafayette, Indiana, USA [email protected] Nathan McNeill University of Florida Gainesville, Florida, USA [email protected] Osman Cekic Canakkale Onsekiz Mart University Canakkale, Turkey [email protected] Matthew Frye CTS Automotive Products Elkhart, Indiana, USA [email protected] Melissa Stacer University of Southern Indiana Evansville, Indiana, USA [email protected] Abstract From 1999 to 2007, the Vanderbilt-Northwestern-Texas-Harvard/MIT (VaNTH) Engineering Research Center focused on improving bioengineering education through the applications of learning science, learning technology, and assessment and evaluation within the domain of bioengineering. This paper discusses results from a survey to explore the impact of the VaNTH experience on participating faculty and postdoctoral professionals. The results note that respondents differed in their familiarity with and applications of dimensions of the “How People Learn” framework and in their operationalization of effective instruction after their participation in VaNTH. Implications for teaching and learning with the context of a Center model are discussed along with next steps for exploring the experiences of faculty and professionals engaged in the VaNTH ERC. Keywords: pedagogy; faculty professional development; Engineering Research Center; “How People Learn” Introduction The White House, industry, and academia joined together in 1984 to request that the National Science Foundation create the Engineering Research Center (ERC) program.


2

With a focus on revitalizing industry, ERCs were multi-university, multi-disciplinary collaborative centers that were each created with $10 million in seed funding from the National Academies of Engineering (Boardman & Bozeman, 2007). Since their founding, ERCs have developed into places to nurture innovations and ideas, to produce bettereducated individuals, and to promote collaborations among educational institutions, industry, and the government (Suh, 1986). Previous studies have explored the impacts of ERCs upon a variety of stakeholders including faculty and students. In a report on the impact that seventeen ERCs had on institutional and cultural norms at participating universities, Ailes, Feller, and Coward (2001) identified several outcomes pertaining to the roles of ERC-affiliated faculty. They found that: •

• •

•

Faculty participating in ERCs appreciated the interdisciplinary nature of the ERC structure and the resulting impact on their research even though the collaborative nature of the ERC structure was contrary to traditional, individualistic notions of promotion and tenure. The structure of ERCs allowed faculty to engage both graduate and undergraduate students from a variety of disciplines in the research process. In integrating industry, faculty sometimes faced the challenge of educating university research administrators programs about the nature of industry funding and contracts. Faculty engaged with industry partners were able to provide opportunities for students to connect theory and practice.

Additional benefits of ERCs as reported by Ailes et al. (2001) include the development of new courses or course curricula, increased enrollment in newly developed or improved courses, and the creation of new degree programs within the academic units of ERC universities. Boardman and Bozeman (2007), writing about the impact of ERCs upon faculty, report that untenured faculty participating in ERCs were particularly susceptible to strain while learning to balance their responsibilities within ERCs with those of their academic units. Junior faculty engaged in the tenure process had to learn how to balance both ERC and departmental duties more strategically than senior faculty members. In an effort to understand more about the individual experiences of faculty and postdoctoral researchers participating in ERCs, the current exploratory study was conducted with a sample of respondents from the Vanderbilt-Northwestern-TexasHarvard/MIT (VaNTH) ERC for Bioengineering Educational Technologies, an ERC focused on the implementation of the educational principles of the “How People Learn” (HPL) framework (Bransford, Brown, & Cocking, 1999). The research question behind this study asked: What perceptions did respondents hold about the impact of participation in VaNTH on their professional development? Respondents in this study completed a survey which questioned them about their professional development experiences before, during, and after their affiliation with the VaNTH ERC. The survey focused specifically on how respondents implemented elements of the HPL framework over time. Implications for professional development of faculty and postdoctoral professionals are described based upon the survey results. Background The VaNTH Engineering Research Center The Vanderbilt-Northwestern-Texas-Harvard/MIT (VaNTH) ERC for Bioengineering Educational Technologies was created in 1999 to “unite educators and engineers, in


3

industry and academia, to develop curricula and technologies that will educate future generations of bioengineers” (VaNTH, 2010). With a primary focus on integration of the “How People Learn” (HPL) framework principles with undergraduate bioengineering curricula, this multi-million dollar ERC brought together expertise in learning science, learning technology, assessment and evaluation, and bioengineering. The VaNTH ERC differed from other ERCs in that it was the first and only ERC funded to explore bioengineering education in combination with advanced technologies, cognitive science, and assessment and evaluation. During its eight-year existence, hundreds of faculty, postdoctoral researchers, undergraduate and graduate students representing the four primary institutions (Vanderbilt University, Northwestern University, the Texas at Austin, and Harvard University) as well as five additional institutions (the University of Wisconsin, Fisk University, the University of Texas-Pan American, the University of Memphis, and the University of Pittsburgh) engaged in a multitude of educational activities within the center (VaNTH ERC, 2008). Prior publications have explored the research impact of VaNTH (Cordray, Pion, Harris, & Norris, 2003), the development of VaNTH-inspired educational innovations (Roselli & Brophy, 2006), and the effects of VaNTH upon student populations (Martin, Rivale, & Diller, 2007). However, no research to date has reported the longitudinal impact of the VaNTH ERC upon faculty and postdoctoral respondents. For this reason the current paper presents self-reported outcomes from an exploratory study focusing on the professional development experiences of respondents before, during, and after their formal participation in the VaNTH ERC. Implementation of HPL Framework Principles in the VaNTH ERC VaNTH ERC researchers conducted several empirical studies to identify the pedagogical practices that would maximize the achievement of bioengineering students at VaNTHaffiliated universities (VaNTH ERC, 2008). The majority of these studies centered on the integration of the educational dimensions of the HPL framework. According to the HPL framework, an effective learning environment is simultaneously knowledge-centered, learner-centered, assessment-centered, and community-centered (Bransford et al., 1999). Knowledge-centered environments emphasize that students exhibit a deep understanding of course content as well as an ability to apply this knowledge; learnercentered environments build upon students’ preconceptions, misconceptions, and ideas about course concepts; assessment-centered environments provide opportunities for both formative and summative opportunities so that students and faculty can learn from one another; and community-centered environments engage students with peers inside the classroom as well as members of the larger community outside of the classroom. From its beginning, VaNTH focused primarily on the impact of HPL principles upon student learning and engagement, analyzing the HPL framework and its effectiveness among both undergraduate and graduate engineering populations. Subsequent studies have synthesized implementation methods and assessment tools in the hopes of confirming (both quantitatively and qualitatively) the positive benefits of implementing HPL framework principles in traditional bioengineering courses. Research on the implementation of the HPL framework in bioengineering has been published extensively. Birol, McKenna, Smith, Giorgio, and Brophy (2002) tested and implemented several biomedical engineering modules that incorporated principles from the HPL framework and the Star Legacy Cycle, an educational model (also developed by VaNTH researchers) that allows students to engage in an interactive cycle of learning that represents the integration of HPL principles (Schwartz, Lin, et al., 1999). Research on effective implementation of principles of the HPL framework then extended to modules in tissue engineering, biomechanics of human movement, and Fourier spectrum


4

analysis (Greenberg, Smith, and Newman 2003; Barr, Pandy, Petrosino, Austin, & Goldberg, 2004; Birol, Liu, Smith, & Hirsch, 2006). Several studies have supported the efficacy of the educational research projects carried out by VaNTH researchers. The final report published by VaNTH boldly claims that research reveals “that VaNTH sponsored innovations can be beneficial in enhancing the learning of students” (VaNTH ERC 2008, 37). Two types of research efforts were undertaken by VaNTH assessment and evaluation researchers to document the effectiveness of educational modules and courses developed by VaNTH bioengineering experts. These were: (1) surveys of students’ and instructors’ perceptions of the degree to which the four dimensions of the HPL framework were present in all VaNTH biomedical engineering courses (Cordray et al., 2003), and (2) direct observation of the pedagogical practices of instructors in selected VaNTH courses using the VaNTH Observation System (VOS), an observational system developed by VaNTH researchers (Harris & Cox, 2001). Via surveys, Cordray et al. (2003) directly compared biomedical engineering courses implementing HPL framework principles to traditional “non-HPL” courses, thereby highlighting the relationship between pedagogy and effective learning within courses taught at VaNTH institutions. Assessments of the impact of the HPL framework in biomedical engineering courses further provided a comparison of students’ experiences at multiple universities (Giorgio, Brophy, Birol, McKenna, & Smith, 2002). VaNTH researchers developed the VOS for direct observation of interactions within classrooms. This system provides a method for both quantifying and qualifying student engagement as well as the presence of elements of the HPL framework in interactions that occur within classrooms (Harris & Cox, 2001). Deployment of the VOS in various bioengineering classrooms at VaNTH institutions revealed that classrooms in which instructors intentionally incorporated elements of the HPL framework into their course designs exhibited a greater number of instances of collaborative group work and higherorder thinking and questioning than courses where instructors had not intentionally incorporated elements of the HPL framework (Cox & Cordray, 2008). Despite the prominent role of faculty within the VaNTH ERC, only a few research studies conducted by VaNTH researchers have focused specifically on faculty experiences. One such study by Cordray et al. (2003) used surveys to examine the instructional perceptions of instructors who had participated in the VaNTH ERC. Another study by McKenna and Yalvac (2007) used interviews with sixteen bioengineering faculty to identify differences in teaching strategies between participants and non-participants in the VaNTH ERC. In particular, this study explored relationships between faculty levels of teaching engagement and their approaches to teaching. Cox and Cordray (2008), as well as Cox (2009), identified pedagogical differences between three classes of faculty: (1) those who did not purposefully integrate HPL-based curricula in their courses (non-HPL faculty), (2) faculty who implemented HPL-based curricula for the first time (“novice” HPL faculty), and (3) faculty who had implemented HPL-based curricula over multiple semesters (“seasoned” HPL faculty). Cox and Harris (2010) explored differences in the pedagogical practices of pretenured and tenured faculty and found that pretenured faculty were more comfortable teaching using HPL framework principles than tenured faculty designated to teach HPL courses. VaNTH researchers noted that faculty exposed to HPL framework innovations differed in their pedagogical approaches compared to control groups with no exposure to HPL innovations. None of these studies, however, have explored (1) why faculty engaged in innovative teaching practices are more likely to use approaches linked to learner-centered pedagogy; (2) the impact of VaNTH curricular innovations upon VaNTH faculty, and (3) qualitative questions exploring why pedagogical differences exist between novice faculty and seasoned (i.e., tenured ) faculty who participated in VaNTH.


5

Methods The study presented in this paper explores the experiences of both faculty and postdoctoral professionals before, during, and after their affiliations with the VaNTH ERC and the impact these experiences had on these individuals. In particular, this study examines how respondents have implemented elements of the HPL framework both during and after participation in VaNTH. The findings reported in this paper were obtained from a web-based survey. Respondents One hundred thirty-three individuals who had participated in VaNTH at some time during its eight-year existence were invited to participate in this study. Because of the diversity of backgrounds of VaNTH respondents, anyone who was not a postdoctoral professional or a graduate student was classified in the “faculty” category. As a result, the final population for this study consisted of academic consultants, tenure-track faculty, academic staff, and postdoctoral professionals. After sending an initial request to respondents asking for confirmation of contact information, researchers narrowed the list of possible respondents down to 119 individuals. Invitations were then e-mailed to the individuals in this population requesting that they complete a web survey (Appendix). Thirty individuals responded to the invitation and completed the survey resulting in a final response rate of 25.2%. This is consistent with the median response rate for most web surveys (Kaplowitz, Hadlock & Levine, 2004; Trouteaud, 2004; Marcus, Bosnjak, Lindner, Pilischenko, & 2007). The titles, responsibilities, methods of recruitment to VaNTH, and length of participation varied across respondents. When asked to describe their titles in VaNTH, six defined themselves as educational module developers, and four defined themselves as researchers. Other respondents described their titles within VaNTH as bioengineering domain consultants, developers of web-based materials, course developers, and affiliates with the K-12 component of the ERC. Respondents also had engaged in multiple responsibilities within VaNTH – research (80%), teaching (50%), and administration (23%). Over two-thirds of respondents were recruited to VaNTH by an individual or group already engaged in the ERC. Finally, during the eight-year existence of the VaNTH ERC, 50% of survey respondents participated for more than three years, while 10% participated for less than a year. Data Collection The survey used in this study was created by the authors. This instrument was developed to obtain from respondents perceptions of both their past and present experiences with HPL-oriented instruction. Survey questions were constructed so that: (1) respondents could explain how they came to participate in the ERC, (2) respondents’ pre-VaNTH understandings of the HPL framework as well as concepts of effective teaching could be reconstructed, (3) respondents could comment on their experiences within VaNTH, particularly with respect to learning about the HPL framework and the STAR Legacy Cycle, (4) researchers could identify whether participation in VaNTH contributed to respondents’ continued use of the HPL framework in their post-VaNTH careers, and (5) if so, how? Nineteen closed-ended quantitative questions and five open-ended qualitative questions were developed to gather self-reported responses from respondents. The quantitative questions asked respondents to provide responses using a Likert scale. An initial draft of survey questions was developed and piloted with several researchers who had experience applying the HPL framework and were familiar with the unique features of the VaNTH ERC. These individuals also served as expert judges providing content validity to the survey. After the survey was piloted it was deployed on-line with a respondent


6

consent form as a cover page. Approval for this study was obtained from the Purdue University Institutional Review Board (IRB). Data Analysis Responses to the quantitative, closed-ended survey questions were analyzed using chisquare tests to identify statistically significant items and/or trends. Responses to the open-ended questions were analyzed using open coding and grounded theory (Lincoln & Guba, 1985). During analysis, the resolution of responses to several of the quantitative questions was reduced in order to more easily identify trends in the data. This was accomplished by reducing the number of Likert scale items on several questions. The process for coding, identifying, and making assertions from the qualitative data began with several initial readings of the data in order to become immersed in participants’ responses. After several passes, the data were parsed using key words that captured the essence of each response. An entire response to a question was coded as opposed to single key words or phrases. Coded responses were then grouped into thematic categories. Assertions reported in this paper are based upon how participants described their ideas of effective teaching before, during, and after participation in the VaNTH ERC. Results This section presents results from the open- and closed-ended survey questions in two separate sections. Quantitative Results To explore participants’ perceptions of the impact that participation in VaNTH had on their professional development, responses from several questions were combined to create Figures 1-4. Significantly statistical findings are presented later in this section of the paper. Figure 1 display shows respondents’ familiarity with the four dimensions of the HPL framework prior to engagement in the VaNTH ERC. Familiarity was rated on a four-point Likert scale where 1=not at all familiar, 2=not too familiar, 3=somewhat familiar, and 4=very familiar. Responses of 1 and 2 have been presented as “not familiar” and responses of 3 and 4 have presented as “familiar”. The HPL dimension that participants were most familiar with prior to participation in VaNTH was the learner-centered dimension.


7

Figure 1. Respondents’ familiarity with each of the dimensions of the “How People Learn” framework prior to participation in VaNTH.

Respondents were also asked how frequently they use HPL framework elements in their current educational and research activities (Figure 2). Again, frequency was rated on a four-point Likert scale where 1=never, 2=seldom, 3=frequently, and 4=always. The majority of respondents reported that they frequently or always use HPL framework elements. In addition, respondents reported that they are most likely to apply the learner-centered dimension. Figure 2. Respondents’ frequency of use of “How People Learn” framework elements within their current activities.

The survey also asked participants to rate the impact of participation in VaNTH on their research interests, teaching, interactions with students outside of class, interactions with colleagues, and career choices (Figure 3). Although the majority of respondents identified participation in VaNTH as having at least some impact in each area,


8

respondents felt that the area of greatest impact had been teaching. In contrast, the area in which participants reported the least impact was career choices. Figure 3. Impact of participation in VaNTH upon respondents.

A number of additional findings emerged as a result of Chi-square tests. Respondents who engaged in research while participating in VaNTH report using the assessmentcentered dimension in their current work more often than those who did not engage in research as participants in VaNTH (chi-square=8.611, p