An online web application was constructed using PHP, jQuery, and HTML, allowing .... associated with 81 institutions (with one individual acting as a freelancer).
Post-interdisciplinary frames of reference: Exploring permeability and perceptions of disciplinarity in the social sciences Timothy D. Bowman*, Andrew Tsou, Chaoqun Ni, and Cassidy R. Sugimoto School of Informatics and Computing Indiana University Bloomington {tdbowman; atsou; chni; sugimoto}@indiana.edu *corresponding author ProQuest Dissertations and Theses database contains records for approximately 2.3 million dissertations conferred at 1,490 research institutions across 66 countries. Despite the scope of the Dissertations and Theses database, no study has explicitly sought to validate the accuracy of the ProQuest SCs. This research examines the degree to which ProQuest SCs serve as proxies for disciplinarity, the relevance of doctoral work to doctoral graduates’ current work, and the permeability of disciplines from the perspective of the mismatch between SCs and disciplinarity. To examine these issues we conducted a survey of 2009-2010 doctoral graduates, cluster-sampled from Economics, Political Science, and Sociology ProQuest SCs. The results from the survey question the utility of traditional disciplinary labels and suggest that scholars may occupy a post-interdisciplinary space in which they move freely across disciplinary boundaries and identify with topics instead of disciplines. INTRODUCTION AND BACKGROUND Disciplines are conceptual entities that are institutionalized into academe in the forms of schools, departments, and centers. As such, these units are sustained by the propagation of new scholars under the same disciplinary label (Turner, 2000). This label serves as a symbol of students’ knowledge and skills, as well as facilitating the exchange of students on the academic labor market (Abbott, 1999). However, given the emphasis on interdisciplinarity from institutions and funding agencies (e.g., Morillo, Bordons, & Gomez, 2003), it is no wonder that disciplinary lines are being continually crossed as doctoral students from one discipline are hired on the faculties of new disciplines (Sugimoto, Ni, Russell, & Bychowski, 2011). Furthermore, many of these doctoral students go into non-academic sectors— perhaps only partially relying on the domain expertise of their doctoral work. Studies of disciplinary mobility frequently rely on the disciplinary classification systems embedded in various indexing systems. For example, citation indices such as the Web of Science (WoS) and Scopus are often used to extract data on disciplines. The disciplinary variables are operationalized as the Subject Categories (SCs) of these databases—that is, SCs are proxies for disciplines. Criticisms of the validity of SCs as proxies for disciplinarity have been raised across many decades (e.g., Garfield, 1978; Ni, Sugimoto, & Jiang, 2013). However, subject categories for major databases such as PubMed and WoS remain the gold standard in evaluative bibliometrics, and practices of using these as proxies for micro and macro evaluations of the system of science persist (e.g., Leydesdorff, Carley, & Rafols, 2013; Leydesdorff & Rafols, 2009; Leydesdorff, Rotolo, & Rafols, 2012). These issues are intensified by the growth of interdisciplinarity across sectors. Research pertaining to interdisciplinarity has become more common in the past forty years with “ “a corresponding increase in the discussion of interdisciplinarity across disciplinary, professional, and general literature” (Klein, 1990, p. 38). Porter and Rafols (2009) have suggested that “science is indeed becoming more interdisciplinary, but in small steps – drawing mainly from neighboring fields and only modestly increasing the connections to distant cognitive areas” (p. 719). However, other studies suggest the ubiquity of interdisciplinarity in modern scholarly activities. Even disciplines such as physics can no longer be
considered to be “a single, isolated discipline” (Klein, 1993, p. 201);“modern research knowledge” is no longer “exchange restricted within narrow silos” (Porter & Rafols, 2009, p. 740). The degree to which disciplines interact has been described in levels of “permeability” (Baumann, 1975, p.15; Klein, 1996, p. 38). One investigation of permeability has analyzed the degree to which doctoral students from one field are hired in another and begin training new students (Sugimoto, Ni, Russell, & Bychowski, 2011). However, few studies have used dissertation data as a platform for studies of disciplinarity and interdisciplinarity. One large-scale index providing information on doctoral education is the ProQuest Dissertations and Theses database. This dataset contains records for approximately 2.3 million dissertations conferred at 1,490 research institutions across 66 countries (Ni & Sugimoto, 2012), a scope that “cover[s] 40% of all dissertations from major universities” (Andersen & Hammarfelt, 2011, p. 374). This represents the largest historical database of dissertation and theses, covering works as far back as the early nineteenth century (Sugimoto, 2014). Despite the scope of the Dissertations and Theses database, it has rarely been used to analyze the growth and connectivity of disciplines (exceptions include: Andersen & Hammarfelt, 2011; Ni & Sugimoto, 2013a; Ni & Sugimoto, 2013b; Ying & Xiao, 2012), and no study has explicitly sought to validate the accuracy of the ProQuest SCs. To investigate these issues, we conducted a survey of recent doctoral graduates, cluster-sampled from three ProQuest SCs: Economics, Political Science, and Sociology. The resultant findings were supplemented with data unobtrusively gathered about individuals whose dissertations were classified as being from Chemistry, Computer Science, and Biology. The primary objective was to investigate the degree to which ProQuest SCs could serve as proxies for disciplinarity. The secondary objective was to analyze the relevance of doctoral work to doctoral graduates’ current work and the degree to which they use theories, methods, and literature from their home discipline and other disciplinary areas. Finally, the work sought to analyze the permeability of disciplines from the perspective of the mismatch between SCs and disciplinarity. This work serves to examine the use of ProQuest SCs for disciplinary analyses, as well as to advance our understanding of interdisciplinarity in the careers of doctoral graduates. Implications from this research can be used to inform practices and policies of higher education and indexing of scholarly communication. METHODS Subject Categories. When a doctoral student submits a dissertation to ProQuest, they are instructed to select relevant subject areas. These suggestions are then used by ProQuest dissertation editors, who assign SCs to the dissertation. As noted by a ProQuest representative, “we use the author’s subjects unless our scope for a subject is entirely different from the author’s interpretation of it (as sometimes occurs). Subjects are assigned based on the dissertation topic, not the university department, although naturally those are most often in sync” (McLean, 2013). Each dissertation in the database is associated with at least one SC, although an increasing proportion is associated with more.
2
Figure 1. Data schema of ProQuest dissertation Each SC is identified by a four-digit unique identification number, known as the SCID. Figure 1 provides a schema of a dissertation assignment to more than one SC. Figure 2 lists an example dissertation with ID=15 in the ProQuest database, as well as the three subject categories (identified by SCID) to which it is assigned. Each subject category listed in Figure 2 has two levels: the more general first level category and the qualifying second level category. The first level subject category is considered as a proxy for the discipline, while the second level subject category is considered a proxy for the specialty. Therefore, dissertation 15 is assigned to two disciplines and three specialties. It is a multi-disciplinary dissertation, and can be considered simultaneously as a sociology and health science dissertation. The first item listed (SCID 0344; Sociology) is considered to be the primary category (McLean, 2012).
Figure 2. Example dissertation w/ multiple subject categories Data. We constructed a sampling frame of all dissertations completed in 2009 or 2010 for each of three disciplines: Economics, Political Science, and Sociology. The time frame was selected in order to ensure that recent graduate students had sufficient time to transition from doctoral work into employment. A dissertation was considered as part of the discipline if it contained the disciplinary label as at least one of the first level SCs (not necessarily as the primary SC; that is, where the discipline was listed in the first 3
position). Five hundred dissertations from each SC were randomly selected for inclusion. Given that most dissertations from this time period are associated with multiple SCs, we resampled to ensure that dissertations were only assigned to one discipline. The three disciplines were selected in order to examine an array of social sciences. Identifying sampling frame. An online web application was constructed using PHP, jQuery, and HTML, allowing a research assistant (RA) to search, retrieve, and record additional information about each subject by using the information in ProQuest as a starting point. The RA then attempted to find current information about the subject by examining the links on the first three pages returned from a Google search. The Google searches were conducted using the person’s first name and last name, their dissertation title, or both. We collected information about each subject’s current URL, current email, current departmental affiliation, and current CV link; all of the new information was saved in a PostgreSQL database and linked to the ProQuest data using bridge tables. This allowed us to identify contact information and obtain additional contextual information regarding career trajectories. Contact e-mail addresses for each person could not be located. Therefore, the final sample included 348 individuals in political science, 338 individuals in economics, and 283 individuals in sociology. Questionnaire. The questionnaire was designed to gather a) demographic information, b) career information, and c) information regarding disciplinary identity: that is, the degree to which the individual self-associated with the discipline in which they were trained (see Appendix A for the full questionnaire—note that not all questions were utilized in the present study). The questionnaire was pilot-tested with one faculty member and four graduate students and built using Qualtrics 1 survey software. The questionnaire was available electronically, and the 969 participants were invited to participate between March 29 and April 18, 2013. No reminders were sent as the use of follow-ups has been shown to be negatively related to response rates (Anseel, Lievens, Schollaert, & Choragwicka, 2010). The response data were exported from Qualtrics into Excel and SPSS for further evaluation. The main analysis of the survey results was to determine the degree to which the exact disciplinary labels were used by respondents to describe their area of work. That is, an exact match between disciplinary labels would be one in which the discipline under study was Economics and the respondent said that they received their degree in Economics. A close match would be one that demonstrated the direct application of the subject area (for example, economics of education). Non-matches were those that were either specialized (e.g., forecasting) or distinct disciplines (e.g., History). Validation exercise. Data from three additional disciplines were selected as a validation metric. Using the same mechanism as for the three social science disciplines, 100 dissertations for each of three disciplines (Biology, Chemistry, and Computer Science) were selected at random. These were selected from the same timeframe as the initial dataset and on the basis of the same criteria (that is, containing at least one instance of the SC, but not necessarily in the primary position). We then conducted a series of Google searches to identify the current departmental affiliations of these dissertations’ authors, as well as to identify the discipline in which they received their doctorate degree. RESULTS This section is organized into three main parts. The first part provides a basic description of all ProQuest dissertations (in the entire database, not just in the timeframe under analysis) in the three SCs so as to provide context. The second part provides descriptive data regarding the respondents. The last part is an analysis of the questionnaire. 1
www.qualtrics.com
4
ProQuest description. There were a total of 96,434 dissertations (fincluding all of ProQuest dissertations and theses until 2011) that included Sociology as a SC in ProQuest. Sociology was associated with 13 second-level SCs (specialities) (Table 1). Table 1. Specialities of Sociology by number of occurrences Specialty Sociology, Individual and Family Studies Sociology, General Sociology, Social Structure and Development Sociology, Criminology and Penology Sociology, Public and Social Welfare Sociology, Industrial and Labor Relations Sociology, Theory and Methods Sociology, Demography Sociology, Organizational Theory Sociology, Sociolinguistics Sociology, Environmental Justic
#Diss 1st Appear 23,395 1934 12,768 1930 11,882 1894 11,325 1934 9600 1912 7602 1914 5065 1922 4108 1935 417 1971 166 2008 67 2008
Seventy-five percent of dissertations contained another first-level category (discipline). The ten firstlevel SCs most commonly occurring with the Sociology SC are listed in Table 2. Table 2. Subject Categories most often occuring with Sociology by number of occurrences Subject Category # co-occurrences PSYCHOLOGY 20685 EDUCATION 18022 WOMEN'S STUDIES 9003 HEALTH SCIENCES 8149 POLITICAL SCIENCE 7992 HISTORY 6946 BUSINESS ADMINISTRATION 5185 ECONOMICS 4838 ANTHROPOLOGY 4650 SOCIAL WORK 4333 BLACK STUDIES 4132 There were 76,336 dissertations with the Economics SC. Economics contains 8 second-level subject categories, present in Table 3. Table 3. Specialties of Economics by number of occurrences Specialty Economics, General Economics, Finance Economics, Theory 5
#Diss 1st Appear 29920 1930 18038 1932 11984 1938
Economics, Agricultural Economics, Commerce Business Economics, Labor Economics, History Economics, Environmental
10773 8637 6368 3910 202
1929 1899 1912 1909 2008
Only 34% of dissertations labeled Economics contained another first-level SC. The most frequently cooccurring SCs are listed in Table 4. Table 4. Subject Categories most often occuring with Economics by number of occurrences Subject Category # co-occurrences BUSINESS ADMINISTRATION 7233 POLITICAL SCIENCE 5104 SOCIOLOGY 4838 HISTORY 2628 EDUCATION 1887 HEALTH SCIENCES 1798 AGRICULTURE 1514 ENERGY 1444 ENVIRONMENTAL SCIENCES 1303 URBAN AND REGIONAL 1157 PLANNING GEOGRAPHY 1041 Political science had the lowest number of dissertations in ProQuest at 55,626, as well as the least number of second-level SCs (4). These are listed in Table 5. Table 5. Specialties of Political Science by number of occurrences Specialty Political Science, General Political Science, Public Administration Political Science, International Law and Relations Political Science, International Relations
#diss 31362 15984 12023 639
1st appear 1895 1894 1927 1948
55% of Political Science dissertations list another discipline. Table 6 shows the most frequently cooccurring disciplines. Table 6. Subject Categories most often occuring with Political Science by number of occurrences Subject Category # co-occurrences HISTORY 8035 SOCIOLOGY 7992 ECONOMICS 5104 EDUCATION 3443 BUSINESS ADMINISTRATION 2257 6
LAW PHILOSOPHY ENVIRONMENTAL SCIENCES HEALTH SCIENCES MASS COMMUNICATIONS WOMEN'S STUDIES
1958 1941 1469 1376 1370 1271
Based on the evaluation of the ProQuest SCs, Sociology is both the most specialized and the most interdisciplinary of the three selected disicplines; in addition, Economics is the least interdisciplinary (having the lowest proportion of dissertations with another SC), and Political Science is the least specialized (as represented by having the fewest number of second-level SCs). Description of survey respondents. Our sampling frame of 500 dissertations from each discipline reflects the general patterns of interdisciplinarity, although the sampled dissertations were more likely to contain multiple SCs than the numbers reflected here. This is due to the fact that more recent dissertations are more likely to be associated with multiple SCs. Within the sampling frame, 68% of Economics dissertations, 78% of Political Science dissertations, and 92% of Sociology dissertations were associated with another first-level SC. A 20% response rate was obtained for the survey, with lower levels for Economics (n=45; 13%) compared to Political Science and Sociology (n=81, n=64 [respectively]; both 23%). This counts only those surveys that were fully complete. The gender, race, and age of respondents is presented in Table 7, broken down by discipline. Overall, responders were 51% male (n=96) and 49% female (n=94); 75% of the respondents fell between 31 and 45 years old. As shown, Economics was more male-dominated, whereas Sociology was more female-dominated. Caucasians were the majority in all disciplines, and respondents from Economics tended to be the youngest. Table 7. Socio-demographic characteristics of respondents, by discipline Gender Race Age
Female Male Caucasian Non-Caucasian 26-35 years 36-45 years < 45 years
Economics (n=45) 35.6% 64.4% 85.4% 14.6% 72.7% 18.2% 9.1%
Political Science (n=81) 47.5% 52.5% 90.1% 9.9% 42.0% 35.8% 22.2%
Sociology (n=64) 60.0% 40.0% 79.7% 20.3% 30.8% 36.9% 12.3%
Nearly all respondents (98%) were currently employed, and respondents were from a diverse set of affiliated institutions. Table 8 shows the respondents’ identification with various types of organizations. Respondents were highly associated with academic institutions, particularly when the individuals in question were associated with Sociology. Counting partially and fully complete response sets, the 48 respondents from Economics represented 46 different institutions. The majority of institutions (n=38; 83%) were in the USA: ten other countries were also represented (Brazil, Canada, Chile , Germany, Italy, Lebanon, Mexico, Sweden, Switzerland, and UK). Similarly, 67 affiliations were located for Sociology doctorates. These were also primarily in the USA (n=62; 93%), with only five other countries represented (Canada, France, Peru, United Arab Emirates, and South Africa). Political Science doctorates were associated with 81 institutions (with one individual acting as a freelancer). Ten individuals from Political 7
Science were located outside of the USA (Bangladesh, Canada, Israel, Mexico, Netherlands, Singapore, Spain, Switzerland, and UK). While respondents identified with institutions in 19 countries, 91% were located in the U.S., 7% were in Canada, and only 2% represented the other 17 countries; there is a clear U.S. bias in the response. Table 8. Current institutional affiliation of respondents Acad. Corp. Govt Economics 75.6% 6.7% 11.1% Political Science 79.0% 3.7% 8.6% Sociology 86.2% 3.1% 1.5%
Non-profit 2.2% 3.7% 6.2%
Other 2.2% 2.5% 1.5%
No answer 2.2% 2.5% 1.5%
These high degrees of association with academic institutions across all categories should not be taken to be representative of career trajectories for doctoral students; rather, they are possibly an artifact of collecting the sample based on identifiable contact information, which was potentially more easily located for those holding academic posts (due to the public availability of curricula vitae and other relevant material). Questionnaire results. Figure 3 depicts the degree to which respondents who completed dissertations in certain subject categories identified with the same disciplinary label. The first column represents responses to the question: What was the field of study of your doctorate degree? Questions 2 and 3 asked respondents to identity the disciplines from which they consumed (2) and to which they disseminated (3) information. Questions 4 and 5 asked respondents to identify the disciplines from which they primarily drew theories (4) and methods (5). Question six asked: “How would you selfidentify your expertise or research area to a future employer?” Question seven asked: “How would you
8
self-identify your expertise or research area to persons outside the workplace?” These used the same coding scheme, where explicit mention of the disciplinary label was indicated with the darkest color and applications including the disciplinary label (e.g., sociology of x) were indicated in a lighter color.
Figure 3. Association with disciplinary labels, by each survey question. Grey indicates that no response was received. Cases with non-response for all seven categories were eliminated.
Our goal was to assess the degree to which the respondents identified with the broad disciplinary label (i.e., Political Science, Sociology, and Economics), as well as applied and specialized terms that would be recognizeable to an outsider (e.g., sociology of x, economics of y, politics of z). We did not attempt to differentiate specializations, as our aim was only to see how much the broad category was used as an identify marker for the respondents. For respondents in the Economics set, 50% (23/46) answered “Economics” to the first question (i.e., “What was the field of study of your doctorate degree?”). These are indicated in dark red in the figure. Ten respondents (shown in lighter red in Figure 3) identified a disciplinarity perspective of economics (e.g., agricultural and resource economics, business economics, nursing economics, and health economics). A few identified specialties of economics, including macroeeocnomics and time series econometrics (also identified in the lighter shade). Other areas included industrial organization, history, urban and regional planning, finance, operations research, sociology, and marine biology. No color is represented for these. Grey indicates that the question was not answered. Similarly, 45% of those with Political Science SCs on their dissertations said that their doctoral degree was in “Political Science.” Three provided specialized terms, such as comparative politics or political theory. Many listed disciplines outside of Political Science, such as Anthropology, Education, Geography, French, and Philosophy. Others listed specialties that are closely related (and possibly considered within) the field of political science: public administration, public policy, and international relations (these were not shaded). 9
Sociology demonstrated the highest permeability. Only 15% of respondents with Sociology SCs indicated that they received their degrees in Sociology. There were a few instances of specific applications of sociology (e.g., sociology of gender and sexuality; sociology of culture and inequality; sociology of politics); however, most respondents were outside of the field of sociology (for example, Anthropolgy, Nursing, or Business). Social Work was also fairly highly represented in the responses (this was considered a distinct discipline). In Economics, the highest proportion of respondents associated with the explicit disciplinary label was regarding the use of theories, with 60% of respondents indicating “Economics” as the primary discipline from which they use theories. The majority of respondents indicated “Economics” for all questions except question 6 and 7, where only 20% reported using the term “Economics” to describe their expertise to an employer. Only a third said that they would use this term to describe themselves to people outside their workplace. More individuals identified Political Science as the area of their doctorate (44%) than those who indicated that they primarily used political science theories in their work (38%). Very few used this label to self-identify to employers (11%) or to those outside the workplace (17%). The strongest association with Sociology was in terms of methods, with 19% of respondents indicating that they primarily drew upon sociological methods in their work. Nearly 18% indicated that they drew from sociological theories, fewer than those who indicated that they received a degree in sociology (15%). A very small percentage self-identified with Sociology to describe themselves to employers (5%) or others (4%). The majority of respondents whose dissertations were granted in Sociology did not indicate any apparent association with Sociology. Given that the SC that appears first is considered the “primary SC,” it is possible that more reliable results might have been generated if the analysis were restricted to primary SCs. This was investigated by analyzing the percentage of all 500 dissertations in the sampling frame that were associated with the degree as a primary SC, the proportion of dissertations among the respondents with the selected discipline as the primary SC, and the percent of these that were “exact matches” with the degree in which the respondent received their degree. By exact match, we refer to those respondents which mentioned the disciplinary label in their description of the discipline in which they received their degree. Table 9 provides the percentages of all dissertations (from the sampled 500) that had the given discipline as the primary SC (i.e., the first-listed first-level SC); the percent of dissertations among respondents with the given discipline as the primary SC; the percent of respondents who gave an “exact match” response to question 1 on the survey; the percent of respondents with the designated primary SC dissertation who gave an “exact match” answer; and the percentage of respondents without the given discipline as a Primary SC who gave an “exact match” answer. Table 9. Relationship between the primary SCs assigned to the sampled dissertations and the disciplinary labels used by respondents to describe the discipline in which they received their degree (i.e., Question 1 in Figure 3). Primary SC (out of 500)
10
Primary SCs (from respondents)
Exact % of primary matches SC that were (from exact matches respondents)
% of nonprimary SCs that were exact matches
Economics Political Science Sociology
68% 42% 20%
69% (n=33) 48% (n=40) 23% (n=17)
50% (n=23) 45% (n=37) 15% (n=11)
84% (n=26)* 64% (n=25) 36% (n=5)
50% (n=7) 41% (n=17) 10% (n=5)
*percentage of those who responded to the question for this and all other similar calculations
As shown, our respondent set matched the proportions of dissertations from the entire set with the discipline as the primary SC. In all cases, the proportion of repondents with exact matches improved if only those dissertations with the discipline in the primary position were included. For example, whereas only 50% of respondents stated that they received their degree in Economics, this improved to 84% if only those respondents whose dissertations had Economics as a Primary SC were considered. However, although this eliminates some false positives, it does not completely mitigate the false negative situation: half of the respondents whose dissertations did not list Economics as a Primary SC identified Economics as their discipline. There was also large variability across the disciplines: even when looking at only dissertations with Sociology as a Primary SC, only 36% were exact matches (however, this is limited by the small sample size of this test). Validation exercise. In order to supplement these findings, we investigated three disciplines (Biology, Chemistry, and Computer Science) that lay outside of the social sciences. A series of Google and ProQuest searches were conducted on the authors of 100 dissertations for each of these three disciplines so as to determine the degree to which their field of study accorded with the SC associated with their dissertation. Table 10 presents the percentage of our sample that had the designated discipline as a Primary SC; the percentage for whom the Primary SC and the field of study were an “exact match”; the percentage of dissertations that had the designated discipline as a Primary SC that were an “exact match”; and the percentage of people who had a dissertation with another Primary SC but had received a degree in Biology, Chemistry, or Computer Science (respectively). As with our previous analysis, there was an improvement in accuracy when only the primary SC was considered, although the gain was minimal in Biology and Chemistry. There was also a high degree of potential false negatives in the field of Chemistry. Table 10. Relationship between the primary SCs assigned to dissertations and the discipline in which dissertation authors received their degree Primary SC Exact Matches % of primary SC % of nonthat were exact primary SC that matches were exact matches Biology 90% 37% 39% (n=35) 20% (n=2) Chemistry 82% 72% 77% (n=63) 50% (n=9) Computer Science 48% 70% 96% (n=46) 46% (n=24) We further analyzed the instances in which each discipline was listed as a Primary SC but was not considered an exact match. There were 55 such instances in Biology. Many of the degrees in this category were from related areas: Biochemistry, Bioinformatics, Biological Chemistry, Biological Oceanography, Biomedical Sciences, Biophysics, and Biostatistics (these represented 16 of the dissertations). There were also other disciplines such as Chemistry, Computer Science, Psychology, Ecology, Geology, and Zoology. Finally, there were specializations such as Neuroscience (8 of the sampled dissertations).
11
There were nineteen instances in which Chemistry was listed first, but it was not an exact match. These included some aspects of chemistry (e.g., Chemical and Biochemical Engineering, Chemical and Biomolecular Engineering, Chemical Biology, and Chemical Engineering). Other disciplines represented among these dissertations were Physics, Biology, and Information and Library Science. There were only two instances in which Computer Science was listed as a primary SC but was not an exact match with the discipline of the degree. In both cases it was a very similar term: Engineering (Computer Engineering) and Informatics and Computing. The validation study confirmed the results of the main study by: 1) suggesting improved accuracy when Primary SCs were used; and 2) identifying large variability in both false positives and false negatives across disciplinary areas. DISCUSSION AND CONCLUSION The results of our survey question the utility of traditional disciplinary labels in a time of heightened specialization and interdisciplinary. In examining the entire ProQuest dataset, Sociology was shown to be the most specialized and also the most likely to appear in the presence of another first-level SC. These SCs were spread across the disciplinary spectrum, including medical sciences, social sciences, humanities, and professional fields. Economics was slightly narrower, with little more than one-third of the dissertations in this area appearing in conjunction with another SC. Political Science was the least specialized, according to ProQuest’s schema, but it was still fairly interdisciplinary, with more than half of the dissertations occurring in the presence of another first-level SC. The goal of the questionnaire was to identify the ways in which the dissertation authors associated with the SC listed on the dissertation. Many elements were investigated: individuals’ degrees, their use of theory and methods, the disciplines from which they consumed and disseminated knowledge, and the way they described their expertise to employers and the public. As shown in Figure3, there were varying degrees to which respondents identified with the disciplinary labels into which they were classed. Sociology was the most diffuse, with very few people using this term to describe their scholarly activities or identities. A majority of respondents did not refer to “sociology” in any of their responses, even though their dissertation was classified under this SC. One concern could be that Social Sciences are particularly permeable, interdisciplinary, and are more diffuse than STEM disciplines. To test this, we examined a random sample of dissertations in Biology, Chemistry, and Computer Science. As was shown, there was high variability in the degree to which the SCs captured the disciplinary labels under which the disciplines were conferred. Biology was the most diffuse and had the highest potential for obtaining false positives, even when restricting the analysis to primary subject categories. Computer Science had the most reliable results (although still exhibiting a fairly high level of false negatives) when the sampling was restricted to primary SCs. These findings suggest that caution be exercised when using ProQuest SCs as proxies for disciplinarity. However, the variability across the disciplines suggests that ProQuest may not be systematically misclassifying dissertations; rather, it may well be that the traditional disciplinary labels no longer hold. In the qualitative component of this study, many respondents provided detailed and specialized responses—rather than giving a single word disciplinary label, they described at length the topic that they studied. It is possible that topics carry more weight than disciplines in specialized environments.
12
Specialization may also be characterizing the marketplace, as epitomized by the degree labels for Biology. Degrees were no longer conferred in “Biology,” but rather in Bioinformatics, Biological Oceanography, and Biostatistics. There is a “balance between specialization and integration” (Klein, 1990, p. 38) that must be carefully managed. Furthermore, it begs the question of what institutions gain by specializing their degrees. Is there a newcomer’s advantage to being the first school to offer a degree in a specialized topic? Perhaps traditional disciplines are no longer the Gold Standard, thus reflecting a marketplace that is now nimble enough to evaluate people on their topical distinctions rather than on their disciplinary affiliations. In this case, “interdisciplinary” may not be the appropriate term to characterize the new marketplace for doctoral students. We may live in a post-interdisciplinary space in which scholars wish to move more freely across disciplinary boundaries and aggregate topically more than disciplinarily. Whitley suggested this was the case as early as 1984 when he wrote that “the discipline as a set of research activities has outgrown the departmental basis of employment and careers” and that the “identification of intellectual disciplines with university undergraduate departments is no longer a social reality in many science” (Whitley, 1984, p. 18). There remain, however, some disciplines which are less permeable and present more barriers for entry (Klein, 1993), and it seems that disciplines still remain a dominant organizing agent for institutions of higher education and retain “enormous influence over the organization and production of knowledge” (Klein, 1993, p. 185). This begs an analysis of the ways in which disciplinary labels may connote something different between academic markets and professional labor markets. There is a historically complicated relationship between academic knowledge and professionalism. Abbott (1988) argued that the “ability of a profession to sustain its jurisdictions lies partly in the power and prestige of its academic knowledge” (p. 53-54). However, there are numerous conflicts over jurisdictional boundaries both among and within disciplines, one that is complicated by the growing specialization of research (Jacobs, 2013). There are also many professionals who continue to contribute to academic knowledge once they have formally left academe (Breimer & Leksell, 2011). Future research is needed to understand whether specialization in the labor market is parallel to specialization in the academic research market. Limitations. There are, of course, a number of limitations to the present study. As noted above, our sample is restricted to those with an available e-mail address. It is likely that those with an e-mail address are more likely to be a) employed and b) associated with an academic institution. Given the “risks” associated with interdisciplinary research (Kniffin & Hanks, 2013), it is possible that the sample might skew towards those who are more “disciplinary” in nature. However, current research has shown that individuals conducting interdisciplinary dissertation research are more likely to be employed in academe (Millar, 2013). Our results seem to confirm this. The mode of delivery is also a limitation. E-mail has been shown to have lower response rates than other modes of survey research (Schaefer & Dillman, 1998) and have been declining over time. Our response rates were similar to those found in a review of survey research with academic participants (Dykema, Stevenson, Klein, Kim, & Day, 2013), but still lower than averages for other modes of delivery. We used a number of tested incentives (i.e., personalizing the survey and a monetary incentive), but it’s possible that our response rate could have been improved with another incentive structure (e.g., advance notice). The choice of fields is also a limitation. Our objective was to choose a wide variety of disciplines, each with their own particularistic identities—in both the main and validation studies. However, there is 13
always a somewhat arbitrary nature to discipline selection and we must acknowledge that different results may have been generated given a different choice of disciplines. Furthermore, we have classified Economics, Sociology, and Political Science as social sciences and Biology, Computer Science, and Chemistry as non-social sciences. However, these distinctions may not hold across all subareas of these disciplines. FUTURE RESEARCH Future research should investigate how people can appropriately evaluate these topic-based scholars, what implications exist for the reorganization of institutions, how these topic distinctions play into information consumption and disciplinary identity, and how reputation and authority can be conferred given this highly permeable space. Although interdisciplinarity is often seen as a positive thing (Rafols & Meyer, 2010), there are dangers and consequences for a lack of disciplinary grounding. Disciplines are not only institutional conventions, but are also bound by shared paradigms: i.e., the types of questions that can be asked, the ways in which they can be asked, and the answers that can be retrieved (Kuhn, 1970). Members of disciplines establish their own sets of cultural rules and norms, allowing both members and outsiders to make sense of activities and interactions through the application of disciplinary-relevant frames (Goffman, 1974). Disciplines provide theoretical and methodological frameworks for the education and training of doctoral students. One might question how students can move freely across disciplinary boundaries—appropriating theories and methods as necessary—while still maintaining a deeply contextualized knowledge of the appropriate use of these theories and methods. Furthermore, there is the conceptual problem of what happens when disciplinary work is abandoned in favor of interdisciplinary research. If interdisciplinary research represents “the juxtaposition of disciplines” (Klein, 1990, p. 56), who is contributing to the tool chest from which these researchers are drawing theories and methods? Can interdisciplinary research exist in the absence of disciplines? Klein (1993) suggested that boundary work involves not only “crossing” and “deconstructing,” but also “reconstructing boundaries” (p. 196). Therefore, it is possible that interdisciplinary areas are merely pre-disciplinary areas in the process of reconstructing boundaries. Alternatively, we may be transitioning into a post-interdisciplinary environment, where identities are constructed in topical, rather that disciplinary, frames of reference. ACKNOWLEDGEMENTS This work was funded by the Science of Science Innovation and Policy (SciSIP) program of the National Science Foundation (grant no. 1158670). CITED REFERENCES Abbott, A. (1999). Department and Discipline: Chicago Sociology at One Hundred. Chicago, IL: The University of Chicago Press. Abbott, A. (1988). The system of professions: An essay on the dividion of expert labor. Chicago, IL: The University of Chicago Press. Andersen, J.P., & Hammarfelt, B. (2011). Price revisited: On the growth of dissertations in eight research fields. Scientometrics, 88, 371-383. Anseel, F., Lievens, F., Schollaert, E., & Choragwicka, B. (2010). Response rates in organizational science, 1995-2008: A meta-analytic review and guidelines for survey researchers. Journal of Business and Psychology, 25(3), 335-349. Baumann, B. (1975). Imaginative participation: The career of an organizing concept in a multidisciplinary context. The Hague: Martinus Nijhoff.
14
Breimer, L.H., & Leksell, J. (2013). Longitudinal and cross-sectional study of registered nurses in Sweden who undertake a PhD showing that nurses continue to publish in English after their PhD but male nurses are more productive than female nurses. Scientometrics, 87(2), 337-345. Dykema, J., Stevenson, J., Klein, L., Kim, Y., & Day, B. (2013). Effects of e-mailed versus mailed invitations and incentives on response rates, data quality, and costs in a web survey of university faculty. Social Science Computer Review, 31(3), 359-370. Garfield, E. (1978). The Gordian Knot of journal coverage: Why we can’t put all the journals you want into the Current Contents edition you read. Current Contents, 13, 5-7. Goffman, E. (1974). Frame Analysis: An Essay on the Organization of Experience. New York, NY: Harper & Row. Jacobs, J.A. (2013). In defense of disciplines: Interdisciplinarity and specialization in the research university. Chicago, IL: The University of Chicago Press. Klein, J. T. (1990). Interdisciplinarity: History, theory, and practice. Detroit: Wayne State University Press. Klein, J. T. (1993). Blurring, cracking, and crossing: Permeation and the fracturing of discipline. In E. Messer-Davidow, D. R. Shumway, & D. J. Sylvan (Eds.), Knowledges: Historical and critical studies in disciplinarity (185-211). Charlottesville and London: The University Press of Virginia. Klein, J. T. (1996). Crossing boundaries: Knowledge, disciplinarities, and interdisciplinarities. Charlottesville, VA: University of Virginia Press. Kniffin, K.M., & Hanks, A.S. (2013). Boundary spanning in academia: Antecedents and near-term consequences of academic entrepreneurialism. Cornell Higher Education Research Institute (CHERI) Working Paper 158. Kuhn, T. S. (1970). The structure of scientific revolutions. Chicago: University of Chicago Press. Leydesdorff, L., & Rafols, I. (2009). A global map of science based on the ISI Subject Categories. Journal of the American Society for Information Science and Technology, 60(2), 348-362. Leydesdorff, L., Carley, S., & Rafols, I. (2013). Global maps of science based on the new Web-of-Science categories. Scientometrics, 94(2), 589-593. Leydesdorff, L., Rotolo, D., & Rafols, I. (2012). Bibliometric perspectives on medical innovation using the medical subject Headings of PubMed. Journal of the American Society for Information Science and Technology, 63(11), 2239-2253. McLean, A. (2012). Personal correspondence to Cassidy Sugimoto via email on October 26, 2012. Millar, M.M. (2013). Interdisciplinary research and the early career: The effect of interdisciplinary dissertation research on career placement and publication productivity of doctoral graduates in the sciences. Research Policy, 42(5), 1152-1164. Morillo, F., Bordons, M., & Gomez, I. (2003). Interdisciplinarity in science: A tentative typology of disciplines and research areas. Journal of the American Society for Information Science & Technology, 54(13), 1237-1249. Ni, C., & Sugimoto, C. R. (2012). Using doctoral dissertations for a new understanding of disciplinarity and interdisciplinarity. Poster. Annual Meeting of the American Society for Information Science & Technology. Baltimore, MD: October 25-31, 2012. Ni, C., & Sugimoto, C.R. (2013a). Exploring interdisciplinarity in Economics through academic genealogy: An exploratory study. Proceedings of the International Society for Scientometrics and Informetrics conference [poster]. Ni, C. & Sugimoto, C.R. (2013b). Academic genealogy as an indicator of interdisciplinarity: A preliminary examination of sociology doctoral dissertations. iConference 2013 [poster]. Ni, C., Sugimoto, C.R., & Jiang, J. (2013). Venue-Author-Coupling: A novel measure for identifying disciplines through social structures. Journal of the American Society for Information Science & Technology, 64(2), 265-279. 15
Porter, A. L., & Rafols, I. (2009). Is science becoming more interdisciplinary? Measuring and mapping six research fields over time. Scientometrics, 81(3), 719-745. Rafols, I., & Meyer, M. (2010). Diversity and network coherence as indicators of interdisciplinarity: Case studies in bionanoscience. Scientometrics, 82, 263-287. Schaefer, D.R., & Dillman, D.A. (1998). Development of a standard e-mail methodology: results of an experiment. The Public Opinion Quarterly, 62(3), 378-397. Sugimoto, C.R. (2014). Academic genealogy. In B. Cronin & C.R. Sugimoto (Eds.), Beyond bibliometrics: Metrics-based evaluation of research (pages tbd). Boston, MA: MIT Press. Sugimoto, C.R., Ni, C., Russell, T.G., & Bychowski, B. (2011). Academic genealogy as an indicator of interdisciplinarity: An examination of dissertation networks in Library and Information Science. Journal of the American Society for Information Science & Technology, 62(9), 1808-1828. doi: 10.1002/asi.2156 Turner, S. (2000). What are disciplines? And how is interdisciplinarity different? In P. Weingart & N. Stehr (Eds.), Practising Interdisciplinarity (pp. 46-65). Toronto: University of Toronto Press. Whitley, R. (1984). The rise and decline of university disciplines in the sciences. In R. Jurkovich and J. H. P. Paelinck, Problems in interdisciplinary studies: Issues in interdisciplinary Studies, pp. 10-25. Gower: Aldershot, Hampshire. Ying, T.Y., & Xiao, H.G. (2012). Knowledge linkage: A social network analysis of tourism dissertation subjects. Journal of Hospitality & Tourism Research, 36(4), 450-477.
16
APPENDIX A: Questionnaire Disciplinary Identity Q1 Are you currently employed? Yes (1) No (2) Answer If Are you currently employed? Yes Is Selected Q2 How do you categorize your current employer?
Academic (1) Corporate (2) Government (3) Non-profit (4) Other (5)
Answer If Are you currently employed? Yes Is Selected Q3 Is your doctorate degree required for the position you currently hold? Yes (1) No (2) Answer If Are you currently employed? Yes Is Selected Q6 How relevant is your doctorate degree to your current position?
17
Very Relevant (1) Somewhat Relevant (2) Neutral (3) Somewhat Irrelevant (4) Very Irrelevant (5)
Answer If Are you currently employed? Yes Is Selected Q7 How many other employees working in similar positions at your current place of employment have a Ph.D. ?
All (1) Most (2) Some (3) Few (4) None (13)
Answer If How many other employees working in similar positions at ... None Is Not Selected And Are you currently employed? Yes Is Selected Q8 How many of these employees have Ph.D.s in your field of study?
All (1) Most (2) Some (3) Few (4) None (5)
Q4 How would you self-identify your expertise or research area to a future employer?
Q5 How would you self-identify your expertise or research area to persons outside the workplace?
Q10 What was the field of study of your doctorate degree?
Answer If What was the field of study of your doctorate degree? Text Response Is Not Empty Q13 How strongly do you self-identify with ${q://QID10/ChoiceTextEntryValue}?
18
Very Strongly (1) Somewhat Strongly (2) Neutral (3) Somewhat Weakly (4) Very Weakly (5)
Answer If Are you currently employed? Yes Is Selected And What was the field of study of your doctorate degree? Text Response Is Not Empty Q11 How relevant is ${q://QID10/ChoiceTextEntryValue} to your current work?
Very Relevant (1) Somewhat Relevant (2) Neutral (3) Somewhat Irrelevant (4) Very Irrelevant (5)
Answer If Are you currently employed? Yes Is Selected And What was the field of study of your doctorate degree? Text Response Is Not Empty Q12 How significant is your expertise in ${q://QID10/ChoiceTextEntryValue} to your current employer?
Very Relevant (1) Somewhat Relevant (2) Neutral (3) Somewhat Irrelevant (4) Very Irrelevant (5)
Answer If Are you currently employed? Yes Is Selected Q14 From what sources do you consume information related to tasks at your current place of employment? (Choose all that apply.)
Newspapers (1) Trade Journals (2) Academic Journals (3) Books (4) Listservs (5) Colleagues (6) Workshop/Symposia/Conferences (7) Blogs (8) Facebook (9) Twitter (10) Other (11) ____________________
Answer If Are you currently employed? Yes Is Selected Q15 From which primary discipline do you consume information related to tasks at your current place of employment?
19
Answer If Are you currently employed? Yes Is Selected Q16 From which primary discipline do you disseminate information related to tasks of your current job?
Answer If Are you currently employed? Yes Is Selected Q17 From which discipline do you primarily draw theory used in your current job?
Answer If Are you currently employed? Yes Is Selected Q18 From which discipline do you primarily draw methods used in your current job?
Q20 What is your sex? Male (1) Female (2) Q21 What is your age?
20
Under 20 (1) 20 to 25 years (2) 26 to 30 years (3) 31 to 35 years (4) 36 to 40 years (5) 41 to 45 years (6) 46 to 50 years (7) 51 to 55 years (8) 56 to 60 years (9) 61 to 65 years (10) 66 to 70 years (11) 71 to 75 years (12) Over 75 years (13)
Q22 What is your race?
American Indian / Native American (1) Asian (2) Black / African American (3) Hispanic / Latino (4) White / Caucasian (5) Pacific Islander (6) Other (7)
Q25 Do you want to participate in the Amazon gift certificate drawing? Yes (1) No (2) Answer If Do you want to participate in the Amazon gift certificate... Yes Is Selected Q26 Please provide an email address.
21
