Begin the Journey With the End in Mind - SAGE Journals

Begin the Journey With the End in Mind

Organizational Research Methods Volume 12 Number 2 April 2009 239-252 © 2009 SAGE Publications 10.1177/1094428108317204 http://orm.sagepub.com hosted at http://online.sagepub.com

Richard Klimoski George Mason University This commentary critically examines the issues raised in the article by Bedeian, Van Fleet, and Hyman from a functional perspective, one that might be used in decisions about who should be a member of a prototypic work group. It takes the position that, like a work group, a journal editorial board exists for a purpose, and that purpose (or function) should play a major role in determining the criteria to be used when staffing the board’s membership. When contemporary thinking about the staffing of work groups is applied, the current practices of the journals of interest are found to be appropriate and defensible. Keywords: board membership; effective journal boards; board outcome assessment; functional approach to boards

T

he thesis of the article by Bedeian, Van Fleet, and Hyman (2009) is clearly laid out in the very first sentence of the abstract of their article: ‘‘[Journal] editorial board members should be selected on the basis of their scholarly achievements, as demonstrated by publications in peer-reviewed journals and evidence that their work is of value to others in their disciplines.’’ Furthermore, they assert, ‘‘To do otherwise, threatens both the credibility enjoyed by our discipline and the spirit of scientific inquiry that brings us together as a community of scholars’’ (p. 211-238). In my commentary, I use a functional analysis to argue that, in staffing of the boards of the key journals of our discipline (Academy of Management Journal, Academy of Management Review), certain levels of accomplishment relative to such criteria are necessary. But as characterized in their article, the latter are not well operationalized and hence need to be interpreted. I also detail why these criteria are not ‘‘sufficient.’’ Finally, I also point out that the authors do not present evidence that the ‘‘credibility of our discipline’’ is at risk. It is my position that if the functional requirements of the editorial board are considered and the ‘‘ends’’ or goals of board member service are kept in mind, there is little evidence that the current paradigm for vetting, selecting, and recruiting board members for key Academy of Management journals is problematic.

A Functional Approach to the Stafﬁng of Work Assignments Most staffing models for work organizations require a consideration of the job to be performed by the workers and the standards for performance to be measured or monitored (Schmitt & Borman, 1993). Both are usually related to the outcomes to be accomplished by the job incumbent, as required by the organization. In the case of making individual 239

240

Organizational Research Methods

staffing assignments, these usually imply the need to recruit and select for key worker characteristics thought to be important for both task and ‘‘context’’ performance (Borman & Motowidlo, 1997). In the process of recruiting and selecting individuals, it is typical for decision makers to seek evidence that an applicant possesses enough of the qualities of interest that he or she can function well on the job. Such evidence can be found in the applicant’s educational background and work history, in the portfolio or record of accomplishments he or she can offer, or in actual demonstrations of job-related performance as solicited at the time of recruiting or hiring. In the cases of work groups, boards, or teams, the staffing model is somewhat more complex (Klimoski & Zukin, 1999). This is because in addition to the obvious need to ensure that the individual members are competent and credible, the group or team, as a collective, also needs to possess certain capabilities. This important distinction notwithstanding, the approach to staffing a work group would also follow the same paradigm by starting with a consideration of the performance functions of the work group and the kinds of outcomes that it is expected to attain. Broadly speaking, evidence of a high-performing work group (or team) can be found by looking at such things as the status of its key attributes (as inputs), effectiveness and efficiency of core processes, nature of emerging or proximal outcomes, and quality and impact of its mission-relevant outputs or products (Klimoski, in press). Prototypic processes to consider include patterns of communication, work flow, and influence. Desirable proximal outcomes would be high levels of cohesion and trust, member feelings of group efficacy (pride), and, certainly, the desire to stay a member of the group (Marks, Zaccaro, & Mathieu, 2001). Appropriate metrics for outcomes or products of a work group are usually context specific, depending on its mission or purpose. But usually they relate to the amount and quality of work produced. Moreover, because a work team or group is typically embedded in a larger social system, its patterns of activities (processes) and its outcomes have implications for a variety of stakeholders beyond the members themselves. Whether such stakeholders are suppliers, investors, customers, or competitors, their needs and concerns are relevant to the functioning of the work group and to its viability. It is important that the converse is true: What the work group does will also be of concern and interest to these stakeholders. When evaluating a work group, the extent to which it meets the needs of stakeholders must be considered an important functional requirement. What does all of this imply for a commentary on the Bedeian et al. (2009) article? I argue that by taking a job-analytic or functional framework, certain weaknesses of the critique by Bedeian et al. can be made to stand out. Moreover, a functional perspective also reveals that, contrary to the position of Bedeian et al., the way that our discipline has approached the staffing of journal editorial boards (at least as represented by Academy of Management Journal and Academy of Management Review) may not be at all inappropriate. Finally, I point out that these authors fail to make their case with regard to what may be the major concern of their article (e.g., Bedeian et al., 2009). That is, they do not offer evidence that the status or reputation of our discipline (one plausible ‘‘outcome’’ of editorial board staffing practices) in the eyes of key stakeholders is at risk.

Klimoski / Commentary

241

Building a Credible and Effective Editorial Board: The Member’s Attributes as an Input Bedeian et al. (2009) appear to feel strongly that a major flaw in Academy of Management board staffing practices for the years surveyed relates to the low levels of (inferred) credibility and competency of current board members. The basis for this conclusion rests on two metrics, the average number of publications of board members, and the frequency of citations for their work. In short, compared to some selected peers, Academy of Management journal board members (on average) have not published enough and are not cited enough to do a good job or to covey credibility. As a result, the reputation of our journals and our discipline is at risk. A functional approach to staffing an editorial board would involve a consideration of the job that each board member must perform and the work that each member must do to ensure that the board performs well as a collective. This would then provide the basis for choosing the qualities required in the recruiting and selection of board members. Any fair critique of board staffing practices should consider how well journals are doing to seek members relative to this set of qualities. What might such a set look like? The following are offered for consideration. These are based on my many years of service as a journal reviewer, as an associate editor, and as editor. They are also informed by my experiences as a well-published author.1

Desirable Board Member Attributes Subject-matter knowledge. This seems fundamental. An effective board member must be deeply familiar with the content (findings, theories, data, knowledge gaps, problems to be solved) of the field or fields represented in manuscripts to be reviewed and critiqued.2 Research acumen. This involves the capacity to design, conduct, analyze, write up, and publish original and meaningful research. Note that this requires more than subject-matter or content knowledge. Reviewing skills. Although knowledge and research experience are no doubt components, reviewing capabilities represents an additional dimension to the demands of the job of a board member. The latter must not only recognize when a article reflects weak research but also be able to conceptualize and communicate in a compelling (and humane) way why a manuscript is not suitable for publication. On the other hand, if there are fair prospects for potential publication, the board member must have the skill to work with the author and editor to shape the piece in an appropriate manner. Work habits. Most board members must work within a time frame and according to a specified protocol. This is a nontrivial ‘‘worker requirement.’’

242


Integrity. Board members must exhibit the capacity to respect the intellectual property rights of authors and to their expectations regarding the care and accuracy of reviews, open mindedness, confidentiality, and helpfulness regarding recommendations. Perseverance. Despite the best of intentions, the process of moving a manuscript to the point of publication typically takes time. But it also requires perseverance on the part of board members. Those who are dispositionally inclined to want quick results are not likely to do well in this role. These qualities are not self-evident in people. They usually have to be inferred from behavior exhibited over time and in relevant contexts. Traditionally, in seeking prospective board members, an individual’s own record of publications is used as a surrogate and as a way to get a good ‘‘fix’’ on such key qualities as subject-matter expertise, work habits, and research acumen. The vetting of a candidate for board membership relative to such things as personal standards of work habits and integrity usually involves the use of peer nominations and increasingly ‘‘job samples.’’ With regard to the former, it is common to go to established scholars and other credible sources (e.g., current board members) for the names of fully qualified individuals. Relatively young scholars are also recruited this way, but more likely they are invited to perform as ad hoc reviewers for a time as a way to ascertain their capabilities (Feldman, in press). In any event, even after one has become a regular board member, the better journals have a process of continuous quality control. Reviews are evaluated and scored for timeliness and quality. In this way, a researcher’s credibility as a scholar and competence as a board member are simultaneously achieved. A weakness of the article by Bedeian et al. (2009) is that it takes a narrow and incomplete perspective on the qualities needed to be an effective editorial board member. The full range of attributes as proposed above is not considered. True, the two metrics of publication and citation frequency provide some basis for inferring that a member will be a competent reviewer. But here, I feel that the authors take an extreme position. Specifically, they seem to assume a linear relationship between member scores on such metrics and member suitability. The more publications, the better the board member. The more citations, the better the board member. But this does not necessarily follow. Much like the way that the error rate of surgeons asymptotes after performing a certain number of procedures, one might argue that the capacity to be a credible and effective board member requires a certain level of success in publishing, beyond which there is no advantage. That is, if one is concerned about reviewing competence, it is yet to be established what number of publications or citations would be sufficient to assure us. A similar point can be made, if one is concerned about reviewer credibility (for a more complete discussion of ‘‘cut scores,’’ see Tsui & Hollenbeck, 2009). More telling, however, in their critique of the Academy of Management Journal and the Academy of Management Review board staffing practices, Bedeian et al. (2009) really do not actually address the issue of board member credibility. From a functional analysis perspective, I have made the point that a larger set of considerations is clearly relevant to performing well in the role of a board member. Similarly, one could argue that board member credibility may require holding or reflecting some of these (even other) characteristics, singly or in combination. As in the case of competence, achieving credibility is unlikely to be a function of simply the length of one’s publication list.3


243

Building a Credible and Effective Editorial Board: The Members as a ‘‘Collective’’ Input A functional analysis would also highlight the point that there are certain attributes that are needed for a board to be effective and credible that stem not simply from the qualities of individual members but from the nature of the board as a collective. In this regard, the whole might indeed be more than the sum of the parts (see Feldman, in press). Again based on my editorial service and experience as an author, the following might be considered key board-level attributes: Content coverage. A strong board has a set of members who, collectively, can cover the content domains of interest to journal stakeholders. Methods diversity. In most fields, there is a wide variety of methods and analytic tools that can be employed. It may be unrealistic to expect any one board member to be facile in all of them. Distinctive competency. Board members often prefer and are especially good at examining and critiquing different facets of the research enterprise. For example, some members seem to have most to say about the way the problem case gets spelled out, others stress the features of (e.g., an empirical) study itself. Still others worry more about the significance than the validity of the findings. The considerations raised above imply the need to recognize and manage the collective expertise of a journal board while achieving the diversity (even complimentarity) required to ensure a quality review of the kinds of manuscripts submitted. Size. The number of members appropriate for a board relates to both the diversity of the work submitted and the number of manuscripts to be processed. The challenge in staffing a board in these days of ‘‘journal proliferation’’ is to identify and recruit the appropriate number of members. In this regard, both the Academy of Management Journal and the Academy of Management Review have fairly large editorial boards. This would seem to be functional in light of need to process the large number of manuscripts that they receive each year. Signaling. The attributes of the board as reflected in either a subset or the whole collection of members can serve a signaling function. Such things as the nature and range of expertise among members, their orientation to epistemology, and their reputation or levels of career success serve to attract (or dissuade) authors or to otherwise predispose them to submit certain kinds of work. To several points made in the article by Bedeian et al. (2009), the composition of the board does indeed serve as a proxy for the standards employed by the journal, albeit across a wider range of parameters than addressed by their analysis. Board member composition can also serve to signal credibility. In this regard, it is not at all unusual for journals to have one or more individuals on the board because they provide ‘‘marquis value.’’ Although they may not review much (or even well), they add prestige to the enterprise.

244


Professional development. One additional requirement might be suggested based on a functional analysis associated with building a board as a collective. This relates to the pressure on journals to continuously replenish the pool of potential board members. As implied, the recruiting of prospective board members goes on all the time. But to exclusively use the criterion of extraordinary professional achievement as reflected in publications would be disadvantageous. If journals accepted only these accomplished scholars as board members, not only would the pool of candidates be relatively small, but also there would be less opportunity to promote the orderly replacement of members as senior members resign or retire. Although one could look to successful board membership performance in other journals in the recruitment of younger scholars, the more responsible position would be for all journals (especially major ones) to assume some of the burden of professional development in this key aspect of our science (Feldman, in press). Here again, the critique by Bedeian et al. (2009) relies on rather simple metrics—publications and citations—to make their case that Academy of Management journal boards are somehow inferior. These would seem to be weak surrogates for what should be a much richer approach to insuring key board features. As an aside, the apparent differences in publication or citation scores of the Academy of Management Journal versus the Academy of Management Review board members may in fact reflect differences in board functional requirements, given their respective missions. As a journal of conceptual and theoretical contributions, the Academy of Management Review might indeed need board members who are deeply familiar with the history and full range of contemporary thought leadership relative to a topic. Thus, the Academy of Management Review may need to employ board members who are more mature and well published and truly evaluate the potential of a conceptual contribution. But as the premier outlet for empirical work, one where detecting analytic errors are important, the Academy of Management Journal may need board members with the skills usually associated with a younger, more recently trained methodologist.4

Building a Credible and Effective Editorial Board: Policies and Practices as Inputs Having good members and having an appropriate set of such individuals are certainly on the critical path to a credible and effective Board. But these features are not sufficient. In addition, having effective policies and practices must also be in place. Such things as the nature and clarity of the criteria for publication as specified by the journal, the way that articles get assigned to reviewers, the standard for review cycle time, or the format, completeness, content, and tone of critiques as feedback to authors are usually well known and have an effect on manuscript submissions, and actual board processes clearly have an impact on board credibility.

Building a Credible and Effective Editorial Board: Good Processes Processes are activities that take place over time that are usually done with a goal or outcome in mind. A functional perspective would look to ensuring good processes aligned


245

with clear policies as drivers of and a reflection on board credibility. Clearly, such activities as soliciting, reviewing, and publishing manuscripts represent complex phenomena. In this regard, there are probably too many processes involved in performing the work of the typical editorial board to detail here. But a few examples might be highlighted to make the point that board composition could have an effect on journal credibility thru the impact that members’ activities have on processes (Denisi, in press; Kacmar, in press; Rynes, in press). Some examples of the latter include Managing relationships with authors: Paramount in the minds of most editors and reviewers is the need to treat the author in a respectful manner. Journal policies notwithstanding, the actual way that the needs and interests of authors are managed is often a defining attribute of a journal. Although senior members of boards certainly do more in this domain, reviewers can play a role, for example, by turning reviews in on time or in encouraging authors to submit their work. Effective relationships among reviewers: Most journals arrange to have board meetings at least once a year. These are critical times to create or to maintain a ‘‘reviewer culture,’’ one built around a shared vision for the journal and norms relative to standards for handling manuscripts and dealing with authors. The ways that reviewers are socialized, their needs and concerns are addressed, honest disagreements are handled, and feedback is given to them are all linked to journal quality and credibility.

A major weakness of the critique by Bedeian et al. (2009) is that it has little to say about the drivers of the perceived credibility of a journal as reflected in its processes. In this regard, it is not at all obvious how editorial board member attributes or board-level features play out against such processes. However, if popular stereotypes are to be believed, a case could be made that only using the kinds of academic ‘‘stars’’ that the Bedeian et al. article holds up as the appropriate for journal board membership will not have positive consequences. Given their presumed high levels of scholarly productivity and fame or notoriety, such individuals may not take the time or have the tolerance to contribute to smooth journal operations. Similarly, were only highly published members of our discipline used as board members, the ‘‘system’’ might suffer in additional ways. For example, because only a few colleagues would rise to the standard of achievement being advocated by Bedeian et al., our boards would be relatively small in size. Thus, the absolute number of articles that could be reviewed or the time needed to make a decision would suffer as a consequence. Moreover, these same credible and well-published scholars, even if they were to accede to requests for high levels of journal board service, would likely have to accept a reduction in their own levels of productivity, given real limits on the number of hours available for professional activities. It is not clear that having such productive colleagues reallocate their time this way is necessarily in the best interests of advancing scientific inquiry or the viability of our discipline.5 Because no data on board policies and processes (other than board member staffing) are offered, it is not clear just what can be detailed here relative to the status of Academy of Management journals (see Bedeian et al., 2009). However, because the Academy of Management Journal and the Academy of Management Review are services of the academy, journal editors have traditionally had to report on operations (business processes). If one were interested in documenting policy and practice parameters, the annual reports provided

246


to the Academy of Management Board of Governors by editors might be a place to start. Thus, statistics such as percentage of ‘‘desk rejections,’’ ‘‘average number of reviewers per manuscript,’’ ‘‘percentage of reviews provided by ad hoc reviewers,’’ ‘‘time to make a decision,’’ and so on are available. These could then be examined in a comparative way to establish if Academy of Management journals are indeed weak relative to the journals in other fields or to what are felt (normatively) to be progressive policy and practices. In fact, if these kinds of data were to be gathered, one could then go on to establish the impact, if any, of board member staffing criteria on such process or processing metrics.

Building a Credible and Effective Editorial Board: What if We Focus on Outcomes? Most groups and teams are brought into existence to accomplish something. The editorial board of a journal is no exception. Thus, it follows that any critique of board features (i.e., inputs and processes) should be done with this end (its mission, goal, or desired output) in mind. The mission of the typical academic journal board is to identify, select, or shape a collection of articles deemed worthy of dissemination. The articles themselves must be of high quality and have the potential to shape the intellectual landscape. With the help of professional staff (usually), these must be published in the form of a journal (or increasingly on a Web site) according to a schedule (monthly, bimonthly, quarterly). Although it may be obvious, the potential of a journal to have an impact on key outcomes is related to such things as the number of articles published a year, the lag in moving accepted articles to the public arena, the number of times that the journal comes out, and its circulation. These, then, become nontrivial operational parameters, even ‘‘proximal’’ outcomes to be monitored. With this as context, several more conspicuous outcomes seem relevant. Productivity might be defined relative to the number of articles or pages published a year. Both professional societies and for-profit publishers usually seek to assure members or subscribers that they will receive a certain level of productivity (pages, articles, issues) and according to a predictable schedule. Efficiency might be the ease (i.e., productivity relative to the level of resources available) with which this is done. Questions of effectiveness relate to how well the journal accomplishes its mission (e.g., the distribution of timely, high-quality, and useful articles) and the extent that stakeholder needs continue to be satisfied. Thus, each of these might be considered as a potential outcome of interest in any treatment of a critique of board member characteristics. A major weakness of the article by Bedeian et al. (2009) is that it does not communicate their views on board member attributes and board composition relative to journal outcomes. Some of the outcomes that might be examined in light of their critique include the following: Quantity of articles published. As implied, productivity is an outcome of interest to many stakeholders. Moreover, there are several standards of excellence that might be invoked. For journal owners and subscribers, as noted, there is an expectation that for the price or cost of the journal, a certain number of articles (pages) is to be produced each year. For the scholar, there is the hope that a well-crafted article will be accepted for


247

publication and be made available to the intellectual marketplace in a timely manner. Similarly, for the field as a whole, the amount of work published is a nontrivial issue as (ideally) good science depends on good science dissemination. Establishing the ‘‘right’’ amount of productivity then requires an informed approach. But as a heuristic, the number of article published relative to the number of submissions is often used to establish a benchmark. Clearly, as in the case of both Academy of Management Review and Academy of Management Journal, the number of submissions is quite large. One way to deal with this is, of course, to have an appropriately large editorial board. This, in turn, has implications for staffing. If one looks at the recent acceptance rates for both journals (e.g., 9% to 10%), they would appear to be normative, implying that the journal policy makers are staffing appropriately.6 Quality of articles published. Leaving aside operational metrics such as productivity or efficiency, a reasonable question to pose in light of the Bedeian et al. (2009) critique is how the arguably modest credentials of the Academy of Management Journal or the Academy of Management Review board might play out in terms of this key journal outcome. Are these journals failing in this regard? The authors are silent on this. But how would we know? One way would be to use the very same data cited by Bedeian et al. (2009), but in this case look at citation evidence, this time aggregated to the journal level. Such data examined in this way would reveal that both journals have earned a very favorable ‘‘impact factor’’ (Academy of Management Review = 4.52, Academy of Management Journal = 3.35). That is, it can be reasonably inferred that work published in these journals gets read and is found useful, useful enough to be cited. Given such scores, how can it be then that, as the authors assert, Academy of Management Journal and Academy of Management Review board members are not credible and competent (enough) to meet marketplace expectations? Stakeholder satisfaction. Bedeian et al. (2009) rightly point out that there are many critics of current practices relative to contemporary scientific and professional publications, including management journals. Some writers are clearly concerned that not all is well. On the other hand, there are indications that not all stakeholders are dissatisfied. One could make a case that journal impact indices also imply that, as stakeholders, the consumers of the science being reported in our journal publications are satisfied. After all, fellow investigators and peers are taking the time to read, to use, and to cite articles from our journals. Based on subscription or membership trends, readers of the Academy of Management Journal and the Academy of Management Review also apparently put their trust in the peer-review processes of both journals. Similarly, the producers of scholarship and research manuscripts would also appear satisfied, at least at some level. How else would we continue to see so many manuscripts being submitted to our journals each year? It would seem that, as stakeholders, the publishers of our Academy of Management journals are satisfied as well. That is, to my knowledge, the Academy of Management Board of Governors has not expressed concerns with the Academy of Management Review or Academy of Management Journal editorial board member qualities. Finally, although some might disagree, as stakeholders, the members of our editorial boards themselves appear satisfied. Anecdotal evidence, at least, suggests that very few board members

248


resign in midterm. Of course, from the perspective of the editors, there is probably some variability here across journals. But the two Academy of Management journals highlighted in the Bedeian et al. (2009) article enjoy prestige sufficient such that they seem to be able to recruit and retain board members for stable ‘‘tours’’ of duty. The needs of still other stakeholder groups appear implicated in the Bedeian et al. (2009) critique. To illustrate, the article makes reference to the notion of loosing ‘‘public confidence in our capacity as a discipline’’ (p. 231) and to disappointing ‘‘members of the scientific community’’ (p. 233). In this regard, it would indeed be useful to know just how well we, as researchers and as journal managers, are respected by this amorphous group of stakeholders—scientists and research opinion leaders outside our discipline and opinion leaders, practitioners, or policy makers. When it comes to managers, practitioners, or business opinion leaders, it would seem that we do know how the research in our journals is perceived. As pointed out by many (e.g., Rynes, 2007; including Bedeian et al., 2009), for the most part it is not that favorable. Along these lines, one can go to many sources, even to our leaders in our own discipline for insights as to what is wrong. To simplify, among other things, it is felt that our research, especially that which gets published in our premier journals such as the Academy of Management Review and the Academy of Management Journal, is not particularly accessible, interesting, or useful to those concerned with the problems of business or the practice of management. For example, in a recent issue of the Academy of Management Journal, Bartunek (2007) lamented that few academics are able to empathize with the problems and needs of practitioners to craft research that will uncover findings that can actually be implemented to solve organizational problems. She proposed several activities that would promote what she calls a ‘‘relational scholarship of integration,’’ implying changes in journal practices. Hambrick (2007) asserted that our love of theory has gotten in the way of gathering data that can ‘‘tell a good story’’—that is, he called for more research to be published in our best journals that has ‘‘simply documented and dissected a fascinating, important phenomenon’’ (p. 1348). He felt that this was already being done by premier journals in the areas of finance and marketing. And more to the point of this commentary, Pfeffer (2007) made clear that our journal editing practices may be part of the reason for practitioner dissatisfaction. In his essay, Pfeffer (2007) offered what he calls ‘‘modest proposals’’ (p. 1340). The one that seems most relevant to the Bedeian et al. (2009) article relates to how we go about staffing our journal editorial boards. Specifically, contrary to the position taken by Bedeian et al., Pfeffer suggested that to populate the Boards of our most prestigious journals only with well-published academics is a mistake. To promote the infusion of new thinking or innovation or to better meet practitioner needs for problem-focused research, we must appoint at least some board members who are not totally ‘‘invested in existing ways of thinking and doing things’’ or who are not ‘‘selected in large measure for their demonstrated socialization to the prevailing topics, theories and methodologies of a field’’ (p. 1342). Instead, we must find ways to use our journals, including our premier journals, to promote closer connections (e.g., through shared board membership) with those in professional practice. This would increase the likelihood that our published articles are both rigorous and relevant. This, in turn, would increase the satisfaction of


249

practitioner–stakeholders and assure them that the field’s most accomplished researchers (e.g., contributors to the Academy of Management Review or the Academy of Management Journal) are actually concerned about practice challenges. It seems less clear where we stand relative to our journal practices when it comes to stakeholders in the larger arena of academic research or scientific inquiry. To calibrate our status, however, two suggestions come to mind. One is to look toward the public positions being taken by the broader research community of a nation (or their representative agencies—NSF) relative to what its members consider to be credible scholars, credible journals, and therefore credible science. As an example, the U.K. government, as a matter of policy, has recently chosen to link levels of research funding for universities in the United Kingdom to the scholarly productivity of their faculty, using something called the Research Assessment Exercise (Macdonald & Kam, 2007). Scholarly productivity, in turn, is being operationalized (in part) by faculty publications in credible journals. Business schools are especially affected by this arrangement (Macdonald & Kam, 2007). Concerns about the merits of this approach notwithstanding, from the point of view of this commentary, it is significant that the Academy of Management Journal and the Academy of Management Review are clearly positioned on the lists to be of the highest quality (Harvey, Morris, & Kelly, 2007). These two journals, in fact, receive the same high status as those that served as the basis of comparison in the Bedeian et al. (2009) article. Thus, although there may be differences in certain board member credentials (as inputs) across the sample of journals studied by Bedeian et al., from the point of view of policy makers, there appear to be no differences in outcomes (e.g., in the credibility that they garner). A second notion derives from the capacity for mining citation data derived from journals that are not central to a discipline but are core to other disciplines. Such ‘‘co-citations’’ can be used to infer the extent to which the work of a discipline (e.g., management) is found to be important and useful. Along these lines, as pointed out by Cacioppo (2007) and Boyack, Klavans, and Borner (2005), such efforts have been able to detect that a field such as psychology is actually a ‘‘hub’’ science. It produces discoveries that not only get published in its own discipline-oriented journals, but such work is cited by researchers in many other fields (e.g., nursing, public health, neuroscience, etc). This, or related kinds of analysis (e.g., Agarwal & Hoetker, 2007; Biehl, Kim, & Wade, 2006), might reveal the extent to which articles in the Academy of Management Review or the Academy of Management Journal are being co-cited, and by inference, support the argument that these journals do or do not have credibility in the eyes of the members of the larger scientific community. But such work remains to be done regarding both the methodology and the meaning of such co-citations (Agarwal & Hoetker, 2007). I recognize that each of the potential ‘‘outcomes’’ being suggested in this commentary has limitations as a source of evidence with which to support or contradict the claim that the academy’s board staffing practices are inferior (or inappropriate) and must be changed. However, the major point to be made is that any critique of staffing practices should demonstrate that, when it comes to key outcomes (the ‘‘end in mind’’ cited at the outset of this commentary), such staffing practices are indeed consequential and negative. This has not been done by the authors.

250


Conclusions This commentary takes a functional perspective as a way to evaluate the merits of the position adopted by Bedeian et al. (2009). They assert that Academy of Management journals are comparatively disadvantaged because of the pattern of staffing of their editorial boards, and, as a consequence, they are not up to fulfilling their mission. In my analysis I examined the journal editorial board (as I would any work group) with regard to the status of their inputs (including board member attributes), processes, and products or outcomes. When it comes to fulfilling their mission (products or outcomes) in particular, this analysis failed to reveal any substantial weaknesses. Thus, I find that the critique by Bedeian et al. is, at best, incomplete in its considerations. At worst, the case being made is overstated, even alarmist. I would certainly not argue that there is no room for improvement relative to our journal enterprises. The vetting and distributing scholarly work in our discipline is so important; continuous improvement should be our mantra. This is especially true in light of such things as new ways of doing research (more interdisciplinary), new forms of distribution technology (Web), and emerging publication practices (e.g., consolidation, niche marketing). Indeed, we must be prepared to make what may be major changes in how we go about carrying out this very important part of ‘‘good science.’’ On the other hand, I do not agree with the authors’ position that board staffing is problematic. I do not feel that we must modify or increase our standards relative to criteria for Academy of Management journal editorial board membership. To put it simply, Bedeian et al. (2009) have not made a compelling case that ‘‘the credibility enjoyed by our discipline and the spirit of scientific inquiry’’ (p. 234) is at risk by retaining our current paradigm. As they say, ‘‘If it ain’t broke, don’t fix it!’’

Notes 1. For another person’s view of what might constitute desirable qualities in board members or reviewers, see Feldman (in press). 2. Pfeffer (2007) makes an interesting argument in his examination of what is wrong with our research enterprise. He points out that ‘‘insiders’’ are often not likely to recognize and support innovation or new approaches to problems (or research). If one of our goals is to promote academic management research that has impact and application in mind, it might make sense that we appoint at least some editorial board members who have more experience with the problems facing those managing work organizations and less traditional subject-matter knowledge. Of course, such individuals are not likely to have numerous publications or citations. 3. Although this commentary does not focus on the method of the Bedeian et al. (2009) article, it seems useful to point out a challenging temporal issue in using citation data. The latter require years to accumulate, whereas a scholar’s reputation can be built up on contemporaneous marketplace information such as one’s recent publications, honors, awards, or grant activity. In this regard, the citation data reported by Bedeian et al. for (some) current board members may be misleading, especially if they are only recently published. 4. Thanks to David Kravitz for suggesting this based on his review of an earlier draft of this commentary. 5. Thanks again to David Kravitz for this notion. Also see a recent article by Miller (2007), in which he points out how few senior scholars are available for participating in the peer review of grant proposals to the National Institutes of Health. He calls this ‘‘senior attrition.’’ He too stresses the need to maintain an age


251

(accomplishment) distribution among scholars to ensure an adequate pool of potential reviewers, even when such a prestigious assignment is in the offering. 6. Along these lines, the American Economics Association, publishers of American Economic Review (one of the journals used as a basis for comparison in the Bedeian et al., 2009, article), apparently feel that their capacity for productivity is too low. They are now sponsoring the formation of four new quarterly journals (Glenn, 2008) to increase access to good research.

References Agarwal, R., & Hoetker, G. (2007). A Faustian bargain? The growth of management and its relationship to related disciplines. Academy of Management Journal, 50(6), 1304–1322. Bartunek, J. M. (2007). Academic-practitioner collaboration need not require joint or relevant research: Toward a relational scholarship of integration. Academy of Management Journal, 50(6), 1323–1333. Bedeian, A. G., Van Fleet, D. D., & Hyman, H. H. (2009). Scientific achievement and editorial board membership. Organizational Research Methods, 12(2), 211-238. Biehl, M., Kim, H., & Wade, M. (2006). Relationships among the academic business disciplines: A multimethod citation analysis. Omega, 34, 359–371. Borman, W. C., & Motowidlo, S. J. (1997). Task performance and contextual performance: The meaning for personnel selection research. Human Performance, 10, 99–109. Boyack, K. W., Klavans, R., & Borner, K. (2005). Mapping the backbone of science. Scientometrics, 64, 341–374. Cacioppo, J. (2007). Psychology is a hub science. Observer, 20(8), 3–4. Denisi, A. (in press). Managing the editorial review process: It’s the people that matter. In Y. Baruch, A. Konrad, H. Aguinis, & W. H. Starbuck (Eds.), Opening the black box of editorship. Basingstoke, UK: Palgrave Macmillan. Feldman, D. C. (in press). Building and maintaining a strong editorial board and core of as hoc reviewers. In Y. Baruch, A. Konrad, H. Aguinis, & W. H. Starbuck (Eds.), Opening the black box of editorship. Basingstoke, UK: Palgrave Macmillan. Glenn, D. (2008, February 1). New economics journals: Supply, demand, risk. Chronicle of Higher Education, p. A10. Hambrick, D. C. (2007). The field of management’s devotion to theory: Too much of a good thing? Academy of Management Journal, 50, 1346–1352. Harvey, C., Morris, H., & Kelly, A. (Eds.). (2007). Academic journal quality guide. London: Association of Business Schools. Kacmar, K. M. (in press). Setting up an effective manuscript review process. In Y. Baruch, A. Konrad, H. Aguinis, & W. H. Starbuck (Eds.), Opening the black box of editorship. Basingstoke, UK: Palgrave Macmillan. Klimoski, R. J. (in press). Team performance effectiveness. In J. Levine & M. Hogg (Eds.), Encyclopedia of group processes and intergroup relations. Thousand Oaks, CA: Sage. Klimoski, R. J., & Zukin, L. (1999). Selection and staffing for team effectiveness. In E. Sundstrom (Ed.), Supporting work team effectiveness (pp. 63–94). San Francisco: Jossey-Bass. Macdonald, S., Kam, J. (2007). Aardvark et al.: Quality journals and gamesmanship in management studies. Journal of Information Science, 33, 702-717. Marks, M. A., Zaccaro, S. J., & Mathieu, J. E. (2001). A temporally based framework and taxonomy of team processes. Academy of Management Review, 26, 356–376. Miller, G. A. (2007). NIH examines peer review. Observer, 20(11), 25–27. Pfeffer, J. (2007). A modest proposal: How might we change the process and product of managerial research. Academy of Management Journal, 50(6), 1334–1345. Rynes, S. L. (2007). Editors foreword: Tackling the ‘‘great divide’’ between research production and dissemination in human resource management. Academy of Management Journal, 50(5), 985–986. Rynes, S. L. (in press). Communicating with authors. In Y. Baruch, A. Konrad, H. Aguinis, & W. H. Starbuck (Eds.), Opening the black box of editorship. Basingstoke, UK: Palgrave Macmillan.

252


Schmitt, N., & Borman, W. C. (1993). Personnel selection in organizations. San Francisco: Jossey-Bass. Tsui, A. S., & Hollenbeck, J. R. (2009). Successful authors and effective reviewers: Balancing supply and demand in the organizational sciences. Organizational Research Methods 12(2), 253-258. Richard Klimoski serves as dean of the School of Management at George Mason University in Fairfax, Virginia. His teaching and research interests center on strengthening the human side of work organizations, especially through effective management of teams.