A Multi-dimensional Scoring System for Interdisciplinary Research Proposals

Numerous efforts are underway to promote interdisciplinary research and scholarship, although interdisciplinary is not all that its proponents claim. The term “interdisciplinary” is not synonymous with “innovative,” and interdisciplinary collaboration is not the exclusive road to solving the world’s pressing challenges. Disciplines are not isolated silos, but rather open and dynamic social arrangements that are indispensable for the organization of research and the training of students in our ever-more specialized world. These ideas are developed in my book, In Defense of Disciplines.

But these and other considerations will not and should not preclude the pursuit of interdisciplinary research. In this short essay, I endeavor to advance our understanding of how to assess interdisciplinary research proposals. Considerable ambiguity remains regarding what makes a project interdisciplinary. My goal is to clarify several essential aspects of interdisciplinary research, and to propose a simple extension of the National Institutes of Health (NIH) evaluation system for evaluating the degree to which a project may be credited as being interdisciplinary.

Interdisciplinarity is often equated with cross-disciplinary collaboration. While such collaborations represent one form of interdisciplinarity, we should also recognize that interdisciplinarity can be seen in the roots of ideas as well as the likely impact of the research. What I am suggesting is a reconceptualization of the distinction between disciplinarity and interdisciplinarity as multi-faceted rather than defined by any single indicator. Furthermore, each of these dimensions of interdisciplinarity is best understood as a continuum rather than a dichotomy. Rather than establish a dividing line where disciplinary ideas are on one side and interdisciplinary projects on the other, we can construct a multi-dimensional index which rates some research proposals as more interdisciplinary than others. An interdisciplinarity score could be factored into the assessment of a research project along with evaluations of the study’s innovativeness, research design and likely significance.

I envision the creation of an interdisciplinary scoring system with three components. The first is the diversity of source ideas. Projects are interdisciplinary if they draw on ideas from distinct literatures. The diversity reflected in the reference section may sometimes bring together ideas from several or perhaps a range of disciplinary sources. In some cases, ideas may largely reflect a single discipline but may represent an unusual combination of subfields.

This diversity score could be judged by expert evaluators just as other aspects of proposals are evaluated. The referees could take into account their diversity of articles cited in the reference section, but the evaluation need not necessarily be directly proportional to the number of cross-field references. The evaluation could be based on the extent to which the ideas represented a novel combination of ideas and not simply a count of how many fields were represented in the bibliography.

Just to be clear: a study could get a high score on this component of interdisciplinarity even if there were only one author and thus no interdisciplinary collaboration or team. For example, a project on the future of work that combines insights from labor market economics, organizational theory, and the history of technology could be judged as interdisciplinary whether or not this project was staffed by multiple investigators from different fields. The “diversity of source ideas” score would be based on the use of these disparate bodies of knowledge, not on the CVs of the investigators.

A second facet of interdisciplinary focuses on the backgrounds of the investigators in collaborative projects. This aspect of interdisciplinarity could be assigned an interdisciplinary collaboration score. Again, this dimension should be conceptualized as a continuum. Some cross-field connections involve neighboring fields where collaboration is common, while in other instances interdisciplinary collaboration involves unusual pairings. An interdisciplinarity score could also factor in the field of the researchers’ degree as well as the researcher’s current position.

Should collaboration between a biochemist and a bioengineer be considered interdisciplinary? What about collaboration between three neuroscientists, one of whom is based in a biology department, one in a neuroscience institute and a third in a medical school? These examples suggest that any simple dichotomy that classifies certain collaborative teams as interdisciplinary based on current departmental affiliation runs the risk of arbitrary inclusions and exclusions.

Rather than force a yes-no answer to this question, a review committee could rate collaborative proposals based on the degree to which the team brings disparate skills and interests to the projection.  A review committee might give a higher interdisciplinarity score to a collaboration between economist and an anthropologist exploring the cross-cultural applicability of game theory because this type of collaboration remains uncommon, while a proposal from a political scientist and a sociologist investigating public opinion data might not receive as high a score because the cross-field intellectual distance in this type of survey research is modest. Similarly, the collaboration between an economist and a psychology on a behavioral psychology project certainly qualifies as interdisciplinary, but this type of collaboration is not as novel today as it once was.

Finally, interdisciplinarity should also be measured in terms of the breadth of impact of the proposal. Some disciplinary studies have wide impact across fields; new methods and statistics often fit this bill. For example, advances in molecular-level microscopy launched the field of nanotechnology, and has opened up research frontiers from materials science to medicine to computing, just to name a few.

Conversely, despite popular assumptions to the contrary, cross-field studies are often narrow in scope and do not themselves foment new cross-disciplinary ferment or impact. For example, my study of over 700 scholarly journals established in 2008 found that many journals which included the term “interdisciplinary” or “multidisciplinary” in their mission statements were often quite narrowly focused, sometimes on a particular social issue or a single medical condition. While this type of “specialized interdisiciplinarity” may sound strange to those who assume the term “interdisciplinary” is synonymous with “broad,” my findings suggest that this type of research is quite common. In summary, the extent to which a project is likely to contribute to advances in more than one field is conceptually distinct from the breadth of reference cited and the disciplinary backgrounds of the investigators.

Developing a multi-dimensional interdisciplinarity score would be valuable for many reasons. It would help to more accurately assess the level of interdisciplinarity of a proposal. And it would help to clarify the fact that interdisicplinarity in intellectual origins, in the nature of the collaboration (if any), and in the impact of the research are distinct facets of a research project that are important to consider separately. Over time, research might examine these scores to explore whether these separate indicators correlate with one another, and whether these measures, in isolation or in combination, contribute to major intellectual and practical breakthroughs. The development of scales along these lines would also help to underscore the notion that there are many aspects of research proposals, including innovativeness, design quality, potential impact as well as interdisciplinarity. These and others are all distinct concepts that should not be conflated.

In some ways, interdisciplinary reviews may lead researchers in a conservative direction. In other words, in order to pass muster with a particularly broad range of reviewers, it may be that reliance on established methods and protocols is the safest bet in the efforts to obtain positive reviews and secure funding. Again, the equation of “interdisciplinary” and “innovative” is by no means automatic.

There are many challenging issues pertaining to interdisciplinary review processes, including how to meld the insights of scholars from different disciplines. Michèle Lamont’s work suggests that face-to-face meetings contribute to the achievement of a degree of consensus, although there is no guarantee that this process will always produce optimal decisions or even end amicably.

The three aspects of interdisciplinarity highlighted here are roughly chronological. The intellectual roots are intertwined with the formation of a project; the collaboration occurs during a research project and the impact follows upon the completion of the study. As the focus here is on the evaluation of individual research proposals, still another phase is not discussed, namely how best to institutionalize the cross-field connections into a program of research or perhaps into its own field of study. In other words, it is best to think about a truly successful cross-field collaboration as one that helps to foster a new line of research. There are many complex issues to consider in this phase of interdisciplinarity as well.

This short note will not resolve all of the challenges in this area, but may help to contribute to a clearer understanding of the different facets of interdisciplinarity and how these might be best evaluated in the research proposal stage.