The transition from print to electronic publishing allows us to reconsider many of the norms that have evolved within traditional (print) publishing models. Electronic publishing has created new challenges, but it has also generated opportunities to improve both the publication process and the resulting papers. In this essay, we describe some of the notable changes that distinguish online publication from print publication and outline several of the resulting challenges and opportunities. To illustrate the potential of new forms of publishing, we reflect on our experience helping to launch and serving as the inaugural editors of Socius, the new open access journal of the American Sociological Association.
Perverse costs of traditional print publishing
Two factors underlie many of the goals and behaviors of the traditional publication process: hard page limits and a lengthy (and routinized) production process that discourages public discussion of published papers. Publishing academic print journals is expensive, and restrictions on the total number of pages published are the only real way publishers can control costs. Gatekeepers—including editors, editorial boards, and reviewers—want to ensure rigorous scientific review and recognize the need for a system to ensure that these limited pages are filled with the very best papers. This pushes editors and reviewers away from a simple evaluation of accuracy and towards a ranking, which has led to the creation of the rigorous, but extremely lengthy, review process common in many journals.
Print publication is similarly limited by the physical printing schedule. To achieve some economies of scale, papers are bundled into issues, issues are turned into volumes, and a print schedule is adhered to. In this system, discussion of interesting or controversial claims and findings must wait for subsequent publications or alternative forums such as at conferences; but these are usually separated from the original article by considerable time. And, in the case of a published response, the time lag can be substantial given that the author needs to write the paper and usher it—hopefully successfully—through a similarly rigorous review process. Erroneous findings create a separate problem: the traditional review process weeds out many errors, but it is not perfect, and errors in published papers are inevitable. When an erroneous paper is published, there is no ready forum to identify, discuss, and correct that error. While comments and replies are the norm for addressing the biggest issues, these are also subject to lengthy review and need to be serious enough to warrant using additional print pages.
Over time, these limiting facts have generated serious negative consequences. In an attempt to ensure that only the very best papers appear, well-meaning editors and editorial boards engage in what has become known as a developmental review process in which good papers are supposedly developed into great ones worthy of the scarce pages. When an erroneous paper is published, there is no ready forum to identify, discuss, and correct that error.Since features that distinguish good papers from great papers are amorphous and not necessarily widely agreed on, the already-lengthy review process has become more involved and longer than ever as authors attempt to please editors and multiple, often conflicting reviewers. Editors, seeking to avoid making bad or seemingly arbitrary decisions, seek unanimity amongst reviewers and often add new reviewers in second or third rounds of “revise and resubmit” to ensure that all bases are covered. Because authors need to publish in top-ranked journals to signify important contributions to promotion and tenure committees, who often cannot know the inherent value of the work, they have little choice but to participate in the broken system. The process costs editors and reviewers as well: both can be exhausted by the multiple rounds of reviews and the expectation that they will write detailed, developmental reviews. This limits the number of reviews that a single reviewer can complete, each review takes more time, and it becomes harder for journals to find qualified, available experts to evaluate papers. As this process feeds on itself, the time to publication—and the resulting papers—become longer, reviewers and authors more exhausted and frustrated, and a good deal of excellent research languishes unread behind review backlogs.
The task of pleasing multiple reviewers shapes the style of research and related information that is published. Although some excellent papers are certainly published, many include elaborate theoretical discussions that have been included merely to make a paper seem more important or to provide cover by citing any potential reviewer. It is ironic given the limit on print pages, but papers have become overly long and now include more cites than in the past. Consider figure 1 below, which plots the number of references in sociology papers over time.
Figure 1. Average number of references listed in the bibliography of papers published in sociology from 1970 to 2015. Authors’ construction from Web of Science data.
Other papers appear to be overly safe—and perhaps less interesting than they might be—because authors have learned that a safe paper takes few risks. Our impression is that a paper that says very little that is interesting and takes few theoretical or methodological risks is more likely than a bold, interesting one to please reviewers and editors and more likely to be published as a result. Nearly all published papers have the same structure (i.e., introduction, theory, methods, etc.); and while this makes for efficient distilling of content, it limits the options for publishing papers that do not conform structurally. One of the most unfortunate drawbacks is that the papers that are published quite often appear well after they are relevant to anything but rather arcane academic debates. The lengthy review process—which can often span years—makes it difficult for social scientists to weigh in on current events while they are still current. The lengthy review process is particularly challenging given that traditional journals do not hesitate to reject papers even after years in review, forcing authors to start the process again at a new journal. Of course the length of time to publication varies by journal, and journals are reluctant to publicize this information, making it difficult to substantiate what has become a common event and a popular topic of conversation at sociology conferences. There can also be considerable ambiguity about what constitutes an error. Minor issues with data, changing understanding of statistical processes, and simple disagreements among experts about the best way to discuss and study various issues likely warrant discussion but may not be worthy of pages dedicated to a comment and reply. In traditional print publishing, these issues thus remain unpublished, seen only by authors and reviewers who discuss the issues at length in review and response memos. The reviews and responses can take enormous amounts of time for both authors and reviewers and are sometimes as long as published articles. Sadly, these reviews can contain extremely valuable information; but because there is no forum for making them public, the discussion is never visible to most of those interested in the work.
Opportunities of new media publishing models
New forms of electronic publishing, including open access journals, have the advantage of virtually unlimited space, rapid publication cycles, and easy-to-access forums for discussing claims and findings. These seemingly simple differences between new and traditional forms of publishing are, in reality, monumental, and some open access journals have emerged to provide publication outlets that can capitalize on these opportunities. Socius is an example of a journal that publishes innovative, rigorously reviewed scholarship that spans sociology subfields and provides free and rapid access to users across the world. Although Socius is not modeled after any other journal, the challenges we describe above motivated much of the current practice in this new venture.
Indeed, Socius has goals that differ notably from traditional print publications. One common goal is to publish high-quality, rigorously reviewed scientific research online—similar to other top journals—and all papers must contribute novel and accurate information to expanding current knowledge. However, that is largely where the similarity ends. For readers, online publication makes the findings available worldwide and at no cost. This opens a wider audience to authors, including policymakers and the general public, who might not otherwise get a chance to read the work. Because online journals do not have to bundle articles into issues to save publication and shipping costs, papers can appear as soon as they are accepted and edited, which allows readers access to work in a much timelier manner.
A deeper change comes with the freedom to publish as many good papers as are submitted because we can reasonably abandon the developmental review process and take more risks. Moreover, we have no page limits, and we have no requirement to rank papers. Instead, we can accept any paper that is empirically accurate and theoretically novel. This has carryover effects for editors, reviewers, and most importantly authors and the community of science. First, editors shift the initial burden of writing a clear and novel paper to the author by having a high desk-reject rate. Editors of open access journals typically desk reject submissions at a higher rate than in traditional print publishing; at Socius our rate is about 50 percent. By contrast, in 2014, the American Sociological Review rejected 9 percent of papers outright. Even in traditional publishing, it is often clear that a paper is unlikely to be published, but the notion of a developmental review process involves allowing a larger number of potential papers to continue in the review process in the hopes that they can be groomed into papers that are worthy of scarce print pages.A deeper change comes with the freedom to publish as many good papers as are submitted. At Socius, our assumption is that reviewers will not provide detailed developmental reviews; thus, if a paper clearly needs editing, writing, and analysis help, authors will have to find support from mentors and collaborators. Although this strategy places a greater burden on authors, it also assumes that capable researchers are already trained and do not need further development by reviewers and editors. Second, we do not expect reviewers to review papers multiple times: the review process becomes essentially and up-or-out affair. In fields where consensus is naturally low, such as much of social science, we do not need consensus among reviewers to publish papers. Instead, controversial or not-quite-perfect papers can be published, and the fast turnaround time allows the community of scholars opportunities to build on (and sometimes correct) the work.
There are also advantages of the new model for reviewers. In particular, abandoning the developmental review process allows reviewers to focus on the accuracy, novelty, and presentation of the paper because the costs of publishing a less-than-perfect paper are lower than in traditional print media. Because editors have culled papers that are not yet ready, reviewers can focus on the substantive contribution of the paper as it stands, which shifts reviewing from a relative ranking to an absolute rating: does a paper, perhaps with minor tweaks, contribute to our understanding of the world? Of course, many years practice in the developmental and ranking model predisposes social scientists to critique and rejection, so a good deal of work likely needs to go into thinking carefully about what these absolute standards are and how small a contribution is still a contribution, which again shifts more discretion to editors. Yet the goal is to have a clear and compassionate review process that provides clarity to authors while simultaneously lowering the burden for reviewers.
Papers published in Socius are also not restricted by the structure or length common in print publication. Because the reviewer burden is lower and editors are more hands-on, authors can expect faster turnaround times, clear review expectations, and a pleasant production process that is driven by author schedules rather than print schedules. Turnaround time (from submission to publication) varies with the length of the paper because review times will necessarily vary; however, turnaround is faster for all submissions than it is in traditional journals. Because we are not sending papers back to reviewers over multiple rounds of reviews, reviewers and editors need to provide authors with clear instructions for necessary revisions. If the revisions needed are extensive, then the paper is effectively not ready for publication and is rejected; but with a fast turnaround time authors can work with mentors and collaborators to refine the work for later submission. Having no press deadline for a publication frees authors from unreasonably short turnaround times in the production process, and authors can take all the time they want to copyedit work. Our experience, however, is that authors are motivated to turn their papers around quickly because they want to see them published.
Open access journals like Socius certainly include traditional papers, but they are also logical venues for publishing more innovative papers that are consistent with contemporary modes of knowledge dissemination. For example, it is possible to publish papers that focus on findings. Because page limits are not an issue, it is not necessary to restrict publication to the elusive “best” papers. Many papers that simply identify an important finding do not need to be couched in elaborate theoretical discussions, and open access publication readily allows for these papers to be published, sparking discussion and theoretical elaboration by others. Similarly, papers that are pure theory or formal theory that might have traditionally been neglected can be included. Likewise, papers that rely on qualitative data that might need to be longer than a traditional paper can easily be accommodated. Perhaps most importantly, the online format of an open source journal is the logical place to publish papers that include visualizations and simulations and any other accompanying materials that are increasingly common and a poor fit for the traditional journal.
The rapid and open nature of online publication also opens discussion to the community of scholars. If reviewers disagree with a paper on matters of theory or interpretation that reasonable scientists could argue about, they can publish such concerns as comments rather than burying them in reviews that will go unrecognized. Socius encourages online comments and will also publish comments and replies. Papers are readily available, and an associated forum for discussing them allows readers to comment and discuss the papers without delay.
Despite all of these innovations, one component of the academic publishing process remains critical: the centrality and absolute necessity of a rigorous review process. Print pages may no longer be scarce, but this is not an invitation to publish anything that is written. Likewise, the ready availability of an online forum for discussing papers and correcting errors may reduce the need to hold papers until they are as close to perfect as possible; however, the ability to discuss papers publicly does not excuse sloppy or lax editing or reviewing. Rather, the fact that most anyone can share their perspectives and opinions so readily now creates challenges of its own. The volume of information available today—much of it opinion couched as unsubstantiated fact, in forms such as blogs—suggests that rigorous peer review is more important than ever. The model we have proposed also assumes an active editorial team, which will invariably make mistakes. Tolerance for such mistakes is made easier knowing that rebuttal and publication are simpler, but this is a real tradeoff. There will always be variation among academic journals in the quality of review. There are certainly open access journals that do not maintain high standards for review; of course there are also traditional journals with low standards. But rigorous peer review should always remain central to the scientific process.
