In addition to the work the CHI PLAY Steering Committee (SC) does on improving accessibility to, inclusion within, and the quality of the conference experience itself, we also consider how we can improve the processes and policies around paper review and publication. Through discussions with our community and its leaders over the years, we have identified many challenges in three main categories (quality, equity, and reach), which can all be addressed partially by one single change.
In CHI PLAY 2021, we introduced a revise and resubmit phase to the review process, and moved publications of our full papers to the journal PACMHCI, the home journal of other SIGCHI communities like CSCW, EICS, ISS, and Group. Our extended abstract non-archival venues (e.g., late-breaking work, doctoral consortium, student competitions) are still published using a companion proceedings model.
By replacing the 1-week rebuttal with a multi-week revise and resubmit (R&R) phase, we were able to improve paper and review quality, the equity of the submission and review process, and the reach of our papers.
QUALITY
Quality of review process: By definition, a revision cycle should improve the review process by enabling more interaction between authors and reviewers and by affording authors the chance to meaningfully respond to reviewer feedback.
Paper quality: Being able to meaningfully address reviewer feedback should improve papers that were already strong enough to be accepted and also some that would not have made the cut. We all benefit from being able to engage with, and respond to, critique within the peer-review process.
Review quality: Authors should always be able to expect quality reviews at CHI PLAY, even if they disagree with the reviewer’s opinion (looking at you, R2). An R&R cycle allows for reviewers to engage more deeply with papers over two review cycles, and also affords an opportunity for us to provide additional reviews in cases where initial ones do not meet our quality standards.
EQUITY
English-writing bias: We believe that papers with excellent research methods and contributions should not be rejected due to our requirement of publishing in English. However, we also recognize that there needs to be a communication standard and that this can be hard to reach for researchers who do not write in English. The R&R phase allows for these papers to be improved, rather than rejected outright.
Methods bias: CHI PLAY welcomes many methodological approaches, though with differing prevalence. Reviewer misunderstandings with less-dominant methods can be corrected when there is space for interaction between authors and reviewers in an R&R phase.
Topic bias: CHI PLAY also welcomes a diverse range of topics in games and play. New topics might be less familiar to reviewers and the R&R phase allows for greater communication and building to a shared understanding.
Community / Experience bias: Those who are new to our field may not yet understand our publication norms. The R&R cycle may afford authors from related disciplines to adjust submissions to meet our community’s norms and standards for acceptance, rather than rejecting these papers outright.
Institution / Seniority bias: Because associate editors who handle the papers know the authors and their institutions, there is always the chance of introducing bias in how papers from more recognizable authors or institutions are managed, relative to papers from unknown authors or institutions. Introducing an R&R phase helps to rectify this by giving papers a second look and by allowing authors to engage with the reviewers.
Journal bias for academics in certain fields: Although our parent organization is the Association for Computing Machinery (ACM), many of our authors are in other disciplines with no culture of conference publishing. Publications are important for tenure and promotion in academic contexts, and we aim to include researchers from all disciplines contributing to games and play. By moving to a journal, we welcome submissions from these authors, level the playing field, and encourage researchers to submit their best work to CHI PLAY.
REACH
Citation barriers in disciplines: Our field is interdisciplinary and our colleagues in other disciplines sometimes do not cite our CHI PLAY work when they should, because it is unclear how our conference papers are to be cited. We want to extend the reach of our work into these related disciplines and ensure that our research contributions are cited. Moving to a journal better supports citations of CHI PLAY research from related disciplines. The PACMHCI editorial team is actively considering how to improve the journal metrics.
Inclusion in meta-research: Our field has matured to the point that we now see an increase in systematic reviews, scoping reviews, and meta-analyses. We want our CHI PLAY contributions to be included in these reviews, which often only include peer-reviewed journals, and not conferences. By moving to PACMHCI, we increase the chance of our work being included in meta-research contributions.
We understand that there are tradeoffs to this decision, but we know that there is no other single move that we could make that would better address existing inequities in submission and review, help improve the quality of the review process, and increase the reach of our community’s research. Based on our post-mortems of our previous processes, we have further adjusted how we approach R&R for CHI PLAY. Overall, we conclude that for our community, the introduction of the R&R process was effective at supporting our intended goals of improving quality, equity, and reach.
We look forward to your full paper submissions!
Kathrin Gerling
PACMHCI Associate Editor for CHI PLAY
