This year, CHI PLAY received 208 complete submissions to the full papers track; however, 1 was withdrawn after submission. This is an increase over 2023, which saw 178 full submissions.
In terms of primary contribution, we received the following number of self-identified primary contribution submission types. We continue to see a range of contribution types at CHI PLAY, with no one type dominating over all others.
We received no author requests for alternative contribution types.
Review Process
All papers were initially screened by the three paper chairs for completeness, violation of the anonymization policy, scope within the CHI PLAY 2024 Call for Papers, and in consideration of the desk reject policy outlined in the CHI PLAY 2024 Call for Papers; papers that did not meet these criteria were discussed among the paper chairs and desk rejected.
The paper chairs then assigned a primary associate chair (1AC) and a secondary associate chair (2AC) to each paper. The 1AC and 2AC read the submissions to determine whether they should be assigned reviewers or rejected for the reasons specified above, after a more thorough read. These papers were considered by the three paper chairs, in discussion with the ACs who had been assigned them, before making a decision to send them for review or include them in the desk rejected papers.
Out of the 208 submitted papers, one was withdrawn, and 28 were desk rejected; 179 papers entered the full review process.
Papers were then assigned to two external reviewers and the 2AC provided an additional external review. After reviews were completed and checked for quality, the 1AC initiated discussion amongst the reviewers and came up with a preliminary recommendation (accept, between accept and revise and resubmit, revise and resubmit, between revise and resubmit and reject, reject).
We then held a program committee (PC) meeting over two consecutive days with the ACs to determine decisions for the first round of review (accept with minor revisions, revise and resubmit, reject). We discussed papers with a range of decisions and reviewer agreement to help calibrate the committee and train the new ACs. We then focused the remaining discussion time on papers that were in the middle three categories, asking ACs to raise papers in the clear accept and clear reject categories only if there was a need to reconsider.
The inclusion of the PC meeting at this stage was a new addition to the review process in 2023, and we felt that it was very helpful to better calibrate decision making across the committee at this critical point in the review process, ensure consistency in outcomes, promote reflection and consideration at this stage in the process, and mentor newer ACs in the review process. There will be no synchronous PC meeting after the revisions.
Paper Decisions
- Accept with Minor Revisions (19 papers): Submissions that receive this decision are ready or nearly ready for publication, though they may require a few small changes. The paper is likely to be accepted with minor revisions such as the integration and contextualization of new references, additional information on aspects such as system implementation, analyses, perspectives in the discussion, or acknowledgement of limitations of the work. The revised version of the paper must be submitted by the revisions deadline for consideration by the corresponding associate chairs; however, these papers do not necessarily go out to the external reviewers again.
- Revise and Resubmit (62 papers): Submissions that receive this decision have real potential, but will require major portions to be rewritten or redone, and then re-reviewed. The paper may be accepted pending major revisions such as clarifying the motivation, including new literature, recontextualizing discussion of the work, including new analyses, extension of designs, development of new system components, or adjustment of algorithms. This recommendation was only chosen if the work to be completed is not likely to fundamentally change the paper. The revised version of the paper must be submitted by the revision deadline and will go out to external reviewers again for re-review.
- Reject (98 papers): The submission has profound weaknesses in one or more areas, and should not be included in the conference this year.
- Desk Reject or Withdrawn (29 papers).
For papers that are accepted with minor revisions, the second round of reviews will be a process in which the 1AC and 2AC check the revisions, whereas papers that received Revise and Resubmit will receive another round of external reviews with the same reviewers in most cases, and will also be subject to a final selection process together with the paper chairs, based on reviewer discussion. This makes this process fundamentally different from, for example, a major revision of a journal article. This is treated as a real resubmission, which means that it is given full consideration but no guarantee or preferential treatment towards acceptance; it is likely that many of the revised papers will ultimately be rejected.
Considering the second round of selection for Revise and Resubmit will be competitive, we have asked committee members to give clear indications where borderline decisions were made, in the hope that authors will be able to decide whether they wish to pursue the second round of reviews, or to remove their submission from consideration, so that their work can be submitted to another venue, which might allow more time for changes to be made.
We hope that the first round reviews will provide authors with constructive feedback on how to improve their submissions. We would also like to take this opportunity to thank all committee members for volunteering their time, as well as all reviewers who contributed their expertise. Peer review is fundamental to the academic publishing process, and we are committed to maintaining a high-quality review process for CHI PLAY full papers. We are grateful to those who volunteered their time to contribute to this essential part of publishing research.
We are happy to accept feedback on the review process from authors, reviewers, or committee members so that we can improve or clarify the process in the future. Please feel free to email the paper chairs (papers@chiplay.acm.org) with any feedback.
Regan Mandryk, Alena Denisova, and Julian Frommel
CHI PLAY 2024 Paper Chairs