AAAI-21 Reviewing Process:
Two-Phase Reviewing and NeurIPS/EMNLP Fast Track Submissions
Two Phase Reviewing
AAAI 2021 will use a new, two-phase reviewing process, which will work as follows.
The abstract deadline for AAAI submissions will be September 1, 2020, and the deadline for full paper submissions will be September 9, 2020.
- Every paper will be allocated two reviewers.
- If a paper receives two reviews that are both sufficiently negative, it will be rejected without any opportunity for author feedback.
- Remaining papers will progress to Phase 2.
- Each paper will be allocated two new reviewers.
- These reviewers will not be given access to Phase 1 reviews until after submitting their own reviews.
- A single author feedback phase will give authors of papers from Phase 2 an opportunity to comment on all reviews (from both phases), running from November 3-5.
The program committee and senior program committee will engage in a discussion phase, taking author feedback into account.
Final decisions will be announced on December 1st.
AAAI Timeline Summary
Note: all deadlines are “anywhere on earth” (UTC-12).
- August 15, 2020: AAAI web site open for author registration
- August 22, 2020: AAAI web site open for paper submission
September 1, 2020: Abstracts due by 11:59 PM UTC-12
September 9, 2020: Full papers due by 11:59 PM UTC-12
September 29, 2020: Abstracts AND full papers due for revisions of rejected NeurIPS/EMNLP submissions by 11:59 PM UTC-12
October 13, 2020: Notification of Phase 1 rejections
November 3-5, 2020: Author feedback window
December 1, 2020: Notification of final acceptance or rejection
NeurIPS/EMNLP Fast Track Submissions
The notification dates of NeurIPS and EMNLP do not permit authors of rejected papers to submit by the beginning of AAAI’s Phase 1. In coordination with the NeurIPS and EMNLP program chairs, we have therefore created a process by which such authors can submit revisions of such rejected papers directly into Phase 2. This will work as follows. We will consider that revisions of rejected submissions with an overall score of at least 4.9 (NeurIPS) or 2.8 (EMNLP long papers) have satisfied the conditions of AAAI’s first phase, and hence will accept them directly into AAAI’s second phase. Note that if a paper is rejected from EMNLP but accepted to Findings of EMNLP, its authors must choose between publishing in Findings of EMNLP and submitting to AAAI. Also note that this fast-track submission to Phase 2 is not allowed for EMNLP short papers, and allowed only for EMNLP long papers (and NeurIPS papers).
The deadline for NeurIPS/EMNLP Fast Track submissions is September 29. (Authors anticipating that they may make such a submission are free to submit their abstracts before Phase 1, in which case their paper will be included in Phase 1 bidding; however, authors are free to submit both abstracts and papers until September 29, in which case bidding will occur before the start of Phase 2.)
To be eligible, in addition to supplying all of the information that constitutes a regular AAAI submission, authors must also provide all of the following:
- The paper ID of the NeurIPS/EMNLP submission
- The authors listed on the NeurIPS/EMNLP submission
- The title of the NeurIPS/EMNLP submission
- The original PDF submitted to NeurIPS/EMNLP
- All NeurIPS/EMNLP reviews, including scores and all text
- An author response document describing all changes made since the NeurIPS/EMNLP submission. Authors are encouraged to make a detailed response to each point raised by reviewers, and are free to explain any errors they believe that reviewers made.
In order to submit through this mechanism, authors must explicitly give consent for AAAI to share all submitted information with NeurIPS/EMNLP to verify its accuracy. AAAI reserves the right to summarily reject papers for any reason, including because a submission (a) did not accurately disclose any of the information listed above (either by modifying it or by failing to disclose it, in whole or in part); (b) failed to name any of the authors listed as part of the original submission as authors of the AAAI submission (when appropriate, new author names can be added to AAAI submissions); (c) used a previous NeurIPS/EMNLP submission as a “placeholder” to enable late submission of a materially different AAAI submission; (d) submitted a paper to AAAI that had not (yet) been retracted from Findings of EMNLP.
AAAI reviewers will not have access to any information about a previous NeurIPS/EMNLP submission or previous reviews until after submitting their own reviews; then, they will be given access to all of the above information except for author names, and paper ID. Final decisions will be made based only on the opinions of AAAI reviewers, SPC members and area chairs. However, members of the AAAI program committee will be encouraged to update their own reviews to take into account points made by previous NeurIPS/EMNLP reviewers to the extent they consider appropriate, paying particular attention to points that were not adequately addressed in the author response.