We selected ~10% of NeurIPS papers to be reviewed twice, independently.
170 papers were reviewed by two separate committees.
Each committee was 1/2 the size of the full committee.
Reviewers allocated at random
Area Chairs allocated to ensure distribution of expertise
Notes on the Timeline for NeurIPS
AC recruitment (3 waves): * 17/02/2014 * 08/03/2014 * 09/04/2014
We requested names of reviewers from ACs in two waves: * 25/03/2014 * 11/04/2014
Reviewer recruitment (4 waves): * 14/04/2014 * 28/04/2014 * 09/05/2014 * 10/06/2014 (note this is after deadline … lots of area chairs asked for reviewers after the deadline!). We invited them en-masse.
06/06/2014 Submission Deadline
12/06/2014 Bidding Open For Area Chairs (this was delayed by CMT issues)
17/06/2014 Bidding Open For Reviewers
01/07/2014 Start Reviewing
21/07/2014 Reviewing deadline
04/08/2014 Reviews to Authors
11/08/2014 Author Rebuttal Due
25/08/2014 Teleconferences Begin
30/08/2014 Teleconferences End
1/09/2014 Preliminary Decisions Made
9/09/2014 Decisions Sent to Authors
Decision Making Timeline
Deadline 6th June
three weeks for paper bidding and allocation
three weeks for review
two weeks for discussion and adding/augmenting reviews/reviewers