It is that time of year. The ACM SIGCHI reviews are out, and the lucky recipients have one week and 5000 characters to use rebutting them, trying desperately to get their beautiful papers into CHI 2014 and earn themselves a trip to Toronto. In today’s lab meeting we spent a little time unpicking some of the reviews and how they might be responded to. This post is not about the detail of the papers submitted though (and all lab group names will be left out, just in case!!) – this is more about the underlying process, and how to best go about constructing a sensible, non-sweary rebuttal. (It may take a day or two to get to the point where this is possible.)
The key part of a rebuttal is understanding the reviews. Each paper gets three reviews, and a meta-review that is pulled together by the Associate Chair (“AC”). The meta-review is pretty key. This gives you some kind of insight into whether the AC is “on your side” and is leaning towards including your paper or not. The key is the rating they give you. If they have given you a rating that is lower than the average of the three main reviews, you’re going to have to work pretty hard to convince them to include it.
So, then you need to work out what the key points are that you need to address. The focus should be on clarifications of your position, not major reworking of your arguments. As Hyungyoung Song points out, agreeing to a major rewrite of your key arguments suggest that you agree that there are underlying problems and massive rewriting usually suggests that you may be better off submitting to a later conference. Instead you need to focus on areas where the reviewers have misunderstood, so that you can clarify and help. Although it may not feel like it right now, this may, in the long run, even help your paper.
Part of helping to identify the key points that will need clarifying may well be to understand the reviewer’s form for this conference. There are certain key items that a reviewer is asked to look for, and it may well help to actively sign-post those items. For example, the key contribution of the paper is a good thing to highlight. Two of our lab group are working as ACs this year, so they were able to cast some light on this for people in the group who haven’t done this yet.
The format of the rebuttal was also interesting. The advice from one lab member was to be very specific, and say where you will add sentences to clarify, but also where you will remove something to keep within the page limit. The people who make the decisions are trying to decide whether your rebuttals are strong enough to answer your critics, but also whether they are feasible. This kind of structure makes it much easier for them to see that it is possible, and given the workload on the ACs anything that makes things easier for them is likely to count in your favour.
The best advice by far was not to give up at this point. 5000 characters is worth it. Don’t even think about what else you could do with your paper until after the final rejections have come in. There’s still hope!
Good luck to all who are going through this at the moment…