The ACL deadline is a few days away. My group is not submitting to ACL this year, but judging from experience with other deadlines, this is a week when miraculous transformations happen — barely intelligible drafts fraught with mistakes and confusions become polished submissions, with very little resemblance to the initial version. I suspect this process is not unique to the MIT NLP group, and for most of you, this week is crucial in refining your submissions. To get additional inspiration, we asked a number of prominent ACL researchers to share advice they give to their own students in the last week before submission. We are really grateful to Claire Cardie, Chris Dyer, Hal Daumé, Kathy McKeown (my own advisor), Ray Mooney, and Philip Resnik for their insightful comments.
If I may add my personal advice for the last week: make sure you get enough sleep, healthy food and exercise. This is just another deadline 🙂
Ok, so my real advice for the week before an ACL deadline is TO ALREADY HAVE THE PAPER WRITTEN AT THIS POINT and to use the week to get comments on the paper from as many (qualified) reviewers as possible, making changes to the paper in response. Alas, it seems to be a rare researcher who has completed her paper THAT far in advance :). So, taking a more realistic point of view, I think that there are two key things to work on in the remaining time before the submission deadline. (Of course, there are way more than two, but Regina asked for just one paragraph.) (1) The introduction of the paper has to convince the reader that you are working on a significant problem, that you have strong and/or interesting results, and make clear what the novel contributions of the work are; it has to clearly walk the reader through the “story” of the paper. The rest of the paper has to follow through, but if the intro is bad, you’ve already lost the interest of many reviewers. So make sure that the introduction is impeccable. (2) Think about (and get others to help you think about) what aspect of the work or the writing of the paper itself is (given the paper’s current state) most likely to “kill” the paper in the eyes of the reviewers. Fix this problem as best you can. Iterate this find-and-fix process as many times as is possible before the deadline. There is always something to improve.
When you are putting the finishing touches on your paper, your primary concern should be whether the paper is clear and easy to follow. There are an astounding number of papers that have interesting and important results, but, because they are difficult to follow, they are rejected or don’t have as much impact as they deserve. Reviewers miss the point, and casual readers give up in frustration. While fluency and an elegant turn of phrase are nice, these are really just details (good news for non-native writers!): when it comes to clarity, it’s far more important that your paper cohere globally and that its argument unfold logically. One strategy I find helpful here is to read and edit the paper at different granularities—you might call it “coarse to fine reading”. First, just read the abstract and the introduction. Do you know what the paper’s argument is what its key results are? Next, read the abstract again and the first sentence of every section. Did you make the same argument in both places? Finally, read the first sentence of every paragraph. The outline of the story you are telling should be quite clear, even if the details are missing.
If you can read the paper “coarsely” and still follow its argument, you have a well-organized paper that your reviewers will understand. If you can’t, no amount of beautiful prose is going to fix it.
My biggest advice to authors (to the extent I’m in a position to offer it) is to think about how to make the reviewer’s job easy. It should be obvious from the first two paragraphs what your key contribution is, and a reviewer should be able to easily explain what they’ve learned from your paper. Whatever your central claim is, experiments should be designed to test it. “Winning” on tasks isn’t that interesting: next year, someone else will “win” again. As a reviewer, I need to learn something beyond that: what makes your thing actually work? Finally, pay very very close attention to how you frame the problem in the first page: a good sell helps a lot, though naturally overselling should be avoided because you won’t be able to support it with experiments/theory.
The first advice I give is to try to leave enough time for writing so that it is not all done last minute. There should be time for multiple iterations between the advisor and the student and there should also be time to get input from people who are not at all familiar with your approach. Even your advisor is biased, having heard about the work many times and thus, it is really helpful to get someone to read it who has not heard about your work before (though best if from the natural language field). Make sure to leave enough time so that when you get the feedback, you can take it seriously. Other things I think about: have I motivated the work well? Have I referred accurately to other work that is similar in some way? (Note: accurate is important. As a reviewer, I really am negatively influenced when it’s clear the author hasn’t read the work being cited.) Have I clearly identified my contributions in the first several paragraphs? If you are not a native speaker of the language, get someone who is a native speaker to read it for grammar and word choice. Use a spell-checker to make sure there are no spelling errors. In terms of content, a section that does error analysis or discussion of the results is useful. It is a much more interesting paper if it’s not just about the numbers.
The only non-standard advice I give students which perhaps conveys a somewhat cynical view of the reviewing process, is to check the program committee, determine who might review your paper based on the topic, and make sure to cite any possibly relevant papers they have written, so you avoid the obvious negative reaction if the paper fails to cite the reviewer’s own work which they believe is relevant.
Make sure that you do not fall prey to the “curse of knowledge” — that is, the implicit assumption that your reader already shares your background and assumptions. This means, to start with, that you should let the reader know clearly why they should care about this work. It’s not enough to improve performance on some task by X%; it’s just as important to be clear about why the task should matter in the first place. This also means that you should focus your paper on the logical story, not the technical details: the narrative of the paper should be organized logically, not chronologically; the important points about models and algorithms should be explicated in plain English in the body of the text, rather than requiring detailed parsing of equations or algorithm figures; and derivations or implementation details belong in documentation (or, if they’re really necessary for the conference reader, in an appendix).
In terms of experimentation, be explicit about having separated training and test data, and explain why you chose the parameters you chose. (You used 50 topics for LDA? Why not 20 or 100?) And if the answer is *not* that you either decided a priori or that you tuned on held-out data, but rather that you’re just reporting whatever gave you the best results on your test data, you are reporting a tainted experiment. You can un-taint it somewhat by reporting the results for all the other values you tried also — but then you should be prepared for a reviewer to ask why we should believe your values will be the best value on the next, previously unseen dataset. If all else fails, appeal to previous literature and use parameter values that can be described as “typical in prior work”, and provide citations to back that up.
Finally, the quality of the English writing matters a lot. It doesn’t need to be fancy or poetic, but it does need to be clean, grammatical, and spelled correctly, and the logical flow needs to be clear, or your reviewer is going to have to work too hard to get your point. If you’re not a native speaker of English (and maybe even if you are), make sure to get a native speaker to read the paper and correct language problems.
Edit: if you’d like to offer you own advice, please do so in the comments! We will try to compile these into the main post as a resource for both veterans and newcomers to the field alike. Thanks!