Writing Papers in the Age of AI Reviewers

NeurIPS received 6,743 submissions in 2019 [1]. Five years later, its 2024 main and datasets-and-benchmarks tracks received 17,491 submissions [2]. In 2025, the conference reported 21,575 submissions [3]. The number more than tripled in six years.

That curve is not merely a story about a successful conference. It is a stress test for the entire publication system. If it continues, I would not be surprised to see a major AI conference approach 40,000 submissions during the NeurIPS 2026 cycle. This is a forecast, not a reported number, but the direction is hard to miss.

The Submission Explosion

There are familiar reasons for this growth. AI has become one of the most popular areas of science and engineering. More undergraduates enter research early. More PhD students need publications to graduate or compete for jobs. More companies and investors fund large research teams. An expanding community should produce more papers.

But the production function of a paper has changed too. Since the arrival of capable Large Language Models, researchers can ask GPT to polish prose, brainstorm variants, debug code, summarize related work, and reformat a submission. The emerging idea of AutoResearch pushes this further. None of these tools guarantees a good scientific contribution, but they reduce the cost of turning an idea into something that looks like a conference paper.

When the cost of packaging a result falls, more results get packaged. A small gain can acquire a broad motivation, an elaborate method name, and a polished experimental section. The bottleneck moves downstream: from producing submissions to distinguishing important work from competent-looking work.

Peer Review Is Changing Too

The review system was not designed for this volume. NeurIPS 2025 listed 21,921 technical reviewers and 1,985 area chairs [3]. Recruiting a reviewer pool roughly as large as the submission count is an extraordinary organizational effort, yet scale alone does not create attention, expertise, or time.

Reviewers now use the same class of LLM tools as authors. A large-scale study of reviews from ICLR 2024, NeurIPS 2023, CoRL 2023, and EMNLP 2023 estimated that 6.5% to 16.9% of review text may have been substantially modified by an LLM beyond spell-checking or minor edits [4]. The study does not prove that every such review was poor, nor can it reliably label an individual review. It does establish that AI-assisted reviewing is not hypothetical.

In my experience, the more troubling change is stylistic convergence. Reviews increasingly read as if they were generated from the same checklist. They are fluent, comprehensive in appearance, and often detached from the central scientific idea. Even a meta-review can become little more than a smooth summary of existing scores, with no visible attempt to resolve the substantive disagreement underneath them.

Any one of these requests can be reasonable. Reproducibility matters. Strong baselines matter. Variance matters. The problem is the indiscriminate accumulation of individually reasonable requests. If authors satisfy every possible checklist item, the paper becomes longer, more expensive, and less clear - without necessarily becoming more insightful.

I once asked GPT to review a paper that had received an ICLR Best Paper Award, and it recommended rejection. This was only an anecdotal experiment, and the point is not that a model can never identify a strong paper. It is that checklist compliance and scientific value are different objectives.

From Defensive to Offensive Writing

When I was a PhD student, much of the writing advice I received was defensive. Do not claim more than the evidence supports. Anticipate the reviewer's questions. Disclose limitations. Close obvious loopholes. That advice remains ethically and scientifically correct.

But writing only defensively no longer works when the space of possible objections can be generated without limit. Follow every branch and you will run another baseline, another seed, another ablation, and another analysis. Fifty pages may still not be enough. Worse, the main contribution can disappear beneath the defensive perimeter built around it.

By offensive, I do not mean aggressive language, hidden limitations, or inflated claims. I mean organizing the paper around its genuine advantage. Decide what the work demonstrates better than existing alternatives, and make every major section help the reader understand that result. If an experiment is not central to the claim, do not promote its outcome into a headline conclusion. If the method is not designed to optimize a particular axis, do not let that axis become the spine of the paper.

This requires discipline. State the contribution precisely. Choose experiments that test it directly. Discuss meaningful limitations in context, including why they arise and when they matter. Separate limitations from self-indictment. A limitation section should define the boundary of a claim; it should not provide a list of sentences that can be copied verbatim into a weakness box.

Most importantly, write for a thoughtful human reader even if an AI system will screen, summarize, or criticize the paper. A human can recognize a coherent research vision: why this problem matters, why this particular choice is revealing, and what becomes possible because of the result. A checklist cannot substitute for that narrative.

Who Decides the Value of a Paper?

A few reviewers and an area chair decide whether a paper is accepted to a conference. They do not decide its final value. That judgment is made slowly by the community: by researchers who build on the work, practitioners who use it, teachers who explain it, and later papers that discover whether its central idea survives.

This distinction matters more as the review process becomes noisier. A conference label is still useful, but it cannot carry the full weight of research evaluation. The quality of a researcher's representative work, sustained community impact, reusable artifacts, and longer-term trajectory are increasingly better signals than paper counts or the assumption that every acceptance is equivalent.

That is not an argument to ignore peer review. It is an argument to remember what peer review can and cannot do.

The Review System Must Change

Authors can adapt their writing, but individual adaptation cannot solve a structural problem. Submission caps may curb extreme behavior, yet they do not change the underlying economics: submitting another paper is cheap for the author, while reviewing it consumes someone else's time.

I think major conferences should seriously consider submission fees, with waivers based on need and geography. If a paper is accepted, part of the fee could be returned or credited toward registration. The remaining pool could pay reviewers for timely, substantive work. Payment would not automatically produce good judgment, and it could create new incentives to manage. Still, the current system already assigns an enormous economic value to reviews while pricing the reviewer's labor at zero.

What Should Remain Human

AI can make papers clearer. It can help reviewers find missing controls, inconsistent claims, or overlooked literature. Used carefully, it can reduce mechanical work on both sides. The goal should not be to preserve every old habit simply because it predates language models.

What must remain human is the allocation of attention and responsibility. Someone must decide which question is worth asking, which evidence changes our understanding, and which criticism is important enough to affect the verdict. Someone must stand behind that judgment.

The age of AI reviewers therefore calls for two changes at once. Authors should write more deliberately, constructing papers around their real strengths instead of chasing an infinite checklist. Conferences should redesign incentives so that careful human judgment remains possible at scale. If we do only the first, papers will become better optimized for a deteriorating system. If we do only the second, the submission flood will continue to overwhelm it.

References

1. NeurIPS 2019 Press and Conference Information, Neural Information Processing Systems Foundation.
2. NeurIPS 2024 Fact Sheet, Neural Information Processing Systems Foundation.
3. NeurIPS 2025 Fact Sheet, Neural Information Processing Systems Foundation.
4. Weixin Liang et al., "Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews", ICML 2024.