https://thegradient.pub/how-can-we-improve-peer-review-in-nlp/
> (Anderson, 2009) argues that research paper merit is Zipf-distributed: many papers are clear rejects, while a few are clear accepts. In between those two extremes, decisions are very difficult, and any differences between the best rejected and the worst accepted paper are tiny, even given the best possible set of reviewers.
(And the following is quite discriminating towards non english speaking researchers.)
> Work not-on-English: English is the "default" language to study (Bender, 2019), and work on other languages is easily accused of being "niche" and non-generalizable - even though English only workis equally non-generalizable.
> (Anderson, 2009) argues that research paper merit is Zipf-distributed: many papers are clear rejects, while a few are clear accepts. In between those two extremes, decisions are very difficult, and any differences between the best rejected and the worst accepted paper are tiny, even given the best possible set of reviewers.
(And the following is quite discriminating towards non english speaking researchers.)
> Work not-on-English: English is the "default" language to study (Bender, 2019), and work on other languages is easily accused of being "niche" and non-generalizable - even though English only workis equally non-generalizable.
The Gradient
How Can We Improve Peer Review in NLP?
With so much AI research, and so many poor reviews for this research, how can the community work to improve things?
