Referees often disagree as to whether to accept or reject a manuscript for publication. Recent investigations have often focused on the study of standard rating forms that ask for appraisal of specified attributes of a manuscript. Scott asked reviewers to rate, on seven criteria, manuscripts submitted to the Journal of Personality and Social Psychology. There was some agreement on most of the criteria; correlations were often significant but never large. Importance of the problem and adequacy of design and analysis seemed to affect evaluation most strongly. Whether the use of such a rating form will lead to greater reliability of referees' recommendations is an open question. There are many reasons why referees disagree, of course, and research in this area should be broadly based. The decision-making process of the editor is also very complex and should be the focus of empirical investigation.