bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2022.12.20.521261

A Pilot Survey of Authors' Experiences with Poor Peer Reviews

McCloskey, K. B.·Merz, J. F.
View original source ↗Export metadata ↓

Abstract

ObjectivesTo develop a typology of poor peer review practices (PPRP) and assess researchers experiences with PPRP. DesignExploratory analysis of cross-sectional internet-based survey. ParticipantsWe solicited 500 researchers funded by the NIH extramural grants in 2018 by direct email and 600 bioethicists on a bioethics discussion forum (mcw.bioethics). 112 respondents (~10%) completed the survey. Primary and Secondary Outcomes MeasuresThe total number of reported PPRP and a five-point scale to assess participants views about the effect of PPRP on their ability to disseminate their research. ResultsThe mean number of PPRP experienced per author was 12.5 of 28 (44.6%; range 0-27; 95% CI = 11.2-13.8), with fourteen PPRP experienced by 50% or more of the sample. The number of reported PPRP increased with age (P = 0.01) and total number of published peer-reviewed manuscripts (P = 0.02). Authors belonging to underrepresented groups reported more PPRP compared to represented groups (P = 0.05). Most authors viewed the peer review process favorably, with 67% (74/111) of authors responding "sometimes" or "often" to having received insightful peer reviews that improved the quality of their final papers. However, a total of 57% (63/111) of respondents admitted to previously abandoning a manuscript after receiving what they perceived to be unfair peer reviews. ConclusionsThis study introduces a practical list of PPRP and a framework for a typology of PPRP, which could serve as an educational tool for editors and reviewers and further our understanding of poor peer review practices. Future researchers will expand authors experiences with constructive or helpful peer review practices. STRENGTHS AND LIMITATIONS OF THIS STUDYO_LIThe sample consisted primarily of experienced researchers from diverse fields, which aided in capturing a wide variety of poor peer review examples. C_LIO_LIThis survey included a core set of 28 poor peer review practices and allowed respondents to add other practices they had experienced, which helped to generate an extensive list of poor peer review practices. C_LIO_LIThe generalizability of the prevalence of poor review types and the degree of negative impact on authors should be interpreted with caution due to the low response rate and the potential for response bias. C_LI

Explore related subjects

scientific communication and educationPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

McCloskey, K. B., Merz, J. F.. 2022-12-21. A Pilot Survey of Authors' Experiences with Poor Peer Reviews. https://doi.org/10.1101/2022.12.20.521261

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗
bioRxiv2026

A strong start for sustained success: inclusivity through a national group mentorship program for first-year graduate students

In the United States, STEM graduate programs and workforce do not represent the demographics of the population. Obstacles, including a lack of transparency, community, and accessible information in navigating academia, disproportionately affect students from underserved backgrounds. Peer mentoring networks can address these disparities. Here, we describe Cientifico Latino, Inc.s Graduate Student Engagement and Community (CL-GSEC) program, a nationwide, group-based peer mentorship program that has served first-year graduate students across the U.S., especially those from underserved backgrounds. Surveys indicate CL-GSEC positively impacts the first-year graduate experience. We highlight key program features, challenges, and insights, such as financial strains faced by first-year graduate students. We offer suggestions for how faculty and departments can better support students during this critical early stage of graduate training. We hope that reporting on CL-GSECs program structure, evaluations, and findings will guide educational leaders in expanding programming for junior graduate students.

scientific communication and education↗
bioRxiv2026

Biodesign Buddy: Integrating Generative Artificial Intelligence in Academic Biodesign

Biodesign is an interdisciplinary research domain that incorporates principles from design and the life sciences to develop new systems, processes, and objects. Collegiate biodesign educators face unique pedagogical challenges, including an absence of relevant scholarship on curriculum design and instructional best practices for cultivating student scientific literacy. These difficulties may be overcome with newly available technologies, like generative AI systems, that enable personalized learning through domain-specific semantic spaces. This article examines the instructional value of one such domain-specific LLM, Biodesign Buddy, through a mixed-methods analysis of an eight-week study involving 64 students participating in an international biodesign competition. Results indicate strong support for integrating AI into biodesign coursework. Surveys captured attitudes toward AI, scientific literature, and learning experiences to assess AIs impact on learning outcomes. Findings suggest that integrating AI into biodesign pedagogy can meaningfully redress conceptual issues in biodesign while informing broader debates on AIs role in higher education. Impact StatementThis article introduces Biodesign Buddy, a domain-specific generative AI system for collegiate biodesign education, and reports on its exploratory deployment, offering design principles and preliminary findings to inform the development of AI-supported pedagogies for interdisciplinary biodesign instruction.

scientific communication and education↗