Grant Reviewer Perceptions of Panel Discussion in Face-to-Face and Virtual Formats: Lessons from Team Science?

Gallo, S. A.; Schmaling, K.; Thompson, L. A.; Glisson, S.

doi:10.1101/586685

Home/Archive/bioRxiv

bioRxiv · 10.1101/586685

Grant Reviewer Perceptions of Panel Discussion in Face-to-Face and Virtual Formats: Lessons from Team Science?

Gallo, S. A.·Schmaling, K.·Thompson, L. A.·Glisson, S.

View original source ↗Export metadata ↓

Abstract

BackgroundFunding agencies have long used panel discussion in the peer review of research grant proposals as a way to utilize a set of expertise and perspectives in making funding decisions. Little research has examined the quality of panel discussions and how effectively they are facilitated. MethodsHere we present a mixed-method analysis of data from a survey of reviewers focused on their perceptions of the quality and facilitation of panel discussion from their last peer review experience. ResultsReviewers indicated that panel discussions were viewed favorably in terms of participation, clarifying differing opinions, informing unassigned reviewers, and chair facilitation. However, some reviewers mentioned issues with panel discussions, including an uneven focus, limited participation from unassigned reviewers, and short discussion times. Most reviewers felt the discussions affected the review outcome, helped in choosing the best science, and were generally fair and balanced. However, those who felt the discussion did not affect the outcome were also more likely to evaluate panel communication negatively, and several reviewers mentioned potential sources of bias related to the discussion. While respondents strongly acknowledged the importance of the chair in ensuring appropriate facilitation of the discussion to influence scoring and to limit the influence of potential sources of bias from the discussion on scoring, nearly a third of respondents did not find the chair of their most recent panel to have performed these roles effectively. ConclusionsIt is likely that improving chair training in the management of discussion as well as creating review procedures that are informed by the science of leadership and team communication would improve review processes and proposal review reliability.

Explore related subjects

scientific communication and education Preprint

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Gallo, S. A., Schmaling, K., Thompson, L. A., Glisson, S.. 2019-03-22. Grant Reviewer Perceptions of Panel Discussion in Face-to-Face and Virtual Formats: Lessons from Team Science?. https://doi.org/10.1101/586685

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗

bioRxiv2019

Risk factors associated with loss of hepatitis B virus surface antibody in patients with HBV surface-antigen negative/surface antibody positive serostatus receiving biologic DMARDs to treat rheumatic diseases - a nested case-control study

ObjectivesTo elucidate risk factors for loss of hepatitis B virus (HBV) surface antibody (anti-HBs) in patients with rheumatic diseases and HBV surface-antigen negative/anti-HBs positive (HBsAg-/anti-HBs+) serostatus during biologic disease-modifying anti-rheumatic drug (DMARD) treatment.\n\nMethodsThis nested case-control study prospectively enrolled patients with rheumatoid arthritis, ankylosing spondylitis, psoriatic arthritis/psoriasis, and juvenile idiopathic arthritis, who were treated with biologic DMARDs from January 2013 to September 2017. The analytic sample included patients with HBsAg-/anti-HBs+ serostatus. Anti-HBs titers were monitored, and cases defined as anti-HBs <10 mIU/mL during follow-up. Cases were matched one-to-all with controls with anti-HBs [≥]10 mIU/mL on the same event date and equivalent durations of biologic DMARDs treatment. Between-group characteristics were compared and risk factors for anti-HBs loss elucidated by conditional logistic regression analyses.\n\nResultsAmong 189 enrolled patients, 15 cases were matched with 211 controls. Risk factors associated with anti-HBs loss in multivariate analysis were low baseline anti-HBs titer (adjusted risk ratio = 0.96, 95% CI 0.93-0.99) and chronic kidney disease (adjusted risk ratio = 26.25, 95% CI 1.85-372.35). All cases had baseline anti-HBs titer <100 mIU/mL, and none developed HBV reactivation upon losing anti-HBs.\n\nConclusionsIn addition to low baseline anti-HBs titer, chronic kidney disease is also an independent risk factors associated with loss of anti-HBs in patients with HBsAg-/anti-HBs+ serostatus who receive biologic DMARDs to treat rheumatic diseases.\n\nSignificanceGiven that loss of anti-HBs precedes HBV reactivation and that the use of biologic DMARDs is increasingly widespread nowadays, understanding those who are at risk of loss of anti-HBs is an important and practical clinical issue.\n\nInnovationIn addition to low baseline anti-HBs titer, chronic kidney disease is also an independent risk factors associated with loss of anti-HBs in patients with HBsAg-/anti-HBs+ serostatus who receive biologic DMARDs to treat rheumatic diseases.

scientific communication and education↗

bioRxiv2019

The Mastery Rubric for Bioinformatics: supporting design and evaluation of career-spanning education and training

As the life sciences have become more data intensive, the pressure to incorporate the requisite training into life-science education and training programs has increased. To facilitate curriculum development, various sets of (bio)informatics competencies have been articulated; however, these have proved difficult to implement in practice. Addressing this issue, we have created a curriculum-design and -evaluation tool to support the development of specific Knowledge, Skills and Abilities (KSAs) that reflect the scientific method and promote both bioinformatics practice and the achievement of competencies. Twelve KSAs were extracted via formal analysis, and stages along a developmental trajectory, from uninitiated student to independent practitioner, were identified. Demonstration of each KSA by a performer at each stage was initially described (Performance Level Descriptors, PLDs), evaluated, and revised at an international workshop. This work was subsequently extended and further refined to yield the Mastery Rubric for Bioinformatics (MR-Bi). The MR-Bi was validated by demonstrating alignment between the KSAs and competencies, and its consistency with principles of adult learning. The MR-Bi tool provides a formal framework to support curriculum building, training, and self-directed learning. It prioritizes the development of independence and scientific reasoning, and is structured to allow individuals (regardless of career stage, disciplinary background, or skill level) to locate themselves within the framework. The KSAs and their PLDs promote scientific problem formulation and problem solving, lending the MR-Bi durability and flexibility. With its explicit developmental trajectory, the tool can be used by developing or practicing scientists to direct their (and their teams) acquisition of new, or to deepen existing, bioinformatics KSAs. The MR-Bi can thereby contribute to the cultivation of a next generation of bioinformaticians who are able to design reproducible and rigorous research, and to critically analyze results from their own, and others, work.

scientific communication and education↗