bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2022.09.26.509545

Evaluating user experience with immersive technology in simulation-based education: a modified Delphi study with qualitative analysis

Jacobs, C.·Foote, G.·Williams, M.
View original source ↗Export metadata ↓

Abstract

BackgroundImmersive technology is becoming more widespread in simulation-based medical education with applications that both supplement and replace traditional teaching methods. There is a lack of validated measures that capture user experience to inform of the technology utility. We aimed to establish a consensus of items and domains that different simulation experts would include in a measure for immersive technology use. MethodsA 3-stage modified Delphi using online software was conducted to support the conceptual framework for the proposed measure. The first round was informed by prior work on immersive technology in simulation. In the first round, participants were asked to describe what we could measure in simulation-based education and technology. Thematic analysis generated key themes that were presented to the participants in the second round. Ranking of importance in round 2 was determined by mean rank scores. The final round was an online meeting for final consensus discussion and most important domains by experts were considered. ResultsA total of 16 simulation experts participated in the study. A consensus was reached on the ideal measure in immersive technology simulation that would be a user questionnaire and domains of interest would be: what was learnt, the degree of immersion experienced, fidelity provided, debrief, psychological safety and patient safety. No consensus was reached with the barriers that this technology introduces in education. ConclusionsThere is varied opinion on what we should prioritise in measuring the experience in simulation practice. Importantly, this study identified key areas that aids our understanding on how we can measure new technology in educational settings. Synthesising these results in to a multidomain instrument requires a systematic approach to testing in future research

Source connections

Explore related subjects

scientific communication and educationPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jacobs, C., Foote, G., Williams, M.. 2022-09-27. Evaluating user experience with immersive technology in simulation-based education: a modified Delphi study with qualitative analysis. https://doi.org/10.1101/2022.09.26.509545

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗
bioRxiv2022

Preparing future STEM faculty nationwide through flexible teaching professional development

We report on a five-year initiative that has prepared thousands of future STEM faculty around the world to adopt evidence-based instructional practices by participating in two massive open online courses (MOOCs) and facilitated in-person learning communities. This novel combination of asynchronous online and coordinated, structured face-to-face learning community experiences provides flexible options for STEM graduate students and postdoctoral fellows to pursue teaching professional development, while leveraging the affordances of educational technologies and the geographically clustered nature of this target learner demographic. A total of 14,977 participants enrolled in seven offerings of the introductory course held 2014-2018, with 1,725 participants from approximately 60 countries completing at an average course completion rate of 13%. The preparation of future STEM faculty makes an important difference in establishing high-quality instruction that meets the diverse needs of all undergraduate students, and the initiative described here can serve as a model for increasing access to such preparation.

scientific communication and education↗
bioRxiv2022

Sexual health knowledge acquisition processes among very young adolescent girls in rural Malawi: Implications for sexual and reproductive health programs

Early adolescence is a period characterized by enormous biological, cognitive, sexual, emotional, and social changes. Sexual curiosity and the desire to acquire sexual health (SH) information are part of these developments. Understanding the SH knowledge acquisition process is critical for designing interventions that can best support very young adolescents (VYAs). This study explored the SH knowledge acquisition processes among VYA girls aged 10 to 14 years who attended the DREAMs Girl Only Clubs (GOCs) and those who did not. The GOCs were a part of a larger comprehensive HIV prevention project called DREAMS (Determined, Resilient, Empowered, AIDS-free, Mentored, and Safe) which provided an evidence-based core package of interventions to VYAs to prevent HIV. In-depth interviews were conducted with 43 VYA girls aged 10-14 years in two rural southern districts of Zomba and Machinga in Malawi. Twenty-three VYA girls were GOC participants and 20 VYAs were not. Guided by the Social Ecological Model, a thematic analysis approach was used to analyze the data with the assistance of Nvivo 12 software. The SH knowledge acquisition processes were the interaction of various factors at the microsystem (biological factors, attitudes), mesosystem (parents, older relatives, friends), and exosystem levels (schools and GOCs). Factors influencing this interaction included the beginning of menstruation, supportive parents/adults on SH issues, trust and confidentiality in SH information sources, and preference for well-informed sources of information. Compared to Non-GOC participants, GOC participants demonstrated an in-depth knowledge of SH issues, and positive sexual behaviors such as HIV testing. Limitations to SH knowledge acquisition were adult messages focus on sexual relationship avoidance and on girls who have started menstruation; perception of adults not being knowledgeable about SH and school teachers hiding some SH information. VYA SH interventions that provide VYA with accurate SH information such as the GOCs have the potential to address the limitations that other sources have in reaching VYAs. Integrating such interventions with empowering parents, other adults, and teachers with comprehensive SH information and with skills on how to deliver SH information to VYAs can enhance their SH knowledge acquisition and influence positive behavior change.

scientific communication and education↗