Saving Time with AI: Best Practices for Using ChatGPT and LLMs to Create Simulation Materials

Participants will first receive a brief introduction to ChatGPT, Large Language Models (LLMs), and other generative AI (genAI) technologies and their potential applications for medical simulation. We will then discuss the current landscape of these tools, discussing the benefits and drawbacks of different approaches. We will then discuss the practicalities of utilizing AI in scenario design by first outlining the importance of AI alignment in order to avoid the risks of misinformation, hallucination, bias, or other negative outcomes. Using this alignment-driven design strategy, we will then outline specific prompt engineering techniques to generate simulation content. We will first explain the basics of a prompt structure and output evaluation methods, then explain specific methods that improve the quality of generated output.

Following this explanatory section, learners will break into small groups of 2-5 and be assigned a specific, real-world task for designing a scenario and be instructed to use an LLM to complete the task, making use of the specific strategies and approaches covered in the session. Course faculty will circulate among the small groups to provide feedback and guidance on addressing the assigned task.

After successfully generating an initial result, the groups will identify iterative techniques to refine their prompt approach. The groups will then work independently to refine their prompt templates, generating their content for the scenario design task. Following this group exercise, the audience will have an open discussion about their experiences, and volunteers will share their results. We will then conclude by discussing the practical implications of deploying generative AI at the institutional level, including regulatory, ethical, and privacy constraints.