Improving the Performance of LLM-Based Semi-Automated Psychiatric Case Diagnosis using Decision Tree-Based Prompting

The knowledge and reasoning capacity of LLMs in psychiatry is an active topic of interest. In this work, we demonstrate a decision tree-based approach to prompting LLMs to provide semi-automated case diagnoses for standardized psychiatric scenarios, drawing from previously developed structured diagnostic pathways within psychiatry. We found that this approach improved the recall and significantly improved the precision of LLM-predicted diagnoses from case vignettes, suggesting that LLMs have promise in detecting and reasoning about psychiatric symptoms and diagnoses.