Fig. 1: Workflow for adapting generalist LLMs for medical AI through adaptation techniques.

a Generalist AI models, such as proprietary systems (e.g., Open AI's GPT-4) and open-source models (e.g., LLaMA), serve as foundational technologies for developing specialized medical AI models. b Adapting generalist AI to medical tasks involves several techniques, including model finetuning, prompt optimization, and the development of AI agents or AI chains. These methods use diverse medical datasets, such as medical images, electronic health records (EHRs), clinical notes, publications, and omics data, to enhance AI model training and performance. c Effective system engineering for medical AI entails integrating AI modules into comprehensive chains to support tasks like cohort extraction, eligibility assessment, and result verification. This process emphasizes human interaction and AI, resulting in tailored AI modules for specific applications. d Generalist AI applications in medicine span various domains, including conversational diagnosis, radiology report generation, clinical note summarization, automated medical coding, drug design, patient-trial matching, and systematic literature reviews. All require advanced system integration for optimal performance.