Table 1 Performance on professional examination question banks

From: A personal health large language model for sleep and fitness coaching

Sleep medicine

Approx. CME grade

Expert

Gemini Ultra 1.0

PH-LLM

AMA PRA Category 1/ABIM MOC

70%a

76%

77%

79%

Fitness

Approx. pass grade

Expert

Gemini Ultra 1.0

PH-LLM

NSCA-CSCS coaching certification

70%b

71%

88%

88%

  1. ahttps://www.boardvitals.com/sleep-medicine-moc-recertification.
  2. bhttps://www.nsca.com/certification/cscs/certified-strength-and-conditioning-specialist-exam-description. Accuracy of PH-LLM compared to Gemini Ultra 1.0 and human experts on MCQs from AMA PRA Category 1 - Sleep Medicine and ABIM MOC - Sleep Medicine MOC and NSCA-CSCS coach certification examination question banks. Bold values indicate the highest performance. Approx., approximate.