Table 4 Comparative performance analysis with different embeddings in Model #3 (Appendix A).

From: AI and narrative embeddings detect PTSD following childbirth via birth stories

Model

AUC

F1 score

Sensitivity (Recall)

Specificity

Model #3

0.80

0.82

0.81

0.72

Model #3 with text-embeddings of All-mpnet-base-v247

0.75

0.76

0.80

0.70

Model #3 with text-embeddings of Mental-roberta-base6

0.67

0.71

0.80

0.55

Model #3 with text-embeddings of Mental-xlnet-base-cased7

0.65

0.70

0.80

0.50

Model #3 with text-embeddings of Bio+ClinicalBERT24

0.65

0.63

0.60

0.70

Model #3 with text-embeddings of BioGPT25

0.62

0.59

0.55

0.70

Model #3 with text-embeddings of Mental-bert-base-uncased6

0.63

0.58

0.60

0.70

  1. The average performance results of the ten-fold cross-validation process conducted on the same analyzed dataset that was used in47 are presented. Note: The dataset used here is a subset of the dataset used in Table 3. OpenAI’s text-embedding-ada-002 embeddings in Model #3 (first row in Table 4) outperform all other embeddings in Model #3, demonstrating superior ability in identifying CB-PTSD using narrative data only. Results are ordered by descending F1 score value.
Back to article page