Fig. 2: Results of independent evaluation of bias in Med-PaLM 2 answers.
From: A toolbox for surfacing health equity harms and biases in large language models
We report the rate at which raters reported minor or severe bias in Med-PaLM 2 answers for physician and health equity expert raters for each dataset and dimension of bias. The numbers of answers rated for each dataset are reported in Table 2 and the Methods. Statistics for multiply rated datasets (Mixed MMQA–OMAQ and Omiye et al.) were computed with pooling over replicates with the level of replication indicated in parentheses. Data are reported as proportions with 95% CIs.
