Fig. 5: Perplexity-based method to calibrate the confidence of ChatNT answers while preserving performance.
From: A multimodal conversational agent for DNA, RNA and protein tasks
a, A cartoon describing the perplexity-based classifier based on ChatNT answers. b, A calibration plot for the task of human chromatin (chrom.) accessibility (cell line HepG2). The scatter plot compares the predicted probability and fraction of positives over ten bins for the original (green) and calibrated (purple) perplexity-based classifiers. c, A histogram with the predicted probability over ten bins for the original perplexity-based classifiers. d, A histogram with the predicted probability over ten bins for the calibrated perplexity-based classifiers. e, A comparison of the performance (MCC) of the ChatNT answers (yes versus no) and its derived perplexity-based probabilities for all binary classification tasks.
