Table 1 Evaluation of the PR classification and reconstruction tasks for human VH sequences

From: Assessing antibody and nanobody nativeness for hit selection and humanization with AbNatiV

VH

Classification (PR-AUC)

Reconstruction accuracy

Rhesus versus

Mouse versus

PSSM-generated versus

T

D

T

D

T

D

T

D

AbNatiV

0.965

0.923

0.996

0.988

1.000

0.998

0.960

0.935

OASis (relaxed)

0.570

0.829

0.897

0.965

0.982

0.992

N/A

N/A

Sapiens

0.626

0.883

0.982

0.994

0.993

0.997

0.918

0.949

AbLSTM

0.721

0.892

0.963

0.986

0.998

0.998

0.807

0.856

AbLSTM retrained

0.777

0.866

0.967

0.979

0.997

0.996

0.822

0.849

  1. The assessment is carried out for AbNatiV trained on human VH sequences (first row) and other computational approaches that can assess humanness (other rows). AbLSTM retrained corresponds to the AbLSTM model retrained on the same training set of AbNatiV (Methods). The first six columns report the area under the PR curve (shown in Fig. 2 and Supplementary Fig. 8), assessing the ability of the models to separate sequences in the human test (T) or the human diverse >5% (D) sets from those from mouse, rhesus and PSSM-generated (column headers). The human diverse >5% dataset is used here as a control to specifically assess the ability of the AbNatiV to generalize to sequences distant from those in its training set. The last two columns quantify the ability of each model to reconstruct human sequences in each dataset (column header). The OASis method does not carry out reconstruction (N/A, not applicable). Many sequences of the D datasets belong to the Sapiens training set. Corresponding ROC results are in Supplementary Table 1.