Mislabeled metadata are especially problematic in large-scale collaborations where data analysis is far removed from data generation: researchers cannot reconfirm sample labels or resequence questionable samples, and must rely on the accuracy of available metadata. However, we have found that supervised classifiers are able to detect and even correct erroneous metadata in some datasets. We have demonstrated the efficacy of this approach by applying two common classifiers to Monte-Carlo simulations of mislabeled metadata from published 16S rRNA microbial community datasets examining (1) variation of bacterial communities among human body habitats (Costello et al., 2009), and (2) the relationship of bacteria on computer keyboards to the users’ hands (Fierer et al., 2010).

When the ‘alleged’ data labels (that is, the metadata supplied by the experimenter) contain errors, we refer to this as ‘metadata error’. Normally when we build a supervised classifier we can only compare its predictions to the alleged labels, and so the classifier's ‘reported error’ may be inaccurate. Here we work through an example in which all metadata error is simulated, and so we can compare a classifier's predicted labels to the true labels to measure the classifier's ‘true error’. When all or most of the samples are mislabeled, we expect the classifier to be useless, but what if only a few of the labels are wrong?

Log in or create a free account to read this content

Gain free access to this article, as well as selected content from this journal and more on nature.com

Access through your institution

or

Sign in or create an account
Continue with Google
Continue with ORCiD