Table 1 Performance characteristics of the UCSF viral respiratory mNGS assay

From: Laboratory validation of a clinical metagenomic next-generation sequencing assay for respiratory virus detection and discovery

Metrics

Method

Expected Target

Results

Limit of detection (LoD)

Detection of PC dilution by probit analysis

<1000 copies/mL

Target

SARS-CoV-2

Influenza A

Influenza B

RSV

LoD

439 copies/mL

706 copies/mL

493 copies/mL

563 copies/mL

Linearity

Correlation of PC with assay quantification

R2 > 90%

R2 = 100 %

 

Precision

Intra-Assay: PC and NC within the same run across 20 runs.

Concordance

100% EA

Log-transformed CV

<10%

Concordance

100% EA

Log-transformed CV

<10%

Inter-Assay: PC and NC across 20 separate runs

100% EA

<30%

100% EA

<30%

Inclusivity

Detection of viruses from diluted culture supernatant

100% detection

100% detection (17/17)

Detection of viruses in positive BAL/CSF diluted samples

100% detection

100% detection (11/11)

Exclusivity

Detection of viruses in known organism mixturesa

No false-positive

No false-positive

Contamination

Detection of cross-contamination on the sample wells

No carryover contamination

Cross-contamination of 0.1% between adjacent wells but no carryover contamination

Interference

Detection of PC spiked with hemolytic blood

Detection at all concentrations

Detection at all concentrations

Detection of PC spiked with Human RNA

Detection at all concentrations

Detection at all concentrations

Detection of PC spiked with bacterial DNA/RNA

Detection at concentration ≤ 107 cells/mL

Detection at concentration ≤107 cells/mL

Detection of virus-positive overtly mucoid BAL samples

Detection in all BAL samples

Target detected in 13/14 (92.9%) valid sample runs

Stability

Detection of targets in samples held at 4 °C for 7 days or after 3 freeze-thaw cycles

100% concordance

100% concordance

Accuracy

Detection in virus positive and negative samples (n = 191)

Sensitivity > 90%

Specificity > 90%

Accuracy > 90%

PPA > 90%

NPA > 90%

Original testing

Sensitivity: 93.6%

Specificity: 93.8 %

Accuracy: 93.7 %

After discrepancy testing and clinical adjudication

PPA: 98.7%

NPA: 98.1%

Overall: 97.9%

Detection of divergent viruses

Detection by an in silico analysis of divergent viruses (n = 70)

Sensitivity > 95%

Specificity > 95%

Sensitivity: 98.6%

Specificity: 100%

  1. PC positive control consisting of 4 respiratory viruses spiked into pooled nasopharyngeal swab matrix, IC spiked internal control consisting of a RNA MS2 phage, NC negative control, EA essential agreement, CV coefficient of variation, PPA positive percent agreement, NPA negative percent agreement.
  2. aTwo mixtures were assessed. The first mixture included detectable concentrations of CMV, HIV, Klebisella pneumoniae, Streptococcus agalactiae, Aspergillus niger, Cryptococcus neoformans and Toxoplasma gondii, and corresponds to positive control material from a previously validated CSF assay21. The second mixture was a commercial reference panel, the ZymoBIOMICS Microbial Community Standard (Zymo Research, Tustin, CA), and consisted of 10 bacterial and fungal pathogens at varying concentrations (Listeria monocytogenes—12%, Pseudomonas aeruginosa—12%, Bacillus subtilis—12%, Escherichia coli—12%, Salmonella enterica—12%, Lactobacillus fermentum—12%, Enterococcus faecalis—12%, Staphylococcus aureus—12%, Saccharomyces cerevisiae—2%, and Cryptococcus neoformans—2%) that were spiked into negative nasopharyngeal swab matrix.
Back to article page