Fig. 4: FDR and power comparisons using synthetic data from Poisson-Gamma distributions.

The False Discovery Rate (FDR) and power of various differential abundance (DA) analyses (two-sided) are shown in a and b, respectively. The variability in sampling fractions is set to be large. The Y-axis denotes patterns of proportion of differentially abundant taxa. The solid vertical line is the 5% nominal level of FDR, and the dashed vertical line denotes 5% nominal level plus one standard error (SE). By default, ANCOM-BC implements Bonferroni correction and other DA methods implement BH procedure to adjust for multiple comparisons. Color and the name of the corresponding DA method are shown at the bottom within the graph. Two simulation scenarios are considered: small and unbalanced data (n₁ = 20, n₂ = 30), as well as large and balanced data (n₁ = n₂ = 50); number of simulations = 100. Results show that only ANCOM and ANCOM-BC control the FDR under the nominal level (5%) while maintaining power comparable with other methods. Gaussian model version of metagenomeSeq has highly inflated FDR, while the log-Gaussian version has substantial loss of power, sometimes well below 5%. Other than ANCOM-BC and ANCOM, as the sample size within each group increases, so does the FDR for all other existing methods.