Introduction

The incidence of pre-diabetes, metabolic syndrome, and type 2 diabetes are increasing world-wide and pose a critical need for improved therapeutics. A promising approach is activating brown adipose tissue (BAT), which increases the rate of systemic metabolism by oxidizing metabolic fuels like glucose and fatty acids and secreting signaling molecules. We and others have discovered that BAT can produce omega (ω)-6 and ω-3 polyunsaturated fatty acids (PUFAs)-derived bioactive lipids (i.e. lipokines) such as 12,13-dihydroxy-octadecaenoic acid (12,13-diHOME), which increases BAT activity1,2, skeletal muscle fatty acid uptake3, cardiac function4, and endothelial function to reduce atherosclerosis5. Furthermore, 12,13-diHOME in mother’s milk is associated with lower infant fat mass6 and an open-label clinical trial has found that sustained BAT activation over 4 weeks improved systemic metabolism7.

We had previously identified 12,13-diHOME due to its inverse association with BMI (body mass index) in a cohort of 55 individuals (Study 1) with a broad range of BMI8 and then showed that 12,13-diHOME mechanistically induces translocation of fatty acid transport proteins to the surface of cells8. Other groups also observed a relationship between human obesity and 12,13-diHOME; however, the signaling pathway that drives this effect is unknown. To confirm this association in another study using analogous methods and gain the power to identify additional signaling lipids that benefit systemic metabolism, we recruited a new cohort.

In this work, we profile signaling lipids in a cohort of 83 individuals (Study 2) that was partitioned based on BMI into obese (BMI > 40 kg/m2) and non-obese (BMI < 30 kg/m2). We first test if we can replicate the significant, negative rank correlation between plasma concentration of 12,13-diHOME at room temperature and BMI in Study 2. Secondly, we examine if other lipids’ association with BMI replicates in both studies8. Replication of two studies tests a mathematically equivalent null hypothesis as mediation analysis, because both settings evaluate two independent hypotheses (mediation tests if each lipid or analyte is both associated with the exposure, and separately if it is associated with the outcome given the exposure). Replication and mediation can be tested by calculating each analyte’s p-value as the analyte’s maximum p-value over the two studies (i.e. the joint significance or MaxP method’s p-value) and then adjusting these maximum p-values for multiple testing of many analytes9. The MaxP p-values were found via simulations to control the error rate and have more power than many competing approaches10,11.

However, such application of the MaxP method has two limitations: it does not account for direction, whereas it is natural to require an effect to have the same direction in both studies to be considered replicated, and there are improved methods of adjusting p-values in this context. We previously accounted for directionality in High-throughput mediation analysis (Hitman), which proposed a Directional MaxP Test (DMT) and use of empirical Bayesian linear regression modeling to improve power12, although here we apply non-parametric methods to mirror our previous analysis of 12,13-diHOME8. The directional MaxP test assesses mediation in a direction consistent with the exposure’s effect on the outcome and its p-value is half the (non-directional) MaxP p value if the direction is consistent, whereas otherwise its p-value is one. DMT was mathematically proven to control the error rate and was found to have stronger p-values than the MaxP and other methods12.

We improved MaxP’s adjusted p-values by developing ScreenMin13, which reduces the multiple testing burden by filtering out analytes whose minimum p-value is so weak that their MaxP p-value could never be significant. This idea was further developed in AdaFilter14, which also proposes a method to account for direction. There are other methods that improve upon MaxP adjusted p values and can account for direction, such as radjust-sym15, where analytes with strong p-values from each study are selected and only analytes selected from both studies are tested for replicability, and the empirical Bayesian method RepFdr16.

Results

Development and validation of ScreenDMT

To identify which lipid’s association with BMI is replicated, we sought a test that accounted for direction and had the strongest adjusted p-values in terms of the false discovery rate (FDR). Because we cared about directionality it was natural to consider DMT. By comparison, it was recently shown that the MaxP test is the likelihood ratio test (LRT) for the non-directional mediation null hypothesis17, which is mathematically equivalent to testing replication without regard for direction in two studies. The LRT has many beneficial properties such as often being optimal, which is why LRTs so often used in practice, e.g. the t-test for means. For directional mediation and replication, we mathematically prove that DMT is the LRT in Supplementary Note 1. To our knowledge, analyses of replication and mediation had not been previously connected to analysis of qualitative interactions, which occurs when an effect changes sign between groups of subjects, e.g. a treatment benefits one group of patients while it harms another. Surprisingly, we find that directional replication and mediation are connected to qualitative interaction, because applying the DMT in search of effects that have opposite signs between groups is the LRT for qualitative interactions18,19.

We considered how to improve the adjusted p-values from our directional MaxP test with screening or filtering. The filtering approach in AdaFilter when applied to two studies uses the minimum p-value (pmin) per analyte as the filtering p-value and the maximum p-value as the selection p-value. We cannot directly use AdaFilter by naively setting the DMT p-value (pDMT) as the selection p-value, since the selection p-value must exceed the filtering p-value, which would not always be true. Instead, we developed a way to apply screening to our directional MaxP test, which we term “ScreenDMT”. ScreenDMT defines its selection p-value per analyte as the maximum of pDMT and pmin. We mathematically prove ScreenDMT to be valid for controlling the FDR and the family-wise error rate (FWER; the rate controlled by the Bonferonni method) in Supplementary Note 1 for large sample sizes. For small sample sizes, the screening-based multiple testing adjustment is very slightly too strong, but this is where the DMT p-value, like similar statistical tests, is weak anyway. Like in AdaFilter14, theoretical control of the FWER requires independence of p-values within each study, while control of the FDR for a large number of analytes allows weak dependence between analytes. Such weak dependence typically holds, for example, between gene loci and expression14. Importantly, not all FDR methods theoretically allow dependence, such as RepFdr16.

We compare ScreenDMT’s FDR and power via simulation against competing directional replication methods, including the directional approaches of AdaFilter14, radjust-sym15, RepFdr16, and DMT (Fig. 1). We also compare against a directional alternative to the MaxP test that was adapted from a meta-analysis due to Pearson14,20, which applies MaxP for each analyte to right-sided p-values of both studies, then applies MaxP to left-sided p-values, and doubles the minimum from the two applications. This procedure also underlies AdaFilter’s directional approach.

Fig. 1: Comparison of ScreenDMT’s power and false discovery rate to competing approaches in replication of two datasets via simulation.
figure 1

Simulation of two datasets with strong or weak dependence between analytes when the signal strength or effect size per analyte is equal between the two datasets and when it is unequal. The x-axis represents the percent of analytes that do not truly directionally replicate but show some effect in one of the studies or show effect in both studies of opposite direction. a Probability of detecting true associations (power). b False discovery rate. The FDR threshold is 5%.

Full size image

We see in Fig. 1 that RepFdr often does not control its FDR below the nominal level, and apart from RepFdr, ScreenDMT retains the most power across the scenarios. We also see that DMT has more power than the approach adapted from Pearson14,20, which helps explain ScreenDMT’s power advantage over AdaFilter. We perform the analogous simulation for FWER using methods that account for directionality and provide FWER in Supplementary Fig. 1. We see that all methods properly control FWER, DMT is slightly more powerful than the method adapted from Pearson14,20, and ScreenDMT’s power is very similar but slightly stronger than AdaFilter’s.

Inverse association of 12,13-diHOME with BMI replicates

We wanted to test if 12,13-diHOME negatively associates with BMI in the Study 2 cohort of 83 individuals (Table 1). Our previous association of 12,13-diHOME to BMI in Study 1 was assessed using Spearman rank correlation, so we also use a nonparametric test in Study 2 (Figs. 2a, b). Since Study 2 is split into obese and non-obese subjects, we evaluate whether lipids are differentially abundant between groups with a non-parametric t-test (i.e. Wilcoxon rank sum or Mann-Whitney U test) (Figs. 2c, d). We apply this test in a one-sided manner to test if 12,13-diHOME is lower in obese subjects. Since 12,13-diHOME had already been prioritized, its p value does not need to be adjusted for multiple testing. We obtain a one-sided p-value of 0.00093 against a significance p value threshold of 0.05, which demonstrates replication (Fig. 2d).

Table 1 Anthropometrics of Study 2 cohort
Full size table
Fig. 2: 9,10-diHOME and 12,13-diHOME versus BMI in both cohorts.
figure 2

Plasma levels of 9,10-diHOME and 12,13-diHOME versus BMI in participants from the original cohort (Study 1) and from the new cohort (Study 2). To improve visualization, 9,10-diHOME values above its 99th percentile were winsorized to its 99th percentile. The y-axis represents log2-transformed area ratios, which are in arbitrary units (A.U.). Spearman’s rank correlation coefficient or Wilcoxon rank sum’s z-score are provided. a 12,13-diHOME in Study 1. b 12,13-diHOME in Study 2. c 9,10-diHOME in Study 1. d 9,10-diHOME in Study 2.

Full size image

Application of ScreenDMT to test replication of other lipids’ association with BMI

In addition to 12,13-diHOME, we measured a panel of more than a hundred oxidized lipid species, including metabolites of “parent” lipids: linoleic acid (LA), α-linolenic acid, arachidonic acid, dihomo-γ-linolenic acid, Docosahexaenoic acid and Eicosapentaenoic acid (Fig. 3a). To assess if any lipids other than 12,13-diHOME have an association with BMI that replicates, we applied the non-parametric t-test to all the lipids except 12,13-diHOME and then assessed replication with ScreenDMT (Supplementary Data 1). The only other lipid that replicated was 9,10-dihydroxy-12Z-octadecenoic acid (9,10-diHOME), which is a regioisomer of 12,13-diHOME (Fig. 3b), whose FDR is 5%. Whereas if we had tested replication with DMT, the FDR of 9,10-diHOME would be nearly double at 9%. We show the plots of 9,10-diHOME vs. BMI and of 12,13-diHOME vs. BMI in both cohorts in Fig. 2.

Fig. 3: Biosynthesis of linoleic acid diols 9,10-diHOME.
figure 3

a Lipids measured in our lipidomic panel are shown as a product of their precursor fatty acids and the oxidative pathways that are the first step in their biosynthesis. b Biosynthesis of linoleic acid diols.

Full size image

We next assessed lipid sets for enrichment of lipids whose nominal DMT replication p-value was below 5%. We defined one lipid set per parent lipid from the left-most column of Fig. 3a, so we created 6 lipid sets. For example, the LA set contained the eight lipids in the first row of Fig. 3a. We tested if each set was over-represented among the nominally significant lipids using a one-sided Fisher exact test and adjusted the six p-values for multiple hypothesis testing using the Benjamini-Hochberg FDR (Supplementary Data 1). We found that the LA set was the only significant set (FDR < 10−5) and that all of the LA-derived lipids had an inverse (but not necessarily significant) relationship with BMI in both studies.

9,10-diHOME and 12,13-diHOME activate adipocyte calcium fluxes

12,13-diHOME and its stereoisomer 9,10-diHOME are replicably associated with BMI, and we had previously shown that 12,13-diHOME increases cellular fatty acid uptake by inducing the translocation of fatty acid transporters FATP1 and CD36 to the cell surface; however, the signaling pathway that mediates this effect is unknown. 12,13-diHOME and 9,10-diHOME have both been found to activate ion flux in primary neurons and CHO cells21, so we measured calcium flux in adipocytes treated with each diol. We used in vitro differentiated WT1 mouse brown preadipocytes and in vitro differentiated 3T3-L1 mouse white preadipocytes to model brown and white adipocytes, respectively. Both cell lines were differentiated using a standard adipogenic induction media, and then loaded with Fura-2. After we recorded baseline calcium flux, cells were treated with Hanks Buffered Saline Solution (HBSS) containing 1% fatty acid-free bovine serum albumin (BSA) alone or containing different concentrations of linoleic acid diol, and calcium flux was monitored every 50 seconds for approximately 9 minutes in a plate reader (Fig. 4). Both 9,10-diHOME and 12,13-diHOME triggered calcium influx in brown and white adipocytes, suggesting the signaling pathway activated downstream of these two lipids overlaps.

Fig. 4: Linoleic acid diols activate calcium flux in cultured adipocytes.
figure 4

X-axis is timepoint after baseline (baseline is timepoint zero). Each timepoint is approximately 50 seconds apart. Immediately after baseline, vehicle control or different concentrations of linoleic acid diol were injected. Y-axis is FLUOFORTE fluorescence quantification in arbitrary units (A.U.). To normalize, values were divided by their mean baseline value. Each data point is from an independent well; n = 3 independent wells per group. a Differentiated WT1 brown adipocytes treated with 12,13-diHOME or vehicle control. b Differentiated WT1 brown adipocytes treated with 9,10-diHOME or vehicle control. c Differentiated 3T3-L1 white adipocytes treated with 12,13-diHOME or vehicle control. d Differentiated 3T3-L1 white adipocytes treated with 9,10-diHOME or vehicle control. For each panel, normalized post-baseline fluorescence was regressed on continuous timepoint and categorical dose; each dose was compared to vehicle control with a two-sided t-test; and p-values from all panels were Bonferroni corrected. ***Bonferroni-corrected P < 0.001 vs. vehicle control. Statistical results are in Supplementary Data 1.

Full size image

Discussion

We test replication which has been defined as obtaining the same result in new data22. Although often used synonymously, reproducibility can be defined as obtaining the same result on the same data with the same analysis22, whereas meta-analysis combines statistics from datasets, which can yield significance due to a strong effect in a single study. Replication is fundamental to science and its mathematical treatment has been found to have numerous applications9. We focus on replication in a concordant direction and show that our test for directional replication is mathematically equivalent to testing for qualitative interactions and directional mediation.

An example of a qualitative interaction would be a pair of genes that are positively associated in healthy participants and negatively associated in those with disease. Finding such pairs is important for identifying the gene interactions causing disease and other questions in differential network biology23. Another example would be a biomarker being positively associated with an outcome in one population, such as those of European ancestry, while being negatively associated in an under-represented population. Although a limitation of our work is that Study 1 and Study 2 are comprised of people of European ancestry, our method can powerfully identify whether LA diols inverse associations to BMI (or any other findings in Europeans) are consistent or of the opposite direction across populations, promoting equity in medicine24.

We demonstrated the value of mediation accounting for direction in a previous study where Hitman, which is based on DMT, identified growth hormone receptor as a mediator of gastric bypass surgery’s reduction of glycemia12. We found that surgery reduced GHR levels, and lower GHR was associated with lower glycemia. Mediation methods that do not account for direction would have called the metabolite retinol as a mediator, even though surgery reduced retinol levels, and lower retinol was associated with glycemia being increased. Thus, surgery’s apparent action via retinol would be for glycemia to increase. This type of inconsistent mediation is unlikely and does not help explain the mechanism of action, so our directional MaxP test saves power by assigning such an analyte a p-value of one. ScreenDMT improves upon the directional MaxP test with more efficient adjusted p-values14. These improved adjusted p-values correspond to selection p-values that are based on, but not necessarily the same, as the directional MaxP test p-values. Thus, at times, ScreenDMT’s adjusted p-values may not have the same order as the DMT p-values. As usual when testing many hypotheses, significance should be based on adjusted p-values.

A possible reason that both 12,13-diHOME and 9,10-diHOME levels are associated with BMI is their shared biosynthetic pathways downstream of LA. LA is a polyunsaturated omega-6 fatty acid that is one of two essential fatty acids for humans and can be used as an energy source25. However, it can also be metabolized into a diverse group of signaling molecules, including the LA diols26,27. LA diols are synthesized in a two-step process that begins with oxidation of LA at either the double bond at the 9,10 position or the 12,13 position, resulting in one of two regioisomer epoxides. This step is catalyzed by Cytochrome P450 oxidases, however little is known regarding the identity of the specific enzyme(s) that catalyze this step or if and how the position of the epoxide moiety is determined. Biosynthesis of 12,13-diHOME from its parent epoxide is mediated by an epoxide hydrolase, most likely soluble epoxide hydrolase encoded by the gene Ephx2, as mice lacking Ephx2 have circulating levels of 12,13-diHOME 8-fold lower than wildtype or Ephx1 knockout controls28. Similarly, 9,10-diHOME is generated from an epoxide precursor called 9,10-epOME, however only genetic ablation of both Ephx2 and Ephx1, which encodes the microsomal isozyme of epoxide hydrolase, is sufficient to lower the levels of circulating 9,10-diHOME28.

In rodents and humans, dietary LA can raise circulating concentrations of LA diols, increasing 12,13-diHOME levels as quickly as 1 hour after intravenous infusion of a lipid emulsion mainly composed of LA29,30,31. Thus our findings on LA diols and the LA-derived lipid set could be explained by LA intake. If our findings were entirely explained by LA intake, then we would expect LA intake to be inversely associated with BMI. Although some studies have found that LA supplementation decreases body weight, most clinical studies have found no effect32,33,34. Moreover, a cross-sectional analysis of American adults’ BMI and dietary intake from the National Health and Nutrition Examination Survey (NHANES) found that those with a BMI between 30 to 40 consumed significantly more LA than those with a normal weight (BMI 18.5 to 25), so our findings are likely due to regulation of LA metabolism. Future work could investigate to what degree concentrations of LA diols in humans are explained by LA intake or the regulation of LA metabolism (e.g. epoxide hydrolase activity).

In mice and cells, 12,13-diHOME can activate changes in metabolism by recruiting fatty acid transport proteins to the cell surface, however the signaling pathway or pathways that mediate this effect are unknown. A candidate transport protein is CD36, and translocation of CD36 can be activated by calcium flux35. Since both 12,13-diHOME and 9,10-diHOME can activate changes in membrane polarization21 and we have recently shown that 12,13-diHOME can activate calcium flux in bovine endothelial cells5, we tested whether both lipids in their physiological range from 1 to 100 nM8,36,37 have a similar effect on adipocyte calcium flux and found that they do.

The beneficial effect of 9,10-diHOME and 12,13-diHOME are historically surprising, because they were originally tested in the micromolar range where they were found to cause mitochondrial damage in leukocytes. Recently, 12,13-diHOME in the micromolar range was found to reduce T regulatory cell abundance and secretion of IL-1038. Whereas 9,10-diHOME at physiological concentrations between 1-100nM was found to activate neutrophil chemotaxis in vitro39, so pro-inflammatory effects even at nanomolar doses should be kept in mind when considering these compounds as therapeutic targets.

Overall, given our findings there is an evidence base to test the beneficial effects of 9,10-diHOME on systemic metabolism, and given the overlap between signaling that we observed with 12,13-diHOME, the signaling pathway targeted by these two molecules could prove to be a valuable therapeutic target in the treatment of obesity and its sequalae. Our experimental validation also lends credence to our statistical approach. To summarize its benefits over competing approaches in the setting of replication, DMT yields stronger p-values by focusing on effects that are in the same direction across studies and ScreenDMT yields stronger adjusted p-values, providing more sensitivity to detect real effects.

Methods

Study 2 inclusion and ethics

We included 83 individuals with (n = 41) or without (n = 42) obesity defined by a BMI > 30 kg/m2 (Table 1). Study participants were consecutively recruited to the Leipzig Obesity BioBank (LOBB). To minimize the effect of diet, all blood samples were collected between 8 and 10 am after an overnight fast. The study was approved by the local Ethics Committee of the University of Leipzig, Germany (Reg. numbers: 363–10‐13122010 and 017–12‐230112) and all subjects gave written witnessed informed consent before inclusion into the LOBB repository. All ethical regulations relevant to human research participants were followed.

Study 2 lipidomics

Plasma was analyzed by liquid chromatography-tandem mass spectrometry (LC-MS/MS) to obtain relative measurements on a panel of 113 signaling lipids. Plasma samples were thawed at room temperature and immediately placed on ice. Aliquots of 100 µL were taken and added to 300 µL of methanol (stored at −20 °C) for a protein crash. Ten µL of a mixture of 5 deuterated internal standards (Cayman Chemical, Ann Arbor, MI, USA), each at 100 pg/µL, were spiked into the samples, then samples were vortexed for 10 seconds and stored at −20 °C overnight. Samples were then subjected to a solid phase extraction. C18 cartridges at 500 mg/6 mL (Biotage, Uppsala, Sweden) were conditioned with 10 mL of methanol followed by 10 mL of water. The samples were centrifuged at 14000 g, and the pH of the supernatant of the samples was adjusted by adding 3 mL of pH 3.5 water before loading the samples onto the C18 cartridges. The cartridges were washed with 5 mL water followed by 5 mL hexanes. Fractions were dried down under a stream of N2 gas, and reconstituted in 50 µL methanol:water (1:1, by vol). Samples were vortexed and then transferred to LC-MS vials for analysis.

Electrospray ionization (ESI) LC-MS/MS was performed on a QTRAP 6500 (Sciex, Framingham, MA, USA) coupled to an Agilent Infinity 1290 (Agilent, Santa Clara, CA, USA) LC system with a InfinityLab Poroshell 120 EC-C18 (4.6 ×100 mm, 2.7 µm; Agilent) analytical LC column with a column oven heated to 60°C. Ten µL of sample was injected at a flow rate of 400 µL/min and components were separated by reversed-phase chromatography with mobile phases A (100% H2O, 0.1% acetic acid) and B (100% MeOH, 0.1% acetic acid). The gradient started at 5% B and increased to 20% B by 3 minutes, then increased to 60% B by 8 minutes, increased to 90% B by 23 minutes, held at 90% B for 3 minutes, then returned to 5% B for the last 3 minutes. The total LC run time was 29 minutes. Samples were only acquired in negative polarity due to the chemical structure of the targeted lipids. The ESI source parameters were ion source gas 1 (GS1) 30, ion source gas 2 (GS2) 30, curtain gas (CUR) 30, ion spray voltage (IS) −4500 V, and temperature 500 °C. The declustering potential (DP), entrance potential (EP), collision energy (CE), and exit potential (CXP) were tuned for each individual targeted lipid and internal standard. The MS method was a targeted scheduled MRM method. There were 101 MRM scans scheduled for optimized windows ranging from 120–360 seconds. The mass spectrometer acquisition time was set to 29 minutes. The LC-MS/MS data was acquired by Analyst 1.6.2 software (Sciex, Framingham, MA, USA) and processed with MultiQuant 3.0.1 (Sciex, Framingham, MA, USA) for peak integration. The targeted lipid species were reported as area ratios of the peak area of the analyte divided by the peak area of the corresponding internal standard (Supplementary Data 1).

Bioinformatics analysis of lipidomics data

We reanalyzed our previous lipidomics data from Study 18 and Study 2’s lipidomics data using the R software. Both data sets included values of zero for missing data. Our only processing step was to filter out lipids with 20% or more missing values, which was applied to both datasets. To reproduce our previous study, we tested Spearman rank correlation with R function cor.test, which yielded the same coefficient and p-value as previously reported8. For the Study 2 lipidomics dataset, we tested differential abundance between groups using the R function wilcox.test. Wilcoxon test z-scores were calculated from the wilcoxonZ function in the R package rcompanion. Replication FDRs from ScreenDMT were considered significant using FDR < 15%, as we used previously12.

Cell culture

WT-1 cells were generated in the Tseng Laboratory and 3T3-L1 cells were purchased from ATCC. All cells were monitored monthly for Mycoplasma infection. WT-1 mouse preadipocytes and 3T3-L1 mouse fibroblasts were cultured in high glucose DMEM supplemented with 10% FBS. When cells reached confluence, adipocyte differentiation was induced by using induction media (DMEM high glucose media with 10% FBS, 33 µM Biotin, 17 µM　Pantothenate, 0.5 µM human insulin, 500 µM IBMX, 2 nM T3, 0.1 µM dexamethasone and 30 µM indomethacin) for 2 days, then cells were switched to a differentiation media (DMEM high glucose media with 10% FBS, 0.5 µM human insulin, 2 nM T3) for a final 7 days. After fully differentiated, adipocytes were washed with PBS and starved for 1 hour in DMEM. Adipocyte authentication was confirmed morphometrically.

Calcium flux analysis

FLUOFORTE fluorescence was measured in adipocytes according to manufacturer’s directions (Enzo Scientific, Farmingdale, NY, USA). Briefly, starved adipocytes were incubated with FLUOFORTE Dye-loading solution for 1 hour at room temperature, then baseline fluorescence was read on a plate reader. After treatment with vehicle control or different concentrations of linoleic acid diol, FLUOFORTE fluorescence was read every 50 seconds.

Replication simulations

We assessed the performance of methods that provide adjusted p-values by simulating two studies with the same 1,000 features or analytes following the simulations in Djordjilović et al.40. We simulated normally distributed statistics with variance of one per analyte per study. The statistics for analytes without signal were simulated with mean zero in both studies. In the Equal Signal simulation, the statistics for analytes that were truly directionally replicated were simulated with a mean of 3.5 or −3.5 in both studies, and these comprised 5% of the analytes. The statistics for analytes that were not directionally replicated but had some signal (“Percent of null analytes with signal”) were simulated with mean zero in one study and mean magnitude of 3.5 in the other, or else they were simulated with a mean of 3.5 in one study and −3.5 in the other study. In the Unequal Signal setting, the mean magnitude for one study was 2.66 while the other study had mean magnitude of 4.66. Like the AdaFilter simulations14, the statistics were simulated under weak dependence, where analytes were correlated with correlation coefficient of 0.5 to other features in their block with block size of 10, whereas under strong dependence, features were similarly correlated but in block sizes of 100. Features were randomly assorted to blocks.

For each setting 1000 simulations were run. RepFdr using its default parameters was too slow to run this many simulations, so the tolerance specified in RepFdr’s package vignette was used. So that RepFdr calculates FDRs when it estimates the fraction of nulls to be one in both studies, we modified it to calculate each analyte’s FDR as one instead of producing an error. For the FWER case, we do not compare to Bogomolov and Heller15 because its replication implementation radjust-sym does not include a FWER procedure and its mediation implementation does not account for direction41.

Statistics and reproducibility

The replicates in the human lipidomics data sets were distinct individuals. Replicates in the calcium flux experiments were the same cell line in different wells that were independently treated with vehicle or linoleic acid diol. Statistics were calculated using the R statistical software v4.4.0 and data was plotted using the R package ggplot2.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.