Fig. 3: Empirical risk vs. R-distributions for the four candidate LP models.
From: Selecting fitted models under epistemic uncertainty using a stochastic process on quantile functions
(top) The empirical risk(5) for each of the four candidate LP models. (bottom) Our proposed BEMD criterion replaces the risk by an R-distribution, where the spread of each distribution is due to the replication uncertainty for that particular model. R-distributions are distributions of the R functional in equation (20); we estimate them by sampling the quantile function (i.e., inverse cumulative density function) q according to a stochastic process \({\mathfrak{Q}}\) on quantile functions. We used an EMD sensitivity factor of c = 2−2 (see later section on calibration) for \({\mathfrak{Q}}\) and drew samples \(\hat{q} \sim {\mathfrak{Q}}\) until the relative standard error on the risk was below 3 %. A kernel density estimate (KDE) is used to display those samples as distributions. The R-distribution for the true model is much narrower (approximately Dirac) and far to the left, outside the plotting bounds.
