Fig. 1: Overview of the approach for computing R-distributions.
From: Selecting fitted models under epistemic uncertainty using a stochastic process on quantile functions
We assume the model has already been fitted to obtain a set of candidate parameter sets θA, θB… (not shown: the models may also be structurally different). Each candidate parameter set ΘA defines a candidate model \({{{\mathcal{M}}}}_{A}\), for which we describe the statistics of the loss with two different quantile functions: the purely synthetic \({\tilde{q}}_{A}\) (equation (22)) which depends only on the model, and the mixed \({q}_{A}^{*}\) (equation (19)) which depends on both the model and the data. A small discrepancy \({\delta }_{A}^{{{\rm{EMD}}}}\) (equation (23)) between those two curves indicates that model predictions concord with the observed data. Both \({q}_{A}^{*}\) and \({\delta }_{A}^{{{\rm{EMD}}}}\) are then used to parametrise a stochastic process \({\mathfrak{Q}}\) which generates random quantile functions. This induces a distribution for the risk RA of model \({{{\mathcal{M}}}}_{A}\), which we ascribe to epistemic uncertainty. The stochastic process \({\mathfrak{Q}}\) also depends on a global scaling parameter c. This is independent of the specific model, and is obtained by calibrating the procedure with simulated experiments Ωi that reflect variations in laboratory conditions. The computation steps on the right (white background) have been packaged as available software74.
