Table 3 Summary of the Extended \(\left( R, s, Q\right)\) Policy’s Performance by Backtest Horizon. The metrics are derived from backtest runs covering a full year (October 2023 to September 2024) and are presented as uplift relative to the human benchmark decision. Values in brackets denote \(95\%\) confidence intervals. pp is percentage points.
Backtest horizon | GMV Uplift wrt Human | Uplift of GMV After Fulfillment costs wrt Human | % Merchants with Positive GMV Uplifts | Weighted Weekly Availability Rate | Weighted Demand Fill Rate | Weighted Weekly Availability rate uplift wrt Human | Weighted Demand fill rate uplift wrt Human |
|---|---|---|---|---|---|---|---|
4 weeks | \(19.45\%\) \([17.97\%, 21.09\%]\) | \(19.66\%\) \([17.90\%, 21.47\%]\) | \(73.43\%\) \([72.40\%, 74.44\%]\) | 82.85 [82.60, 83.11] | 88.14 [87.97, 88.30] | \(+31.51\%\) \([+30.55, +32.51]\) \(+19.85\text {pp}\) \([+19.38, +20.29]\) | \(+21.31\%\) \([+20.68, +21.93]\) \(+15.48\text {pp}\) \([+15.12, +15.85]\) |
6 weeks | \(22.11\%\) [20.44, 23.92] | \(21.95\%\) \([20.03\%, 24.03\%]\) | \(75.36\%\) [74.34, 76.34] | 86.40 [86.18, 86.63] | 91.14 [91.00, 91.28] | \(+33.63\%\) \([+32.70, +34.59]\) \(+21.75\text {pp}\) \([+21.29, +22.19]\) | \(+23.63\%\) \([+22.99, +24.26]\) \(+17.42\text {pp}\) \([+17.05, +17.78]\) |
8 weeks | \(21.90\%\) \([20.19\%, 23.83\%]\) | \(21.46\%\) \([19.47\%, 23.63\%]\) | \(75.38\%\) \([74.37\%, 76.37\%]\) | 87.62 [87.39, 87.85] | 92.27 [92.13, 92.40] | \(+32.58\%\) \([+31.70, +33.52]\) \(+21.53\text {pp}\) \([+21.07, +21.98]\) | \(+23.54\%\) \([+22.93, +24.16]\) \(+17.58\text {pp}\) \([+17.22, +17.94]\) |
