Table 2 Model updating procedures described in this paper. The performance guarantees from these methods require the stream of data to be IID with respect to the target population. Note that in general, online learning methods may provide only weak performance guarantees or none at all.
Method(s) | Update frequency | Complexity of model update | Performance guarantees |
|---|---|---|---|
One-time model recalibration (e.g. Platt scaling, isotonic regression, temperature scaling) | Low | Low | Strong |
One-time model revision | Low | Medium | Strong |
One-time model refitting | Low | High | Strong |
Online hypothesis testing for approving proposed modifications | Medium | High | Strong |
Online parametric model recalibration/revision | High | Low/Medium | Medium |
