Fig. 1: Problem setup and architecture of our model.
a A schematic of a canonical scenario that our model might see during inference. We observe the clinical variables of a patient over an observation window of length tcond, at which point we are interested in predicting the survival outomes, potential adverse events, and evolution of relevant biomarkers. We denote the length of the time window over which we forecast the biomarkers, i.e., the forecasting window, as thorizon. The model architecture was evaluated over various combinations of tcond and thorizon. b Diagram of the SCOPE architecture, indicating input variables, transformer layers, the number of which can be tuned, as well as the prediction heads. Additional architectural details can be found in the Methods section. c The training workflow consisted of two steps: pre-training and fine-tuning. We pre-trained the transformer model on the forecasting objective, and then fine-tuned on the event prediction task, keeping the rest of the model parameters frozen.
