Table 2 Properties and benefits of different fusion strategies.

	Early	Joint	Late
Able to make predictions when not all modalities are present	×	×^a	✓
Able to model interactions between features from different modalities	✓	✓	×
Able to learn more compatible features from each modality	×	✓	×
Does not necessarily require a large amount of training data	×	×	✓
Does not require training multiple models	✓^b	✓	×
Does not necessarily require meticulous designing efforts	✓	×	✓
Flexibility to join input at different levels of abstraction	×	✓	×

Different properties and benefits for each fusion strategy.
^aSpecialized joint fusion architecture such as Kawahara et al.’s multi-modal multi-task model is capable of handling missing data.
^bEarly fusion requires training of multiple models when the imaging features are extracted using CNN.

Quick links

Search