Transfer Learning and Semi-Supervised Learning in High-Dimensional Data

High-dimensional datasets, typified by genomics, proteomics and high-resolution imaging, pose acute challenges due to the vast number of features relative to available labels. Transfer learning addresses this gap by leveraging knowledge from one domain or task—often with abundant annotated data—to enhance performance on a related but label-scarce target domain. It achieves this by pretraining models on a rich source dataset and fine-tuning them on the target environment, or by aligning feature spaces across domains through domain adaptation. Semi-supervised learning complements this approach by exploiting large pools of unlabelled data alongside a limited set of labelled examples. Techniques range from graph-based propagation and manifold regularisation to self-training and pseudo-labelling, each designed to infer structure in the joint distribution of features and labels. When combined, transfer and semi-supervised strategies can substantially mitigate the curse of dimensionality, bolster generalisation and reduce dependency on costly annotations. Applications span medical diagnosis, where pretrained imaging models adapt to new modalities, to environmental monitoring, where satellite imagery annotations are rare but unlabelled archives abound. Recent advances emphasise scalable architectures, robust debiasing against domain shifts and principled frameworks for integrating unlabelled observations without propagating error.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

A cross-prediction framework has been introduced for semi-supervised inference in high-dimensional settings. By imputing missing labels with machine-learning models trained on a small labelled subset and then applying a debiasing correction, this method delivers confidence intervals with controlled error rates and enhanced statistical power compared with fully supervised approaches.

A federated transport learning algorithm has been developed for survival risk modelling across heterogeneous healthcare centres. This approach fits penalised proportional hazards models locally, shares only summary statistics and optimises a transfer learning objective to produce robust coefficient estimates for a target population. It demonstrates improved accuracy and privacy preservation in high-dimensional electronic health records.

An in-depth analysis of the Lasso in partially labelled, high-dimensional regression has yielded non-asymptotic risk bounds. By extending oracle inequalities to semi-supervised contexts, the work clarifies how unlabeled features can offset poor restricted eigenvalues of the design matrix, guiding adaptations of sparsity-inducing estimators when labels are scarce.

Transfer Learning and Semi-Supervised Learning in High-Dimensional Data publication trend

The graph below shows the total number of articles in transfer learning and semi-supervised learning in high-dimensional data across all publications each year (not limited to Nature Index journals).

Technical terms

Transfer learning: A strategy for reusing parameters or representations learned on one domain to improve performance on another related domain with limited labels.

Semi-supervised learning: An approach that combines a small labelled dataset with a larger unlabelled dataset to enhance predictive modelling.

High-dimensional data: Datasets in which the number of features greatly exceeds the number of labelled instances, leading to sparse learning challenges.

Domain adaptation: A subclass of transfer learning that aligns feature or label distributions between source and target domains to mitigate distributional shifts.

Manifold regularisation: A semi-supervised technique that imposes smoothness of the predictive function along the estimated low-dimensional structure (manifold) of the data.

References

1. Cross-prediction-powered inference. Proceedings of the National Academy of Sciences of the United States of America (2024).
2. SurvMaximin: Robust federated approach to transporting survival risk prediction models. Journal of Biomedical Informatics (2022).
3. On the prediction loss of the lasso in the partially labeled setting. Electronic Journal of Statistics (2018).