Weakly Supervised Semantic Segmentation Techniques

Weakly supervised semantic segmentation (WSSS) seeks to assign a class label to every pixel in an image while relying on annotations that are coarser, cheaper or easier to obtain than full pixel-level masks. By exploiting image-level tags, sparse scribbles, bounding boxes or individual points, WSSS methods aim to recover fine-grained object boundaries with minimal manual effort. Core strategies include using class activation maps to generate initial localisation cues, employing saliency detection to guide region proposals, adopting graph-based affinity learning to propagate labels, and iteratively refining pseudo-masks through self-training or expectation–maximisation frameworks. Advances in network architecture, such as encoder–decoder designs with attention modules, have been combined with consistency regularisation and adversarial training to suppress noise in weak labels. Emerging approaches also integrate generative modelling to synthesise plausible masks and leverage multi-modal cues—for example, combining depth or motion information with weak annotations. WSSS has demonstrated impact across domains including autonomous driving, medical imaging and remote sensing, enabling applications from tumour delineation with image-level reports to land-cover mapping using limited point annotations. The field continues to balance the trade-off between annotation cost and segmentation accuracy, with current research focused on closing the gap to fully supervised performance while ensuring scalability and robustness to annotation noise.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Weakly Supervised Semantic Segmentation Techniques publication trend

The graph below shows the total number of articles in weakly supervised semantic segmentation techniques across all publications each year (not limited to Nature Index journals).

Technical terms

Semantic segmentation: Partitioning an image into regions corresponding to predefined object classes.

Weak supervision: Learning from imprecise, incomplete or inexact labels instead of full pixel-level annotations.

Class activation map (CAM): Heatmap indicating regions most relevant to a classifier’s decision, used to localise objects under weak labels.

U-Net: Encoder–decoder convolutional network architecture with skip-connections designed for image segmentation.

Pseudo-mask: Estimated pixel-level annotation generated from weak labels, used to train segmentation networks.

Scribble annotation: Sparse hand-drawn strokes marking a subset of pixels for each class, serving as weak labels.

References

1. Semantic Segmentation of Remote Sensing Images With Sparse Annotations. IEEE Geoscience and Remote Sensing Letters (2021).
2. WSF-NET: Weakly Supervised Feature-Fusion Network for Binary Segmentation in Remote Sensing Image. Remote Sensing (2018).