Fig. 7: Demonstration of supernet dynamic pretraining and LLM-driven neural architecture search. | npj Digital Medicine

Fig. 7: Demonstration of supernet dynamic pretraining and LLM-driven neural architecture search.

From: Large language models driven neural architecture search for universal and lightweight disease diagnosis on histopathology slide images

Fig. 7

a Illustration of supernet model for single path one-shot architecture search. During pretraining, numerous subnetworks with independent choice block path are trained via uniform sampling. We search the kernel size of each convolution block in ShuffleNet search for classification and U-Net search for segmentation. We search the depth of transformer layers, number of attention heads, the hidden scale ratio of FFN layer in ViT search for classification. b Illustration of prompt template for searching U-Net architectures via LLM recommendation on pathological tasks. The search prompt template include task formulation, network architecture implementation, search space of different variables and LLM response format.

Back to article page