Fig. 2: GPU memory consumed by each delta-tuning method and fine-tuning.
From: Parameter-efficient fine-tuning of large-scale pre-trained language models
We choose three T5 models with different scales to assess the GPU memory. All evaluations are conducted on NVIDIA A100 GPUs.
