Fig. 1: Motivating example: side effects for naive application of traditional unlearning method.
From: Nexus scissor: enhance open-access language model safety by connection pruning
After unlearning the entire harmful response, the model loses the entity concept and related knowledge required to perform general task.
