Table 5 PEGASUS model performance after layer manipulation.
From: Brain-model neural similarity reveals abstractive summarization performance
Ablated layer | Rouge1 | Rouge2 | RougeL | Perplexity | Noisy layer | Rouge1 | Rouge2 | RougeL | Perplexity |
|---|---|---|---|---|---|---|---|---|---|
w/o | 0.4835 | 0.2485 | 0.3978 | 4.4455 | w/o | 0.4835 | 0.2485 | 0.3978 | 4.4455 |
1 | 0.4839 | 0.2482 | 0.3981 | 3.8162 | 1 | 0.4843 | 0.2487 | 0.3982 | 5.8081 |
2 | 0.4837 | 0.2482 | 0.3975 | 4.6588 | 2 | 0.4836 | 0.2484 | 0.3979 | 4.5298 |
3 | 0.4816 | 0.2463 | 0.3943 | 4.4042 | 3 | 0.4833 | 0.2479 | 0.3973 | 4.4933 |
4 | 0.4818 | 0.2462 | 0.3955 | 4.6315 | 4 | 0.4838 | 0.2485 | 0.3976 | 4.5074 |
5 | 0.4827 | 0.2478 | 0.3967 | 4.4057 | 5 | 0.4844 | 0.2488 | 0.3980 | 4.4795 |
6 | 0.4813 | 0.2466 | 0.3957 | 4.5811 | 6 | 0.4834 | 0.2488 | 0.3974 | 4.5506 |
7 | 0.4828 | 0.2481 | 0.3974 | 4.8670 | 7 | 0.4837 | 0.2490 | 0.3984 | 4.5029 |
8 | 0.4816 | 0.2472 | 0.3959 | 4.7064 | 8 | 0.4824 | 0.2477 | 0.3966 | 4.5299 |
9 | 0.4813 | 0.2465 | 0.3957 | 4.2938 | 9 | 0.4817 | 0.2468 | 0.3955 | 4.5912 |
10 | 0.4819 | 0.2469 | 0.3961 | 4.7257 | 10 | 0.4813 | 0.2466 | 0.3956 | 4.9480 |
11 | 0.4790 | 0.2440 | 0.3926 | 4.6511 | 11 | 0.4803 | 0.2447 | 0.3942 | 4.6066 |
12 | 0.4811 | 0.2459 | 0.3950 | 4.7428 | 12 | 0.4812 | 0.2460 | 0.3950 | 4.5174 |
13 | 0.4801 | 0.2448 | 0.3938 | 4.7486 | 13 | 0.4793 | 0.2449 | 0.3934 | 4.4523 |
14 | 0.4794 | 0.2442 | 0.3926 | 4.5026 | 14 | 0.4797 | 0.2448 | 0.3938 | 4.4388 |
15 | 0.4809 | 0.2461 | 0.3940 | 4.5126 | 15 | 0.4788 | 0.2433 | 0.3916 | 4.4305 |
16 | 0.4801 | 0.2452 | 0.3935 | 4.6827 | 16 | 0.4774 | 0.2419 | 0.3910 | 4.5789 |
All | 0.1790 | 0.0275 | 0.1351 | 4.2754 | all | 0.4364 | 0.2011 | 0.3459 | 6.5883 |
