Fig. 5: Improvement of stagnation and recognition of chirality by the introduction of pre-LN. | Nature Communications

Fig. 5: Improvement of stagnation and recognition of chirality by the introduction of pre-LN.

From: Difficulty in chirality recognition for Transformer architectures learning chemical structures from string representations

Fig. 5

a, b Temporal change of perfect accuracy started from 14 different initial weights with post/pre-layer normalization(post-LN/pre-LN) structure. Lines with the same color corresponds to training from the same initial weight. a is reproduced from Fig. 3a for comparison. c, d Temporal change of perfect accuracy when each one of the characters in Simplified Molecular Input Line Entry System (SMILES) was masked for trainings with post-LN/pre-LN structure. c is reproduced from Fig. 4b for comparison. Source data are provided as a Source Data file.

Back to article page