Fig. 7: Schematic of an attention block in the CrabNet architecture.
From: Compositionally restricted attention-based network for materials property predictions
a The initial projection of the input EDM into the Q, K and V tensors. b The scaled dot-product attention operation obtaining the self-attention matrix and the updated Z element representation. The batch dimension is not shown in b to improve legibility.
