Fig. 4
From: Automated OSAHS detection from ECG using temporal convolutional network
The overall architecture of Fast Attention Via Positive Orthogonal Random Feature. Left: standard attention module computation, which computes the final expected result by matrix multiplication using the attention matrix A and value tensor V. Right: by decoupling the matrices Q′and K′used in the low-rank decomposition A, and perform matrix multiplication in the order indicated in the dashed box, we obtain a linear attention matrix without the need to explicitly construct A or its approximation.
