Table 2 Attention scoring function model.
Functional model | Formula |
|---|---|
Additive model | \(s\left( {h,q} \right) = v^{T} \tanh \left( {Wh + Uq} \right)\) |
Dot product model | \(s\left( {h,q} \right) = hq\) |
Scaling dot product model | \(s\left( {h,q} \right) = \frac{{h^{t} q}}{\sqrt d }\) |
Bilinear model | \(s\left( {h,q} \right) = h^{T} Wq\) |
