Fig. 1
From: Scene flow based deep network for hand reconstruction using depth images
HandFlowNet architecture. Given an estimated hand shape \({H}_t\) and a pair of consecutive point sets (\({PC}_t\), \({PC}_{t+1}\)), the Hand Scene Flow Estimator (SFE) module generates scene flow (\({SF_M}\)) of hand mesh vertices and joints. For the first frame, \({H}_t\) is extracted from \({PC}_t\) through the reference hand shape estimator (HSE) module. For subsequent frames, the hand shape of the previous frame is fed back into SFE for tracking purposes. \({SF_M}\) is added to \({H}_t\) to obtain an initial estimate of hand shape for the next frame. To compensate for error propagation due to scene flow uncertainty in subsequent frames, \({H}_t+{SF_M}\) is refined by the hand refinement module (RM) using the local and global features (\({LF}_{t+1}\) and \({GF}_{t+1}\), respectively) extracted from \({PC}_{t+1}\).
