Hand tracking is the automatic detection and displaying of the user hands captured by a camera throughout a video. This paper proposes a practical solution to two main hand tracking problems: the mutual occlusion between hands and the recovery from a lost tracking. This is done by alternating the use of a priori information about past hand locations according to the presence of a problematic situation. We propose a deterministic solution capable of working at different video resolutions without the need of any early learning stage or hard-coded initial calibration. Thus, it is practical in the sense of its implementation and use. Our experimental results, which are compared to the state of the art, shows that our solution is computationally efficient and has a good performance in different lighting and motion situations, both in terms of speed and precision error. © 2014 IEEE.