Intelligent rehabilitation system is an active research topic. It is motivated by the increased number of limb disabled patients. Human motion tracking is the key technology of intelligent rehabilitation system, because the movement of limb disabled patients needs to be localized and learned so that any undesired motion behavior can be corrected in order to reach an expectation. This paper introduces a real-time tracking system of human hand motion, specifically intent to be used for home rehabilitation. Vision sensor (camera) is employed in this system to track the hand movement, and the improved Camshift algorithm and Kalman filter are used to implement dynamic hand tracking in the video. CAMSHIFT algorithm is able to track any kind of target colors by building a histogram distribution of the H channel in HSV color space from the region of interests selected by users at the initial stage. Kalman filter is able to predict hand location in one image frame based on its location data detected in the previous frame. The experimental results show that this system can track 2D hand motion and has acceptable accuracy by using the two algorithms properly. The new algorithm proposed in this paper can not only deal with the skin color interference problems, but also deal well with the track of complex background.