This paper focuses on a state sharing method for an artificial neural network (ANN) and hidden Markov model (HMM) hybrid on line handwriting recognition system. A modeling precision based distance measure is proposed to describe similarity between two ANNs, which are used as HMM state models. Limiting maximum system performance loss, a minimum quantification error aimed hierarchical clustering algorithm is designed to choose the most representative models. The system performance is improved by about 1.5% while saving 40% of the system expense. About 92% of the performance may also be maintained while reducing 70% of system parameters. The suggested method is quite useful for designing pen based interface for various handheld devices. This paper focuses on a state sharing method for an artificial neural network (ANN) and hidden Markov model (HMM) hybrid on line handwriting recognition system. A modeling precision based distance measure is proposed to describe similarity between two ANNs, which are used as HMM state models. Limiting maximum system performance loss, a minimum quantification error tacking hierarchical clustering algorithm is designed to choose the most representative models. The system performance is improved by about 1.5% while saving 40% of the system expense. About 92% of the performance The suggested method is quite useful for designing pen based interface for various handheld devices.