This paper focuses on a state sharing method for an artificial neural network (ANN) and hidden Markov model (HMM) hybrid on-line handwriting recognition system. A modeling precision-based distance measure is proposed to describe similarity between two ANNs, which are used as HMM state-models. Limiting maximum system performance loss, a minimum quantification error aimed hierarchical clustering algorithm is designed to choose the most representative models. The system performance is improved by about 1.5% while saving 40% of the system expense. About 92% of the performance may also be maintained while reducing 70% of system parameters. The suggested method is quite useful for designing pen-based interface for various handheld devices.