A novel modulation recognition algorithm is proposed by introducing a Chen-Harker-Kanzow-Smale(CHKS) smooth function into the C-support vector machine deformation algorithm. A set of seven characteristic parameters is selected from a range of parameters of communication signals including instantaneous amplitude, phase, and frequency. And the Newton-Armijo algorithm is utilized to train the proposed algorithm, namely, smooth CHKS smooth support vector machine(SCHKS-SSVM). Compared with the existing algorithms, the proposed algorithm not only solves the non-differentiable problem of the second order objective function, but also reduces the recognition error. It significantly improves the training speed and also saves a large amount of storage space through large-scale sorting problems. The simulation results show that the recognition rate of the algorithm can batch training. Therefore, the proposed algorithm is suitable for solving the problem of high dimension and its recognition can exceed 95%when the signal-to-noise ratio is no less than 10 d B. A novel modulation recognition algorithm is proposed by introducing a Chen-Harker-Kanzow-Smale (CHKS) smooth function into the C-support vector machine deformation algorithm. A set of seven characteristic parameters is selected from a range of parameters of communication signals including instantaneous Compared with the existing algorithms, the proposed algorithm not only solves the non- differentiable problem of the second order objective function, but also reduces the recognition error. It significantly improves the training speed and also saves a large amount of storage space through large-scale sorting problems. The simulation results show that the recognition rate of the algorithm can batch training. Thus, the proposed algorithm is suitable for solving the problem of high dimension and its recognition can ex ceed 95% when the signal-to-noise ratio is no less than 10 d B