Proportional integrator (PI) is always adopted in the resonant frequency servo loop in a resonator micro optic gyro (RMOG). The oscillation phenomenon is observed when adjusting the loop gain surpassing a threshold. This phenomenon limits system performance on step response speed and residual error. Based on the experiment system, a simulation model was set up. Further analysis shows that the threshold gain is related to the system loop filter setting and the loop delay. The traditional PI frequency servo loop technique in the RMOG system cannot keep up with the environment’s disturbance quickly enough, which leads to a large residual error. A compensating method is proposed to optimize the tracking performance, solve the oscillation problem, and speed up the system response. Simulation and experiment results show that the compensated system is superior in performance. It has less residual error in the stable state and is 10 times quicker than the uncompensated system on the step response. Proportional integrator (PI) is always adopted in the resonant frequency servo loop in a resonator micro optic gyro (RMOG). The oscillation phenomenon is observed when adjusting the loop gain surpassing a threshold. This phenomenon limits system performance on step response speed and residual error Based on the experiment system, a simulation model was set up. Further analysis shows that the threshold gain is related to the system loop filter setting and the loop delay. The traditional PI frequency servo loop technique in the RMOG system can not keep up with the Simulation of the oscillation problem, and leads up to a large residual error. A compensating method is proposed to optimize the tracking performance, solve the oscillation problem, and speed up the system response. Simulation and experiment results show that the compensated system is superior in performance. It has less residual error in the stable state and is 10 times quicker than the uncompensated system on the step respo nse.