In the simulation of rupture processes of seismic sources by using either numerical method or rock mechanics experiments, improper setting of the specimen size will influence the stress field near the faults. In this study, 2D finite element method (FEM) was used to calculate the stress field of rock specimens in different sizes with fixed-size elliptic holes. The calculated stress field was compared with analytic solution for elliptic-hole problem in an infinite medium. Numerical results showed that boundary effect of a rock specimen with an elliptic hole on stress field under uniaxial compression cannot be neglected. Critical aspect ratio of the specimen is about 3:2, and critical ratio of distance between the tip of the hole and the border of specimen (d) to the major axis of the elliptic hole (l) is about 7.3. Numerical analysis on rock specimen size can provide theoretical reference for rock specimen experiments, and it is also helpful for setting of model sizes in numerical simulations of fault movement.