The contradiction between data quantity and rendering quality is a rather troublesome issue in Image-Based Rendering (IBR). In this paper, we present an adaptive sampling method to relieve the contradiction. This method determines the sampling positions by using minimum expectation error criterion of the signal waveform. It is a non-iterative sampling process and could reach a quite good rendering quality with a little date quantity. Also, the method uses each ray in a camera as an independent sample and adjusts them individually. We apply this method to the IBR sampling and reconstruction with a light field setup. The experimental results show that the rendering quality is 3.7~7.8dB higher than that of the traditional method with the same sample size. The PSNR is above 30dB with less than 15％ of the original samples. This method can apply to 1-D, 2-D signals and other kinds of IBR technologies.