In this paper, we present an innovative method of double balanced differential configuration, in which two adjacent single photon avalanche diodes (SPADs) from the same wafer are configured as the first balanced structure, and the output signal from the first balanced stage is subtracted by the attenuated gate driving signal as the second balanced stage. The compact device is cooled down to 236 K to be characterized. At a gate repetition rate of 400 MHz and a 1 550 nm laser repetition rate of 10 MHz, the maximum photon detection efficiency of 13.5% can be achieved. The dark count rate is about 10^-4ns^-1at photon detection efficiency of 10%. The afterpulsing probability decreases with time exponentially. It is shown that this configuration is effective to discriminate the ultra-weak avalanche signal in high speed gating rates.