Gaussian Similarity Analysis (GSA)algorithm can be used to estimate the similarity between two Gaussian distributed variables with full covariance matrix. Based on this algorithm, we propose a method in speaker adaptation of covariance. It is different from the traditional algorithms, which mainly focus on the adaptation of mean vector of state observation probability density. A binary decision tree is constructed offline with the similarity measure and the adaptation procedure is data-driven. It can be shown from the experiments that we can get a significant further improvement over the mean vectors adaptation.