We present a novel vocabulary tree data structure for adaptive SIFT matching. Our matching process contains an offline module to cluster features from a group of reference images and an online module to match them to the live images in order to enhance matching robustness. The main contribution lies in constructing two different vocabulary structures cascaded in one tree, which we have called cascading vocabulary tree that can be used to not only cluster features but also implement exact feature matching as k-d tree does. Cascading keyframe selection using our vocabulary structure can be put the matching process forward, which gives us a way to employ a cascading feature matching strategy to combine matching results of cascading vocabulary tree and keyframe. Experimental results show that our method not only dramatically enhances matching robustness but also has enough flexibility to adaptively adjust itself to meet diverse requirements of domain applications for efficiency and robustness of SIFT matching.