The Scale Invariant Feature Transform (SIFT) algorithm has been widely used for its excellent stability in rotation, scale and affine transformation. The local SIFT descriptor has excellent accuracy and robustness. However, it is only based on gray scale ignoring the overall color information of the image resulting in poorly recognizing to the images with rich color details. We proposed an optimized method of SIFT algorithm in this paper which shows superior performance in feature extraction and matching. RGB color space normalization is used to eliminate the effects of illumination position and intensity invariant on the image. Then we proposed a novel similarity retrieval method, which used K nearest neighbor search strategy by constructing K-D tree (k-dimensional tree), to process the key points extracted from the normalized color space. The key points of RGB space are filtered and combined efficiently. Experimental results demonstrate that the performance of the optimized algorithm is obviously better than the original SIFT algorithm in matching. The average matching accuracy of test samples is 87.05%, an average increase of 18.21%. © 2018 SPIE.