As an external manifestation of human emotions, expression recognition plays an important role in human-computer interaction. Although existing expression recognition methods work perfect on constrained frontal faces, there are still many challenges in expression recognition in natural scenes due to different unrestricted conditions. Face recognition in natural scenes is a problem that the intra-class gap is larger than the inter-class gap. In order to solve this problem, we propose a method of generating a reference expression using GAN and comparing it with the original expression to generate differential features, so as to avoid interference of irrelevant information on expression recognition. Besides, we have specifically optimized the GAN network that generates reference expressions to make the generated reference expression more natural. We used Resnet50-V2 pre-trained on ImageNet to better present the differential features of the original expression and the reference expression. After testing on the two datasets, our model achieves higher accuracy than other models.