The ability to identify strange conspecifics in societies is supported by social memory, which is vital for gregarious animals and humans. The function of hippocampal principal neurons in social memory has been extensively investigated; however, the nonprincipal neuronal mechanism underlying social memory remains unclear. Here, we first observed parallel changes in the ability for social recognition and the number of parvalbumin interneurons (PVIs) in the ventral CA1 (vCA1) after social isolation. Then, using tetanus toxin-mediated neuronal lesion and optogenetic stimulation approaches, we revealed that vCA1-PVIs specifically engaged in the retrieval stage of social memory. Finally, through the in vivo Ca2+ imaging technique, we demonstrated that vCA1-PVIs exhibited higher activities when subjected mice approached a novel mouse than to a familiar one. These results highlight the crucial role of vCA1-PVIs for distinguishing novel conspecifics from other individuals and contribute to our understanding of the neuropathology of mental diseases with social memory deficits.