Domain-Oriented Semantic Embedding for Zero-Shot Learning

doi:10.1109/TMM.2020.3033124

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 多媒体计算与图形学团队

	Domain-Oriented Semantic Embedding for Zero-Shot Learning
	Min, Shaobo 2; Yao, Hantao 1; Xie, Hongtao 2; Zha, Zheng-Jun 2; Zhang, Yongdong 2
刊名	IEEE TRANSACTIONS ON MULTIMEDIA
	2021
卷号	23 页码:3919-3930
关键词	Semantics Visualization Image recognition Image reconstruction Training Gallium nitride Search problems Zero-shot learning multi-modality embedding recognition
ISSN号	1520-9210
DOI	10.1109/TMM.2020.3033124
通讯作者	Xie, Hongtao(htxie@ustc.edu.cn) ; Zhang, Yongdong(zhyd73@ustc.edu.cn)
英文摘要	Zero-Shot Learning (ZSL) targets to recognize images from new classes. Existing methods focus on learning a projection function to associate the visual features and category descriptions in the seen domain, which is directly transferred to the unseen domain. However, due to the inherent domain shift, a single shared projection cannot fully capture the domain difference and similarity, thereby making the unseen samples tend to be recognized as seen categories. In this paper, we propose a novel Domain-Oriented Semantic Embedding (DOSE) network that learns specific projections for different domains to better capture the domain characteristics for unbiased ZSL. Besides a domain-shared projection, DOSE learns two auxiliary domain-specific sub-projections to model the semantic-visual association in respective seen and unseen domains. Specifically, the domain-specific projections are learned in a cycle consistency way to capture domain characteristics, and a domain division constraint is developed to penalize the margin between two domain embeddings. Furthermore, to boost semantic-visual association, a semantic-visual dual attention module is designed to automatically remove trivial information in both visual and semantic embeddings under a co-guidance learning manner. Experiments on four public benchmarks prove that the proposed DOSE is robust to the domain shift problem in ZSL and obtains an averaged 5.6% improvement in terms of harmonic mean.
资助项目	National Key Research, and Development Program of China[2017YFC0820600] ; National Nature Science Foundation of China[61525206] ; National Nature Science Foundation of China[62022076] ; National Nature Science Foundation of China[U1936210] ; National Postdoctoral Programme for Innovative Talents[BX20180358] ; Youth Innovation Promotion Association Chinese Academy of Sciences[2017209]
WOS研究方向	Computer Science ; Telecommunications
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000709093100038
资助机构	National Key Research, and Development Program of China ; National Nature Science Foundation of China ; National Postdoctoral Programme for Innovative Talents ; Youth Innovation Promotion Association Chinese Academy of Sciences
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/46306]
专题	自动化研究所_模式识别国家重点实验室_多媒体计算与图形学团队
通讯作者	Xie, Hongtao; Zhang, Yongdong
作者单位	1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100864, Peoples R China 2.Univ Sci & Technol China, Natl Engn Lab Brain Inspired Intelligence Technol, Hefei 230026, Peoples R China
推荐引用方式 GB/T 7714	Min, Shaobo,Yao, Hantao,Xie, Hongtao,et al. Domain-Oriented Semantic Embedding for Zero-Shot Learning[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2021,23:3919-3930.
APA	Min, Shaobo,Yao, Hantao,Xie, Hongtao,Zha, Zheng-Jun,&Zhang, Yongdong.(2021).Domain-Oriented Semantic Embedding for Zero-Shot Learning.IEEE TRANSACTIONS ON MULTIMEDIA,23,3919-3930.
MLA	Min, Shaobo,et al."Domain-Oriented Semantic Embedding for Zero-Shot Learning".IEEE TRANSACTIONS ON MULTIMEDIA 23(2021):3919-3930.