Actively Learning Human Gaze Shifting Paths for Semantics-Aware Photo Cropping

CORC > 厦门大学 > 信息技术－已发表论文

	Actively Learning Human Gaze Shifting Paths for Semantics-Aware Photo Cropping
	Zhang, Luming ; Gao, Yue ; Ji, Rongrong ; Xia, Yingjie ; Dai, Qionghai ; Li, Xuelong ; Ji RR(纪荣嵘)
刊名	http://dx.doi.org/10.1109/TIP.2014.2311658
	2014
关键词	OBJECT RETRIEVAL RECOGNITION CLASSIFICATION MANIFOLD
英文摘要	Project of National Science Foundation of China [61125106, 61327902, 61035002, 61373076, 61002009]; Fundamental Research Funds for the Central Universities [2013121026]; 985 Project of Xiamen University; Key Science and Technology Program of Zhejiang Province of China [2012C01035-1]; Zhejiang Provincial Natural Science Foundation of China [LZ13F020004]; Singapore National Research Foundation under its International Research Centre@ Singapore Funding Initiative; Photo cropping is a widely used tool in printing industry, photography, and cinematography. Conventional cropping models suffer from the following three challenges. First, the deemphasized role of semantic contents that are many times more important than low-level features in photo aesthetics. Second, the absence of a sequential ordering in the existing models. In contrast, humans look at semantically important regions sequentially when viewing a photo. Third, the difficulty of leveraging inputs from multiple users. Experience from multiple users is particularly critical in cropping as photo assessment is quite a subjective task. To address these challenges, this paper proposes semantics-aware photo cropping, which crops a photo by simulating the process of humans sequentially perceiving semantically important regions of a photo. We first project the local features (graphlets in this paper) onto the semantic space, which is constructed based on the category information of the training photos. An efficient learning algorithm is then derived to sequentially select semantically representative graphlets of a photo, and the selecting process can be interpreted by a path, which simulates humans actively perceiving semantics in a photo. Furthermore, we learn a prior distribution of such active graphlet paths from training photos that are marked as aesthetically pleasing by multiple users. The learned priors enforce the corresponding active graphlet path of a test photo to be maximally similar to those from the training photos. Experimental results show that: 1) the active graphlet path accurately predicts human gaze shifting, and thus is more indicative for photo aesthetics than conventional saliency maps and 2) the cropped photos produced by our approach outperform its competitors in both qualitative and quantitative comparisons.
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
内容类型	期刊论文
源URL	[http://dspace.xmu.edu.cn/handle/2288/92691]
专题	信息技术－已发表论文
推荐引用方式 GB/T 7714	Zhang, Luming,Gao, Yue,Ji, Rongrong,et al. Actively Learning Human Gaze Shifting Paths for Semantics-Aware Photo Cropping[J]. http://dx.doi.org/10.1109/TIP.2014.2311658,2014.
APA	Zhang, Luming.,Gao, Yue.,Ji, Rongrong.,Xia, Yingjie.,Dai, Qionghai.,...&纪荣嵘.(2014).Actively Learning Human Gaze Shifting Paths for Semantics-Aware Photo Cropping.http://dx.doi.org/10.1109/TIP.2014.2311658.
MLA	Zhang, Luming,et al."Actively Learning Human Gaze Shifting Paths for Semantics-Aware Photo Cropping".http://dx.doi.org/10.1109/TIP.2014.2311658 (2014).