Reparameterizing and dynamically quantizing image features for image generation

doi:10.1016/j.patcog.2023.109962

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 图像与视频分析团队

	Reparameterizing and dynamically quantizing image features for image generation
	Sun, Mingzhen 1,2; Wang, Weining 2; Zhu, Xinxin 2; Liu, Jing 1,2
刊名	PATTERN RECOGNITION
	2024-02-01
卷号	146 页码:11
关键词	Vector quantization Variational auto-encoder Unconditional image generation Text-to-image generation Autoregressive generation
ISSN号	0031-3203
DOI	10.1016/j.patcog.2023.109962
通讯作者	Liu, Jing(jliu@nlpr.ia.ac.cn)
英文摘要	For autoregressive image generation, vector-quantized VAEs (VQ-VAEs) quantize image features with discrete codebook entries and reconstruct images from quantized features. However, they treat each codebook entry separately, which causes losses of image details. In this paper, we propose to reparameterize image features with weight vectors to treat all codebook entries as an entity, and present a novel dynamically vector quantized VAE (DVQ-VAE) to quantize reparameterized image features. Specifically, each image feature corresponds to a weight vector and we sum weighted codebook entries to obtain values of image features. In this way, image features can incorporate information from different codebook entries. Additionally, a novel continuous weight regularization loss is proposed to improve the reconstruction of image details. Our method achieves competitive results with prior state-of-the-art works for image generation and extensive experiments are conducted to take a deep insight into our DVQ-VAE.
资助项目	National Key Research and De-velopment Program of China[2022ZD0118801] ; National Natural Science Foundation of China[U21B2043] ; National Natural Science Foundation of China[62102419] ; National Natural Science Foundation of China[62102416]
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	ELSEVIER SCI LTD
WOS记录号	WOS:001086812100001
资助机构	National Key Research and De-velopment Program of China ; National Natural Science Foundation of China
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/54358]
专题	自动化研究所_模式识别国家重点实验室_图像与视频分析团队
通讯作者	Liu, Jing
作者单位	1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Sun, Mingzhen,Wang, Weining,Zhu, Xinxin,et al. Reparameterizing and dynamically quantizing image features for image generation[J]. PATTERN RECOGNITION,2024,146:11.
APA	Sun, Mingzhen,Wang, Weining,Zhu, Xinxin,&Liu, Jing.(2024).Reparameterizing and dynamically quantizing image features for image generation.PATTERN RECOGNITION,146,11.
MLA	Sun, Mingzhen,et al."Reparameterizing and dynamically quantizing image features for image generation".PATTERN RECOGNITION 146(2024):11.