Learning Confidence for Transformer-based Neural Machine Translation | |
Yu, Lu2,3; Jiali, Zeng1; Jiajun, Zhang2,3; Shuangzhi, Wu1; Mu, Li1 | |
2022-05 | |
会议日期 | 2022-5 |
会议地点 | 线上 |
关键词 | 神经机器翻译 |
页码 | 2353-2364 |
英文摘要 | Confidence estimation aims to quantify the confidence of the model prediction, providing an expectation of success. A well-calibrated confidence estimate enables accurate failure prediction and proper risk measurement when given noisy samples and out-of-distribution data in real-world settings. However, this task remains a severe challenge for neural machine translation (NMT), where probabilities from softmax distribution fail to describe when the model is probably mistaken. To address this problem, we propose an unsupervised confidence estimate learning jointly with the training of the NMT model. We explain confidence as how many hints the NMT model needs to make a correct prediction, and more hints indicate low confidence. Specifically, the NMT model is given the option to ask for hints to improve translation accuracy at the cost of some slight penalty. Then, we approximate their level of confidence by counting the number of hints the model uses. We demonstrate that our learned confidence estimate achieves high accuracy on extensive sentence/word-level quality estimation tasks. Analytical results verify that our confidence estimate can correctly assess underlying risk in two real-world scenarios: (1) discovering noisy samples and (2) detecting out-of-domain data. We further propose a novel confidence-based instance-specific label smoothing approach based on our learned confidence estimate, which outperforms standard label smoothing. |
语种 | 英语 |
URL标识 | 查看原文 |
内容类型 | 会议论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/51845] |
专题 | 模式识别国家重点实验室_自然语言处理 |
通讯作者 | Jiajun, Zhang |
作者单位 | 1.Tencent Cloud Xiaowei 2.School of Artificial Intelligence, University of Chinese Academy of Sciences 3.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences |
推荐引用方式 GB/T 7714 | Yu, Lu,Jiali, Zeng,Jiajun, Zhang,et al. Learning Confidence for Transformer-based Neural Machine Translation[C]. 见:. 线上. 2022-5. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论