Investigating Gated Recurrent Neural Networks for Acoustic Modeling | |
Zhao, Yuanyuan; Li, Jie; Xu, Shuang; Xu, Bo; Yuanyuan Zhao | |
2016-10 | |
会议日期 | October 17-20 |
会议地点 | Tianjin, China |
关键词 | Gated Recurrent Neural Networks Long Short-term Memory Unit Gated Recurrent Neural Networks Long Short-term Memory Projected Unit |
英文摘要 | Recurrent neural networks (RNNs) with a gating mechanism have been shown to give state-of-the-art performance in acoustic modeling, such as gated recurrent unit (GRU), long short-term memory (LSTM), long short-term memory projected (LSTMP), etc. But little is known about why these gated RNNs work and what the differences are among these networks. Based on a series of experimental comparison and analysis, we find that: a) GRU usually performs better than LSTM, for possibly GRU is able to modulate the previous memory content through the learned reset gates, helping to model the long-span dependence more efficiently for speech sequence; b) LSTMP shows comparable performance with GRU, since LSTMP has the similar ability of information selection and combination by an automatic learned linear transformation in a weight-sharing way. In experiments, a visual analysis method is adopted to understand the historical information selection mechanism in RNNs in contrast to DNN. Experimental results on three different speech recognition tasks demonstrate the above conclusions and 5%-13% relative PER or CER reduction is observed. |
语种 | 英语 |
内容类型 | 会议论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/41081] |
专题 | 数字内容技术与服务研究中心_听觉模型与认知计算 |
通讯作者 | Yuanyuan Zhao |
推荐引用方式 GB/T 7714 | Zhao, Yuanyuan,Li, Jie,Xu, Shuang,et al. Investigating Gated Recurrent Neural Networks for Acoustic Modeling[C]. 见:. Tianjin, China. October 17-20. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论