Attention-based encoder-decoder (AED) models
have achieved promising performance in speech recognition.
However, because the decoder predicts text tokens (such as
characters or words) in an autoregressive manner, it is difficult
for an AED model to predict all tokens in parallel. This makes
the inference speed relatively slow. In contrast, we propose an
end-to-end non-autoregressive speech recognition model called
LASO (Listen Attentively, and Spell Once). The model aggre-
gates encoded speech features into the hidden representations
corresponding to each token with attention mechanisms. Thus,
the model can capture the token relations by self-attention on
the aggregated hidden representations from the whole speech
signal rather than autoregressive modeling on tokens. Without
explicitly autoregressive language modeling, this model predicts
all tokens in the sequence in parallel so that the inference is
efficient. Moreover, we propose a cross-modal transfer learning
method to use a text-modal language model to improve the
performance of speech-modal LASO by aligning token semantics.
We conduct experiments on two scales of public Chinese speech
datasets AISHELL-1 and AISHELL-2. Experimental results
show that our proposed model achieves a speedup of about 50×
and competitive performance, compared with the autoregressive
transformer models. And the cross-modal knowledge transferring
from the text-modal model can improve the performance of the
speech-modal model.
Institute of Automation, Chinese Academy of Sciences
推荐引用方式 GB/T 7714
Ye Bai,Jiangyan Yi,Jianhua Tao,et al. Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing,2021(29):1897 - 1911.
APA
Ye Bai,Jiangyan Yi,Jianhua Tao,Zhengkun Tian,Zhengqi Wen,&Shuai Zhang.(2021).Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT.IEEE/ACM Transactions on Audio, Speech, and Language Processing(29),1897 - 1911.
MLA
Ye Bai,et al."Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT".IEEE/ACM Transactions on Audio, Speech, and Language Processing .29(2021):1897 - 1911.
修改评论