On the Copying Problem of Unsupervised NMT: Training Schedule with a Language Discriminator Loss

Yihong Liu, Alexandra Chronopoulou, Hinrich Schütze, Alexander Fraser

International Conference on Spoken Language Translation (IWSLT) 2023

Although unsupervised neural machine translation (UNMT) has achieved
success in many language pairs, the copying problem, i.e., directly
copying some parts of the input sentence as the translation, is common
among distant language pairs, especially when low-resource languages
are involved. We find this issue is closely related to an unexpected
copying behavior during online back-translation (BT). In this work, we
propose a simple but effective training schedule that incorporates a
language discriminator loss. The loss imposes constraints on the
intermediate translation so that the translation is in the desired
language. By conducting extensive experiments on different language
pairs, including similar and distant, high and low-resource languages,
we find that our method alleviates the copying problem, thus improving
the translation performance on low-resource languages.