Duolingo has announced a new machine translation competition called STAPLE, which will be part of the WNGT and SIGEDU workshops at next year’s ACL conference.
The world’s most popular language-learning app offers its users online learning of more than 30 languages delivered as bite-size lessons in a game format. A big part of the learning within the Duolingo app happens via translation based exercises across the chosen and the learner’s preferred language. However, the translation very often happens to be not unique making a set of possible translations still correct. For this reason, Duolingo is hosting a challenge whose goal is to improve the process of grading such translation-based exercises.
STAPLE, as the new competition was named, will be opened next month and the goal is to predict a set of possible and plausible translations of a given sentence in the English language. The task will cover 5 pre-defined languages: Portuguese, Hungarian, Japanese, Korean and Vietnamese. In each of the languages, for the given sentences there exist multiple translations that are correct.
In order to provide a starting point, engineers from Duolingo designed the competition in such a way that for every sentence there will be a reference translation provided by an automatic translation system from Amazon. All of the prompt sentences within the competition come from Duolingo courses as well as most of the translations which are also scored in a specific way using a defined metric.
As part of the competition, a training dataset will be released on January 13th, 2020, followed by the releases of blind “dev” set and a blind test set on March 2nd and March 30th respectively. The competition will accept submissions until April 6th and the winners will be announced a few days later on April 8th.