Data Science for Society Seminar <> Massive vs. Curated Embeddings for Low-Resourced Languages: the Case of Yorùbá and Twi
Jesujoba Alabi
Talk Details
Abstract
The success of several architectures to learn semantic representations from unannotated text and the availability of this kind of texts in online multilingual resources such as Wikipedia has facilitated the massive and automatic creation of resources for multiple languages. The evaluation of such resources is usually done for the high-resourced languages, where one has a smorgasbord of tasks and test sets to evaluate on. For low-resourced languages, the evaluation is more difficult and normally ignored, with the hope that the impressive capability of deep learning architectures to learn (multilingual) representations in the high-resource setting holds in the low-resourced setting too. In this paper we focus on two African languages, Yorùbá and Twi, and compare the word embeddings obtained in this way, with word embeddings obtained from curated corpora and a language-dependent processing. We analyse the noise in the publicly available corpora, collect high quality and noisy data for the two languages and quantify the improvements that depend not only on the amount of data but on the quality too. We also use different architectures that learn word representations both from surface forms and characters to further exploit all the available information which showed to be important for these languages. For the evaluation, we manually translate the wordsim-353 word pairs dataset from English into Yorùbá and Twi. We extend the analysis to contextual word embeddings and evaluate multilingual BERT on a named entity recognition task. For this, we annotate with named entities the Global Voices corpus for Yorùbá . As the output of the work, we provide corpora, embeddings and the test suites for both languages
Speaker Bio
Jesujoba Alabi is a Computer Science student at Saarland University. He was a research assistant at the Multilinguality and Language Technology research group, German Research Centre for Artificial Intelligence, (DFKI) Saarbrücken where he is currently taking his master’s thesis. He is currently a research assistant at the Database and Information Systems group at the Max Planck Institute for Informatics (MPI-INF), where he works on Question Answering Systems. His research interests include Machine Translation, NLP for low resource setting, Information Extraction and Question Answering Systems.