Announcing PuoBERTa

🎉 Exciting News! After years of dedicated work, coinciding with the challenges of the COVID-19 pandemic, our collaborative effort to bolster NLP resources for Setswana has borne fruit! 🚀

We’re thrilled to unveil PuoBERTa, a tailor-made masked language model for Setswana. Our journey involved collecting, curating, and preparing a diverse set of monolingual texts to breathe life into a model that’s not just technically adept but culturally attuned. 🌐📚 [Example shown is of PuoBERTa-News, finetuned for news categorisation - test it here https://huggingface.co/dsfsi/PuoBERTa-News]

We’ve expanded the horizons for Setswana, enhancing part-of-speech tagging, named entity recognition, and news categorisation, marking a significant stride in reducing the language resource disparity. 💪🏽🌟

Stay tuned for more as we continue exploring this terrain, ensuring languages like Setswana don’t just survive but thrive in the world of AI! Together, we’re weaving a world where every language finds its digital voice. 🗣️💻

Learn more about PuoBERTa:

• 👩🏾‍💻 Github: https://github.com/dsfsi/PuoBERTa
• 🤗 Huggingface: https://huggingface.co/dsfsi/PuoBERTa
• 📖 Arxiv Preprint: https://arxiv.org/abs/2310.09141

Work with Valencia K. Wagner, Moseli Motsoehli, Richard Lastrucci, Isheanesu Dzingirai

We want to acknowledge the feedback received from colleagues at Data Science for Social Impact Research Group and Lelapa AI colleagues.

With generous support from NVIDIA, Google Research and Absa Group