# Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models

**Authors:** Haoqiang Kang, Terra Blevins, Luke Zettlemoyer  
**arXiv:** [2304.13803](https://arxiv.org/abs/2304.13803)  
**Date:** 2023-04-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.13803v1/

## Key points

- The study focuses on Pretrained Language Models (PLMs) and their ability to capture cross-lingual word sense knowledge.
- PLMs can be finetuned for tasks such as translation and multilingual word sense disambiguation (WSD), but they often struggle in a zero-shot setting.
- The authors introduce Contextual Word-Level Translation (C-WLT), an extension of word-level translation that prompts the model to translate a given word in context, to address this issue.
- Larger models perform better at using context to improve WLT performance.
- The authors propose a zero-shot approach for WSD, tested on 18 languages from the XL-WSD dataset. Their method outperforms fully supervised baselines on recall for many evaluation languages without additional training or finetuning.
- The study compares their approach to prior work on multilingual WSD using automatic metrics such as recall and Jaccard index. They found that ensembling English, Chinese, and Russian as target languages with English prompts achieved a balance between recall and Jaccard Index.
- Despite being performed zero-shot from a pretrained language model, their method achieves higher recall compared to prior works in 11 out of the 18 source languages, showing that translation-based approaches can identify correct sense labels as well or better than supervised methods.
- Limitations of their approach include relying on the availability of high quality translations and not considering polysemous words with multiple senses within one language.
- Future research directions are suggested to address these limitations and improve applicability.
- Overall, this study presents a first step towards leveraging cross-lingual knowledge inside PLMs for robust zero-shot reasoning in any language.

## Summary

The study focuses on Pretrained Language Models (PLMs) and their ability to capture cross-lingual word sense knowledge. While PLMs can be finetuned for tasks such as translation and multilingual word sense disambiguation (WSD), they often struggle in a zero-shot setting. To address this issue, the authors introduce Contextual Word-Level Translation (C-WLT), an extension of word-level translation that prompts the model to translate a given word in context. The study investigates how well PLMs encode cross-lingual word sense knowledge with C-WLT and finds that larger models perform better at using context to improve WLT performance. Building on C-WLT, the authors propose a zero-shot approach for WSD, tested on 18 languages from the XL-WSD dataset. Their method outperforms fully supervised baselines on recall for many evaluation languages without additional training or finetuning. The study compares their approach to prior work on multilingual WSD using automatic metrics such as recall and Jaccard index. They found that ensembling English, Chinese, and Russian as target languages with English prompts achieved a balance between recall and Jaccard Index. Despite being performed zero-shot from a pretrained language model, their method achieves higher recall compared to prior works in 11 out of the 18 source languages, showing that translation-based approaches can identify correct sense labels as well or better than supervised methods. However, there are limitations to their approach such as relying on the availability of high quality translations and not considering polysemous words with multiple senses within one language. The authors recognize these limitations and suggest future research directions to address them and improve applicability. Overall, this study presents a first step towards leveraging cross-lingual knowledge inside PLMs for robust zero-shot reasoning in any language.