Running summarizing tools on a new article

Check the paper below, then start its synthesis.


WAD-X: Improving Zero-shot Cross-lingual Transfer via Adapter-based Word Alignment

Ahtamjan Ahmat (Chinese Academy of Sciences, Xinjiang Technical Institute of Physics & Chemistry), Yating Yang (Chinese Academy of Sciences, Xinjiang Technical Institute of Physics & Chemistry), Bo Ma (Chinese Academy of Sciences, Xinjiang Technical Institute of Physics & Chemistry), Rui Dong (Chinese Academy of Sciences, Xinjiang Technical Institute of Physics & Chemistry), Kaiwen Lu (Chinese Academy of Sciences, Xinjiang Technical Institute of Physics & Chemistry), Lei Wang (Chinese Academy of Sciences, Xinjiang Technical Institute of Physics & Chemistry)

ACM Transactions on Asian and Low-Resource Language Information Processing
DOI: 10.1145/3610289 (Computer Science)
License: not stated (all rights reserved)

Abstract: Multilingual pre-trained language models (mPLMs) have achieved remarkable performance on zero-shot cross-lingual transfer learning. However, most mPLMs implicitly encourage cross-lingual alignment in pre-training stage, making it hard to capture accurate word alignment across languages. In this paper, we propose Word-align ADapters for Cross-lingual transfer (WAD-X) to explicitly align word representations of mPLMs via language-specific subspace. Taking a mPLM as the backbone model, WAD-X constructs subspace for each source-target language pair via adapters. The adapters use statistical alignment as the prior knowledge to guide word-level aligning in the corresponding bilingual semantic subspace. We evaluate our model across a set of target languages on three zero-shot cross-lingual transfer tasks: part-of-speech tagging (POS), dependency parsing (DP), and sentiment analysis (SA). Experimental results demonstrate that our proposed model improves zero-shot cross-lingual transfer on three benchmarks, with improvements of 2.19, 2.50, and 1.61 points in POS, DP, and SA tasks over strong baselines.

Published on 19 Jul. 2023