SeaLLMs -- Large Language Models for Southeast Asia

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Not relevant

   PolyLM: An Open Source Polyglot Large Language Model

   Xiangpeng Wei, Haoran Wei, Huan Lin et al. · 2023 · Computation and Language · AI summary available

   Abstract

   Large language models (LLMs) demonstrate remarkable ability to comprehend, reason, and generate following nature language instructions. However, the development of LLMs has been primarily focused on high-resource languages, such as English, thereby limiting their applicability and research in other languages. Consequently, we present PolyLM, a multilingual LLM trained on 640 billion (B) tokens, avaliable in two model sizes: 1.7B and 13B. To enhance its multilingual capabilities, we 1) integrate bilingual data into training data; and 2) adopt a curriculum learning strategy that increases the proportion of non-English data from 30% in the first stage to 60% in the final stage during pre-traini…

2. Close Not relevant

   Large Language Models: A Survey

   Shervin Minaee, Tomas Mikolov, Narjes Nikzad et al. · 2024 · Computation and Language · AI summary available

   Abstract

   Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks, since the release of ChatGPT in November 2022. LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data, as predicted by scaling laws \cite{kaplan2020scaling,hoffmann2022training}. The research area of LLMs, while very recent, is evolving rapidly in many different ways. In this paper, we review some of the most prominent LLMs, including three popular LLM families (GPT, LLaMA, PaLM), and discuss their characteristics, contributions and limitations. We also give …

3. Close Not relevant

   Better to Ask in English: Evaluation of Large Language Models on English, Low-resource and Cross-Lingual Settings

   Krishno Dey, Prerona Tarannum, Md. Arid Hasan et al. · 2024 · Computation and Language · AI summary available

   Abstract

   Large Language Models (LLMs) are trained on massive amounts of data, enabling their application across diverse domains and tasks. Despite their remarkable performance, most LLMs are developed and evaluated primarily in English. Recently, a few multi-lingual LLMs have emerged, but their performance in low-resource languages, especially the most spoken languages in South Asia, is less explored. To address this gap, in this study, we evaluate LLMs such as GPT-4, Llama 2, and Gemini to analyze their effectiveness in English compared to other low-resource languages from South Asia (e.g., Bangla, Hindi, and Urdu). Specifically, we utilized zero-shot prompting and five different prompt settings to …

4. Close Not relevant

   SEM: Reinforcement Learning for Search-Efficient Large Language Models

   Zeyang Sha, Shiwen Cui, Weiqiang Wang · 2025 · Computation and Language · Not summarized yet

   Abstract

   Recent advancements in Large Language Models(LLMs) have demonstrated their capabilities not only in reasoning but also in invoking external tools, particularly search engines. However, teaching models to discern when to invoke search and when to rely on their internal knowledge remains a significant challenge. Existing reinforcement learning approaches often lead to redundant search behaviors, resulting in inefficiencies and over-cost. In this paper, we propose SEM, a novel post-training reinforcement learning framework that explicitly trains LLMs to optimize search usage. By constructing a balanced dataset combining MuSiQue and MMLU, we create scenarios where the model must learn to disting…

5. Close Not relevant

   Several categories of Large Language Models (LLMs): A Short Survey

   Saurabh Pahune, Manoj Chandrasekharan · 2023 · Computation and Language · AI summary available

   Abstract

   Large Language Models(LLMs)have become effective tools for natural language processing and have been used in many different fields. This essay offers a succinct summary of various LLM subcategories. The survey emphasizes recent developments and efforts made for various LLM kinds, including task-based financial LLMs, multilingual language LLMs, biomedical and clinical LLMs, vision language LLMs, and code language models. The survey gives a general summary of the methods, attributes, datasets, transformer models, and comparison metrics applied in each category of LLMs. Furthermore, it highlights unresolved problems in the field of developing chatbots and virtual assistants, such as boosting na…

6. Close Not relevant

   Sentiment Analysis in the Era of Large Language Models: A Reality Check

   Wenxuan Zhang, Yue Deng, Bing Liu et al. · 2023 · Computation and Language · AI summary available

   Abstract

   Sentiment analysis (SA) has been a long-standing research area in natural language processing. It can offer rich insights into human sentiments and opinions and has thus seen considerable interest from both academia and industry. With the advent of large language models (LLMs) such as ChatGPT, there is a great potential for their employment on SA problems. However, the extent to which existing LLMs can be leveraged for different sentiment analysis tasks remains unclear. This paper aims to provide a comprehensive investigation into the capabilities of LLMs in performing various sentiment analysis tasks, from conventional sentiment classification to aspect-based sentiment analysis and multifac…

7. Close Not relevant

   A Survey of Large Language Models

   Wayne Xin Zhao, Kun Zhou, Junyi Li et al. · 2023 · Computation and Language · AI summary available

   Abstract

   Language is essentially a complex, intricate system of human expressions governed by grammatical rules. It poses a significant challenge to develop capable AI algorithms for comprehending and grasping a language. As a major approach, language modeling has been widely studied for language understanding and generation in the past two decades, evolving from statistical language models to neural language models. Recently, pre-trained language models (PLMs) have been proposed by pre-training Transformer models over large-scale corpora, showing strong capabilities in solving various NLP tasks. Since researchers have found that model scaling can lead to performance improvement, they further study t…

8. Close Not relevant

   Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanese

   Rifki Afina Putri, Faiz Ghifari Haznitrama, Dea Adhista et al. · 2024 · Computation and Language · AI summary available

   Abstract

   Large Language Models (LLMs) are increasingly being used to generate synthetic data for training and evaluating models. However, it is unclear whether they can generate a good quality of question answering (QA) dataset that incorporates knowledge and cultural nuance embedded in a language, especially for low-resource languages. In this study, we investigate the effectiveness of using LLMs in generating culturally relevant commonsense QA datasets for Indonesian and Sundanese languages. To do so, we create datasets for these languages using various methods involving both LLMs and human annotators. Our experiments show that the current best-performing LLM, GPT-4 Turbo, is capable of generating …