Search Results for « Large Language Models (LLMs) »

As ever larger language models grow more ubiquitous, it is crucial to consider their environmental impact. Characterised by extreme size and resource use, recent generations of models have been criticised for their voracious appetite for compute, and thus significant carbon footprint. Although reporting of carbon impact has grown more common in machine learning papers, this reporting is usually limited to compute resources used strictly for training. In this work, we propose a holistic assessment of the footprint of an extreme-scale language model, Noor. Noor is an ongoing project aiming to develop the largest multi-task Arabic language models -- with up to 13B parameters -- leveraging zero-shot generalisation to enable a wide range of downstream tasks via natural language instructions. We assess the total carbon bill of the entire project: starting with data collection and storage costs, including research and development budgets, pretraining costs, future serving estimates, and other exogenous costs necessary for this international cooperation. Notably, we find that inference costs and exogenous factors can have a significant impact on total budget. Finally, we discuss pathways to reduce the carbon footprint of extreme-scale models.

We will introduce Large Language Models (LLMs), focusing on their core architecture, training methodologies, practical applications, and some important considerations. We begin with a brief exploration of the transformer architecture, examining how architectural choices and scaling laws have shaped modern language models. We then investigate key training paradigms, with particular emphasis on Reinforcement Learning from Human Feedback (RLHF) and instruction tuning, while briefly touching on emerging approaches like Direct Preference Optimisation (DPO). The discussion progresses through the evolution of knowledge integration techniques, from basic context stuffing to Retrieval-Augmented Generation (RAG), highlighting their practical implications. We will discuss the critical issue of hallucinations in LLM outputs, providing concrete examples and verification strategies. To conclude, a quick overview of the current software landscape and immediate future developments in the field, and a look ahead to the rest of the webinar series.

In recent years, training methods centered on Reinforcement Learning (RL) have markedly enhanced the reasoning and alignment performance of Large Language Models (LLMs), particularly in understanding human intents, following user instructions, and bolstering inferential strength. Although existing surveys offer overviews of RL augmented LLMs, their scope is often limited, failing to provide a comprehensive summary of how RL operates across the full lifecycle of LLMs. We systematically review the theoretical and practical advancements whereby RL empowers LLMs, especially Reinforcement Learning with Verifiable Rewards (RLVR). First, we briefly introduce the basic theory of RL. Second, we thoroughly detail application strategies for RL across various phases of the LLM lifecycle, including pre-training, alignment fine-tuning, and reinforced reasoning. In particular, we emphasize that RL methods in the reinforced reasoning phase serve as a pivotal driving force for advancing model reasoning to its limits. Next, we collate existing datasets and evaluation benchmarks currently used for RL fine-tuning, spanning human-annotated datasets, AI-assisted preference data, and program-verification-style corpora. Subsequently, we review the mainstream open-source tools and training frameworks available, providing clear practical references for subsequent research. Finally, we analyse the future challenges and trends in the field of RL-enhanced LLMs. This survey aims to present researchers and practitioners with the latest developments and frontier trends at the intersection of RL and LLMs, with the goal of fostering the evolution of LLMs that are more intelligent, generalizable, and secure.

This paper explores the advancements in making large language models (LLMs) more human-like. We focus on techniques that enhance natural language understanding, conversational coherence, and emotional intelligence in AI systems. The study evaluates various approaches, including fine-tuning with diverse datasets, incorporating psychological principles, and designing models that better mimic human reasoning patterns. Our findings demonstrate that these enhancements not only improve user interactions but also open new possibilities for AI applications across different domains. Future work will address the ethical implications and potential biases introduced by these human-like attributes.

In recent months, I have been watching with a mix of trepidation and curiosity as the introduction of Large language models (LLMs) has become mainstream in academic research and technical development. As a practitioner and educator in systems engineering and mechanical design, it is my responsibility to keep my knowledge and skill set up-to-date and reflect this in my research and teaching. Part of this includes evaluating new tools and technology within my area of expertise and providing realistic professional guidance based on my experiences. This reflection outlines my thoughts and professional opinion on the use of LLMs to support systems engineering and mechanical design. As a designer with training in classic mathematical optimization techniques and large-scale systems engineering, I first worked to understand LLMs and how they relate to my area of expertise in systems engineering and mechanical design. From an engineering perspective, an LLM is an artificial intelligence (AI) or optimization model which has been designed and trained using human language. Similar to how most of these kinds of tools are trained using labeled images or real-world technical data, LLMs recognize patterns and interactions between words and pieces of written text. The initial models are only as good as their training data, but continue to improve over time as they encounter more and more data. As with many other types of AI models, the initial training is done with a massive dataset (text in the case of LLMs) using an unsupervised learning algorithm. Once the baseline model is trained using the general dataset, it can then be refined using more specialized training data to develop more specialized tools. It is clear that LLMs are not some kind of new technology or “silver bullet” tool but simply a different application and logical progression/development of existing design and AI tools which derive from classic mathematical optimization. This is reassuring for those nervous about the emergence of a technology which seems to blur the lines between human and machine. In reality, it is simply a mathematical model which mimics human speech and writing by learning patterns.

In this study, we delve into the validity of conventional personality questionnaires in capturing the human-like personality traits of Large Language Models (LLMs). Our objective is to assess the congruence between the personality traits LLMs claim to possess and their demonstrated tendencies in real-world scenarios. By conducting an extensive examination of LLM outputs against observed human response patterns, we aim to understand the disjunction between self-knowledge and action in LLMs.

Do Large Language Models have cognitive abilities? Do Large Language Models have understanding? Is the correct recognition of verbal contexts or visual objects, based on pre-learning on a large training dataset, a manifestation of the ability to solve cognitive tasks? Or is any LLM just a statistical approximator that compiles averaged texts from its huge dataset close to the specified prompts? The answers to these questions require rigorous formal definitions of the cognitive concepts of "knowledge", "understanding" and related terms.

Large language models (LLMs) have demonstrated remarkable performances on a wide range of natural language tasks. Yet, LLMs' successes have been largely restricted to tasks concerning words, sentences, or documents, and it remains questionable how much they understand the minimal units of text, namely characters. In this paper, we examine contemporary LLMs regarding their ability to understand character composition of words, and show that most of them fail to reliably carry out even the simple tasks that can be handled by humans with perfection. We analyze their behaviors with comparison to token level performances, and discuss the potential directions for future research.