Résultats de recherche pour « Generative Pre-trained Transformer (GPT) technology »

Mobility trajectories are essential for understanding urban dynamics and enhancing urban planning, yet access to such data is frequently hindered by privacy concerns. This research introduces a transformative framework for generating large-scale urban mobility trajectories, employing a novel application of a transformer-based model pre-trained and fine-tuned through a two-phase process. Initially, trajectory generation is conceptualized as an offline reinforcement learning (RL) problem, with a significant reduction in vocabulary space achieved during tokenization. The integration of Inverse Reinforcement Learning (IRL) allows for the capture of trajectory-wise reward signals, leveraging historical data to infer individual mobility preferences. Subsequently, the pre-trained model is fine-tuned using the constructed reward model, effectively addressing the challenges inherent in traditional RL-based autoregressive methods, such as long-term credit assignment and handling of sparse reward environments. Comprehensive evaluations on multiple datasets illustrate that our framework markedly surpasses existing models in terms of reliability and diversity. Our findings not only advance the field of urban mobility modeling but also provide a robust methodology for simulating urban data, with significant implications for traffic management and urban development planning. The implementation is publicly available at https://github.com/Wangjw6/TrajGPT_R.

Knowledge Graphs popularity has been rapidly growing in last years. All that knowledge is available for people to query it through the many online databases on the internet. Though, it would be a great achievement if non-programmer users could access whatever information they want to know. There has been a lot of effort oriented to solve this task using natural language processing tools and creativity encouragement by way of many challenges. Our approach focuses on assuming a correct entity linking on the natural language questions and training a GPT model to create SPARQL queries from them. We managed to isolate which property of the task can be the most difficult to solve at few or zero-shot and we proposed pre-training on all entities (under CWA) to improve the performance. We obtained a 62.703% accuracy of exact SPARQL matches on testing at 3-shots, a F1 of 0.809 on the entity linking challenge and a F1 of 0.009 on the question answering challenge.

This paper explores the use of Generative Pre-trained Transformers (GPT) in strategic game experiments, specifically the ultimatum game and the prisoner's dilemma. I designed prompts and architectures to enable GPT to understand the game rules and to generate both its choices and the reasoning behind decisions. The key findings show that GPT exhibits behaviours similar to human responses, such as making positive offers and rejecting unfair ones in the ultimatum game, along with conditional cooperation in the prisoner's dilemma. The study explores how prompting GPT with traits of fairness concern or selfishness influences its decisions. Notably, the "fair" GPT in the ultimatum game tends to make higher offers and reject offers more frequently compared to the "selfish" GPT. In the prisoner's dilemma, high cooperation rates are maintained only when both GPT players are "fair". The reasoning statements GPT produces during gameplay reveal the underlying logic of certain intriguing patterns observed in the games. Overall, this research shows the potential of GPT as a valuable tool in social science research, especially in experimental studies and social simulations.

GPT (Generative Pre-trained Transformer) represents advanced language models that have significantly reshaped the academic writing landscape. These sophisticated language models offer invaluable support throughout all phases of research work, facilitating idea generation, enhancing drafting processes, and overcoming challenges like writer’s block. Their capabilities extend beyond conventional applications, contributing to critical analysis, data augmentation, and research design, thereby elevating the efficiency and quality of scholarly endeavors. Strategically narrowing its focus, this review explores alternative dimensions of GPT and LLM applications, specifically data augmentation and the generation of synthetic data for research. Employing a meticulous examination of 412 scholarly works, it distills a selection of 77 contributions addressing three critical research questions: (1) GPT on Generating Research data, (2) GPT on Data Analysis, and (3) GPT on Research Design. The systematic literature review adeptly highlights the central focus on data augmentation, encapsulating 48 pertinent scholarly contributions, and extends to the proactive role of GPT in critical analysis of research data and shaping research design. Pioneering a comprehensive classification framework for “GPT’s use on Research Data”, the study classifies existing literature into six categories and 14 sub-categories, providing profound insights into the multifaceted applications of GPT in research data. This study meticulously compares 54 pieces of literature, evaluating research domains, methodologies, and advantages and disadvantages, providing scholars with profound insights crucial for the seamless integration of GPT across diverse phases of their scholarly pursuits.

The quality of the latent space in visual tokenizers (e.g., VAEs) is crucial for modern generative models. However, the standard reconstruction-based training paradigm produces a latent space that is biased towards low-level information, leading to a foundation flaw: better pixel-level accuracy does not lead to higher-quality generation. This implies that pouring extensive compute into visual tokenizer pre-training translates poorly to improved performance in generation. We identify this as the ``pre-training scaling problem`` and suggest a necessary shift: to be effective for generation, a latent space must concisely represent high-level semantics. We present VTP, a unified visual tokenizer pre-training framework, pioneering the joint optimization of image-text contrastive, self-supervised, and reconstruction losses. Our large-scale study reveals two principal findings: (1) understanding is a key driver of generation, and (2) much better scaling properties, where generative performance scales effectively with compute, parameters, and data allocated to the pretraining of the visual tokenizer. After large-scale pre-training, our tokenizer delivers a competitive profile (78.2 zero-shot accuracy and 0.36 rFID on ImageNet) and 4.1 times faster convergence on generation compared to advanced distillation methods. More importantly, it scales effectively: without modifying standard DiT training specs, solely investing more FLOPS in pretraining VTP achieves 65.8\% FID improvement in downstream generation, while conventional autoencoder stagnates very early at 1/10 FLOPS. Our pre-trained models are available at https://github.com/MiniMax-AI/VTP.

The impact of generative artificial intelligence (AI) on education and human development is currently unknown and may have substantial ethical implications in these fields. In this commentary, we discuss the nature of Generative Pre-trained Transformer (GPT) models that use deep learning to produce human-like text. This is an effort to understand how GPT models can support students, educators, and human development professionals to enhance learning while assisting in developing their professions. We conclude by outlining the need for policy decisions and ethical considerations on incorporating these technologies into educational and human development settings.

The ILC Technology Network (ITN) was established in 2022 by the ILC International Development Team, a subcommittee of the International Committee for Future Accelerators, to advance engineering studies toward the realisation of the International Linear Collider (ILC). While the ITN work packages focus on engineering activities for the ILC, their topics are also relevant to a broad range of accelerator applications in particle physics and beyond. These work packages are being carried out now by laboratories in Asia and Europe in close collaboration. This report summarises the current status of the ITN activities.

Purpose: The absence of digitisation in construction project management, particularly in areas such as dispute resolution, project scheduling, resource allocation, and risk management, remains a critical unresolved issue. This deficiency leads to workflow inefficiencies, increased project costs, and prolonged project timelines. To address these challenges, this study explores the integration of Generative Pre-trained Transformers, an AI technology, into the role of an artificially intelligent construction project manager. It develops a novel hybrid work model between a PrCPM and the CPM-AI model, leveraging digital innovation to specifically target and mitigate these inefficiencies. Theoretical Contributions: The Artificial Pedagogy Phenomenology Theory (APPt) is introduced to elucidate observed phenomena. The theory posits that leveraging both AI capabilities and human insights can optimise future hybrid work models, improving construction project outcomes. It encompasses and addresses ethical concerns associated with AI implementation, such as balancing accountability and job displacement, with an applied focus on Engineering and Built Environment disciplines. Methodology and Design: This study employs a positivist research philosophy and a quantitative deductive approach, utilising pedagogical techniques and secondary data to replicate the decision-making processes of construction project managers. At its core, the pedagogical training of the transformer algorithm uses queries to identify current project issues, keys to integrate relevant historical data for context from historic project training data, values to derive actionable insights from past scenarios, and the dimension of key vectors to ensure balanced and effective decision-making. Findings Based on Empirical Research: Pre-alpha unit testing of the CPM-AI model trained to address dispute resolution, project scheduling, resource allocation, and risk management demonstrated statistically significant results in performance validation testing, the model achieved an accuracy rate of over *88.6% (p = 0.035) (n = 150) in its performance test on decision-making tasks pertaining to dispute resolution. It showed (x) efficiency in programme scheduling, and resource allocation, with (x) results appearing *x% (p = x) (n = x) of the time in its response rate (testing is set to commence in June 2024). The model further demonstrated exemplary efficiency in its ability to answer standardised historical Project Management Professional Exam questions, achieving a *92.8% (p = 0.015) (n = 120) aggregate score across all examination questions tested in relation to risk management. These findings establish standards for developing bespoke BE-AI models in the built environment and demonstrate seamless integration with existing tools and practices. They highlight the potential for broader AI applications across the construction industry and pave the way for future research into other built environment disciplines. Research Limitations: Research limitations include reliance on secondary data quality and potential AI biases to the South African context, requiring ongoing refinement and validation to ensure model accuracy and applicability in diverse construction environments globally, however, the researchers propose regional versioning to mitigate this effect. Practical Implications: Practical implications include enhanced project management efficiency, decision accuracy, workflow efficiencies, reduced project costs, and streamlined project timelines. Value to the Conference: The research offers crucial insights into integrating AI in construction project management, in line with the conference's theme, “Disrupting Tradition.” It introduces an innovative AI application that can redefine built environment practices, making a significant contribution to the dialogue on technological advancements within the built environment profession.