Language Models Trained on Media Diets Can Predict Public Opinion

Auteurs : Eric Chu, Jacob Andreas, Stephen Ansolabehere, Deb Roy

Soumis à arXiv le 28 Mar. 2023
Résumé scientifique

Public opinion reflects and shapes societal behavior, but the traditional survey-based tools to measure it are limited. We introduce a novel approach to probe media diet models -- language models adapted to online news, TV broadcast, or radio show content -- that can emulate the opinions of subpopulations that have consumed a set of media. To validate this method, we use as ground truth the opinions expressed in U.S. nationally representative surveys on COVID-19 and consumer confidence. Our studies indicate that this approach is (1) predictive of human judgements found in survey response distributions and robust to phrasing and channels of media exposure, (2) more accurate at modeling people who follow media more closely, and (3) aligned with literature on which types of opinions are affected by media consumption. Probing language models provides a powerful new method for investigating media effects, has practical applications in supplementing polls and forecasting public opinion, and suggests a need for further study of the surprising fidelity with which neural language models can predict human responses.

Les articles autour de lui, par date et par proximité

Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article

Comment lire cette carte ? Guide illustré

Les articles les plus proches Les travaux antérieurs sur lesquels il s’appuie Les articles qui le citent

Les articles autour de celui-ci

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

  1. Proche
    Public Opinion Mining Using Large Language Models on COVID-19 Related Tweets
    Vu Tran, Tomoko Matsui · 2023 · International Conference on Knowledge and Systems Engineering · cité 1 fois · Pas encore résumé
    Résumé scientifique

    Social media platforms have emerged as a significant source of public opinion, offering a massive user-generated data in which user-opinions are valuable if obtainable. Large language models (LLMs) have been in the spotlight recently with suggestions on the emergent abilities to solve tasks that are not explicitly trained for. Thus, this study explores the potential of utilizing LLMs for opinion mining on social media data by asking LLMs difficult questions, instead of simply asking whether the text's sentiment polarity is either positive, negative, or neutral. This study compares the LLM response statistics and the corresponding public surveys related to COVID-19, including the intention to…

  2. Proche
    MindVote: When AI Meets the Wild West of Social Media Opinion
    Xu-Tao Mao, Ezra Xuanru Tao, Le-Yao Wang · 2025 · AAAI Conference on Artificial Intelligence · Pas encore résumé
    Résumé scientifique

    Large Language Models (LLMs) are increasingly used as scalable tools for pilot testing, predicting public opinion distributions before deploying costly surveys. However, the prevailing paradigm for evaluating these models relies on traditional structured surveys—a methodology misaligned with the more realistic scenarios like social media where opinions are rich in digital contexts. By design, surveys strip away the social and cultural context that shapes public opinion, and LLM benchmarks built on this paradigm inherit these critical limitations. To bridge this gap, we introduce MindVote, the first benchmark for public opinion prediction grounded in authentic social media discourse. MindVote…

  3. Proche Même question
    Whose Opinions Do Language Models Reflect?
    Shibani Santurkar, Esin Durmus, Faisal Ladhak et al. · 2023 · Computation and Language · cité 1 170 fois · Résumé IA disponible

    Whose Opinions Do Language Models Reflect? compare les opinions des modèles aux réponses de groupes démographiques américains dans des sondages, comme l’étude des opinions publiques par modèles médiatiques.

    Résumé scientifique

    Language models (LMs) are increasingly being used in open-ended contexts, where the opinions reflected by LMs in response to subjective queries can have a profound impact, both on user satisfaction, as well as shaping the views of society at large. In this work, we put forth a quantitative framework to investigate the opinions reflected by LMs -- by leveraging high-quality public opinion polls and their associated human responses. Using this framework, we create OpinionsQA, a new dataset for evaluating the alignment of LM opinions with those of 60 US demographic groups over topics ranging from abortion to automation. Across topics, we find substantial misalignment between the views reflected…

  4. Proche
    Comparing the algorithmic fidelity of large language models in predicting human decision making: a case study of vaccination choice
    Shao-Chong Xu, Li-Yue Zhang, Amelia M. Jamison et al. · 2026 · npj Digital Public Health · cité 1 fois · Pas encore résumé
    Résumé scientifique

    This study explores the algorithmic fidelity of five well-established LLM architectures when attempting to replicate complex health decisions like vaccination. The analysis leverages individual survey responses and real-world online news content within prescribed model designs that vary in their relative inputs—namely, basic demographic information, revealed attitudes based on survey data, and personalized media diets. We evaluate the predictive accuracy and sensitivity for each model across all five LLMs, and reveal differing biases inherent to each LLM design. We further explore the role of media exposure on predicted vaccination choice within a counterfactual scenario analysis, by varying…

  5. Proche
    Simulating Public Opinion: Comparing Distributional and Individual-Level Predictions from LLMs and Random Forests
    Fernando Miranda, P. Balbi · 2025 · Entropy · cité 6 fois · Pas encore résumé
    Résumé scientifique

    Understanding and modeling the flow of information in human societies is essential for capturing phenomena such as polarization, opinion formation, and misinformation diffusion. Traditional agent-based models often rely on simplified behavioral rules that fail to capture the nuanced and context-sensitive nature of human decision-making. In this study, we explore the potential of Large Language Models (LLMs) as data-driven, high-fidelity agents capable of simulating individual opinions under varying informational conditions. Conditioning LLMs on real survey data from the 2020 American National Election Studies (ANES), we investigate their ability to predict individual-level responses across a…

  6. Proche Autre approche du problème
    AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction
    Junsol Kim, Byungkyu Lee · 2023 · Computation and Language · cité 85 fois · Résumé IA disponible

    AI-Augmented Surveys prédit des réponses individuelles inédites en adaptant des modèles aux données d’enquêtes, plutôt qu’à des contenus médiatiques.

    Résumé scientifique

    Large language models (LLMs) that produce human-like responses have begun to revolutionize research practices in the social sciences. This paper shows how we can integrate LLMs and social surveys to accurately predict individual responses to survey questions that were not asked before. We develop a novel methodological framework to personalize LLMs by considering the meaning of survey questions derived from their text, the latent beliefs of individuals inferred from their response patterns, and the temporal contexts across different survey periods through fine-tuning LLMs with survey data. Using the General Social Survey from 1972 to 2021, we show that the fine-tuned model based on Alpaca-7b…

  7. Proche
    Simulating Human Opinions with Large Language Models: Opportunities and Challenges for Personalized Survey Data Modeling
    Carolin Kaiser, Jakob Kaiser, Vladimir Manewitsch et al. · 2025 · User Modeling, Adaptation, and Personalization · cité 34 fois · Pas encore résumé
    Résumé scientifique

    Public and private organizations rely on opinion surveys to inform business and policy decisions. Yet, empirical surveys are costly and time-consuming. Recent advances in large language models (LLMs) have sparked interest in generating synthetic survey data, i.e., simulated answers based on target demographics, as an alternative to real human data. But how well can LLMs replicate human opinions? In this ongoing project, we develop and critically evaluate methods for synthetic survey sampling. As an empirical benchmark, we collected responses from a representative U.S. sample (n = 461) on preferences for a common consumer good (soft drinks). Then, we developed ASPIRE (Automated Synthetic Pers…

  8. Proche
    Designing Domain-Specific Large Language Models: The Critical Role of Fine-Tuning in Public Opinion Simulation
    Hao Lin · 2024 · arXiv.org · preprint, non relu par les pairs · cité 7 fois · Pas encore résumé
    Résumé scientifique

    Large language models (LLMs) have transformed natural language processing, yet face challenges in specialized tasks such as simulating opinions on environmental policies. This paper introduces a novel fine-tuning approach that integrates socio-demographic data from the UK Household Longitudinal Study, uniquely using profiling factors, such as age, gender, income, education, and region. This method enhances the accuracy and representation of generated views. By emulating diverse synthetic profiles, the fine-tuned models significantly outperform pre-trained counterparts, achieving measurable improvements in capturing demographic nuances. Evaluation metrics, including Chi-Squared, Cosine Simila…

Les travaux antérieurs sur lesquels il s’appuie

  1. Proche
    From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series
    Brendan T. O'Connor, R. Balasubramanyan, Bryan R. Routledge et al. · 2010 · International Conference on Web and Social Media · cité 2 184 fois · Pas encore résumé
    Résumé scientifique

    We connect measures of public opinion measured from polls with sentiment measured from text. We analyze several surveys on consumer confidence and political opinion over the 2008 to 2009 period, and find they correlate to sentiment word frequencies in contempora- neous Twitter messages. While our results vary across datasets, in several cases the correlations are as high as 80%, and capture important large-scale trends. The re- sults highlight the potential of text streams as a substi- tute and supplement for traditional polling. consumer confidence and political opinion, and can also pre- dict future movements in the polls. We find that temporal smoothing is a critically important issue to supp…

  2. Apparenté
    Media Effects Across Time and Subject: How News Coverage Affects Two Out of Four Attributes of Consumer Confidence
    M. Boukes, A. Damstra, R. Vliegenthart · 2019 · Communication Research · cité 46 fois · Pas encore résumé
    Résumé scientifique

    Previous studies using aggregate-level designs demonstrated that the tone of economic news affects consumer confidence. However, the individual-level mechanisms underlying this effect remain to be investigated: It is not clear which consumer confidence attributes are most susceptible to media effects. Theoretically, we integrate the economic voting literature and extend media system dependency by differentiating between the media effects on sociotropic versus egocentric evaluations and the effects on prospective versus retrospective economic evaluations. Methodologically, data from a manual content analysis are linked to data from a three-wave panel survey, containing repeated measurements o…

  3. Apparenté
    How the news media activate public expression and influence national agendas
    Gary King, Benjamin Schneer, Ariel White · 2017 · Science · cité 303 fois · Pas encore résumé
    Résumé scientifique

    Measuring the impact of the media The active participation of the people is one of the central components of a functioning democracy. King et al. performed a real-world randomized experiment in the United States to understand the causal effect of news stories on increasing public discussion of a specific topic (see the Policy Forum by Gentzkow). Social media posts increased by almost 20% the first day after the publication of news stories on a wide range of topics. Furthermore, the posts were relatively evenly distributed across political affiliation, gender, and region of the United States. Science, this issue p. 776; see also p. 726 Quantifying the effects of the press on public discourse …

  4. Apparenté*
    Measuring Public Opinion with Surveys
    Adam J. Berinsky · 2017 · Annual Review of Political Science · cité 124 fois · Pas encore résumé
  5. Apparenté
    Ethical and social risks of harm from Language Models
    Laura Weidinger, John F. J. Mellor, Maribeth Rauh et al. · 2021 · arXiv.org · preprint, non relu par les pairs · cité 1 735 fois · Résumé IA disponible
    Résumé scientifique

    This paper aims to help structure the risk landscape associated with large-scale Language Models (LMs). In order to foster advances in responsible innovation, an in-depth understanding of the potential risks posed by these models is needed. A wide range of established and anticipated risks are analysed in detail, drawing on multidisciplinary expertise and literature from computer science, linguistics, and social sciences. We outline six specific risk areas: I. Discrimination, Exclusion and Toxicity, II. Information Hazards, III. Misinformation Harms, V. Malicious Uses, V. Human-Computer Interaction Harms, VI. Automation, Access, and Environmental Harms. The first area concerns the perpetuati…

Les articles qui le citent

  1. Proche
    Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Information
    Seungjong Sun, Eungu Lee, Dongyan Nan et al. · 2024 · arXiv.org · preprint, non relu par les pairs · cité 71 fois · Pas encore résumé
    Résumé scientifique

    Large language models exhibit societal biases associated with demographic information, including race, gender, and others. Endowing such language models with personalities based on demographic data can enable generating opinions that align with those of humans. Building on this idea, we propose"random silicon sampling,"a method to emulate the opinions of the human population sub-group. Our study analyzed 1) a language model that generates the survey responses that correspond with a human group based solely on its demographic distribution and 2) the applicability of our methodology across various demographic subgroups and thematic questions. Through random silicon sampling and using only grou…

  2. Proche
    Vox Populi, Vox AI? Using Large Language Models to Estimate German Vote Choice
    Leah von der Heyde, Anna Haensch, Alexander Wenz · 2024 · Social science computer review · cité 28 fois · Pas encore résumé
    Résumé scientifique

    “Synthetic samples” generated by large language models (LLMs) have been argued to complement or replace traditional surveys, assuming their training data is grounded in human-generated data that potentially reflects attitudes and behaviors prevalent in the population. Initial US-based studies that have prompted LLMs to mimic survey respondents found that the responses match survey data. However, the relationship between the respective target population and LLM training data might affect the generalizability of such findings. In this paper, we critically evaluate the use of LLMs for public opinion research in a different context, by investigating whether LLMs can estimate vote choice in Germa…

  3. Proche
    Aligning Language Models to User Opinions
    Eunjeong Hwang, Bodhisattwa Prasad Majumder, Niket Tandon · 2023 · Conference on Empirical Methods in Natural Language Processing · cité 151 fois · Pas encore résumé
    Résumé scientifique

    An important aspect of developing LLMs that interact with humans is to align models' behavior to their users. It is possible to prompt an LLM into behaving as a certain persona, especially a user group or ideological persona the model captured during its pertaining stage. But, how to best align an LLM with a specific user and not a demographic or ideological group remains an open question. Mining public opinion surveys (by Pew Research), we find that the opinions of a user and their demographics and ideologies are not mutual predictors. We use this insight to align LLMs by modeling both user opinions as well as user demographics and ideology, achieving up to 7 points accuracy gains in predic…

  4. Apparenté
    Can Large Language Models Transform Computational Social Science?
    Caleb Ziems, William Held, Omar Shaikh et al. · 2023 · Computation and Language · cité 692 fois · Pas encore résumé
    Résumé scientifique

    Large Language Models (LLMs) like ChatGPT are capable of successfully performing many language processing tasks zero-shot (without the need for training data). If this capacity also applies to the coding of social phenomena like persuasiveness and political ideology, then LLMs could effectively transform Computational Social Science (CSS). This work provides a road map for using LLMs as CSS tools. Towards this end, we contribute a set of prompting best practices and an extensive evaluation pipeline to measure the zero-shot performance of 13 language models on 24 representative CSS benchmarks. On taxonomic labeling tasks (classification), LLMs fail to outperform the best fine-tuned models but…

  5. Apparenté
    Community-Cross-Instruct: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities
    Zihao He, Rebecca Dorn, Si-Yi Guo et al. · 2024 · Conference on Empirical Methods in Natural Language Processing · cité 17 fois · Pas encore résumé
    Résumé scientifique

    Social scientists use surveys to probe the opinions and beliefs of populations, but these methods are slow, costly, and prone to biases. Recent advances in large language models (LLMs) enable the creating of computational representations or “digital twins” of populations that generate human-like responses mimicking the population’s language, styles, and attitudes. We introduce Community-Cross-Instruct, an unsupervised framework for aligning LLMs to online communities to elicit their beliefs. Given a corpus of a community’s online discussions, Community-Cross-Instruct automatically generates instruction-output pairs by an advanced LLM to (1) finetune a foundational LLM to faithfully represent…

Comment ces articles sont-ils trouvés ?

Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.

Mis à jour le 6 novembre 2026.

Le guide illustré des articles autour d’un article