The Consciousness Prior
Auteurs : Yoshua Bengio
Résumé scientifique
A new prior is proposed for representation learning, which can be combined with other priors in order to help disentangling abstract factors from each other. It is inspired by the phenomenon of consciousness seen as the formation of a low-dimensional combination of a few concepts constituting a conscious thought, i.e., consciousness as awareness at a particular time instant. This provides a powerful constraint on the representation in that such low-dimensional thought vectors can correspond to statements about reality which are true, highly probable, or very useful for taking decisions. The fact that a few elements of the current state can be combined into such a predictive or useful statement is a strong constraint and deviates considerably from the maximum likelihood approaches to modelling data and how states unfold in the future based on an agent's actions. Instead of making predictions in the sensory (e.g. pixel) space, the consciousness prior allows the agent to make predictions in the abstract space, with only a few dimensions of that space being involved in each of these predictions. The consciousness prior also makes it natural to map conscious states to natural language utterances or to express classical AI knowledge in the form of facts and rules, although the conscious states may be richer than what can be expressed easily in the form of a sentence, a fact or a rule.
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
Further N-Frame networking dynamics of conscious observer-self agents via a functional contextual interface: predictive coding, double-slit quantum mechanical experiment, and decision-making fallacy modeling as applied to the measurement problem in humans and AIDarren J. Edwards · 2025 · Frontiers Comput. Neurosci. · cité 2 fois · Pas encore résumé
Résumé scientifique
Artificial intelligence (AI) has made some remarkable advances in recent years, particularly within the area of large language models (LLMs) that produce human-like conversational abilities via utilizing transformer-based architecture. These advancements have sparked growing calls to develop tests not only for intelligence but also for consciousness. However, existing benchmarks assess reasoning abilities across various domains but fail to directly address consciousness. To bridge this gap, this paper introduces the functional contextual N-Frame model, a novel framework integrating predictive coding, quantum Bayesian (QBism), and evolutionary dynamics. This comprehensive model explicates how…
-
Artificial intelligence, human cognition, and conscious supremacyK. Mogi · 2024 · Frontiers in Psychology · cité 30 fois · Pas encore résumé
Résumé scientifique
The computational significance of consciousness is an important and potentially more tractable research theme than the hard problem of consciousness, as one could look at the correlation of consciousness and computational capacities through, e.g., algorithmic or complexity analyses. In the literature, consciousness is defined as what it is like to be an agent (i.e., a human or a bat), with phenomenal properties, such as qualia, intentionality, and self-awareness. The absence of these properties would be termed “unconscious.” The recent success of large language models (LLMs), such as ChatGPT, has raised new questions about the computational significance of human conscious processing. Althoug…
-
Neural Consciousness FlowXiaoran Xu, Wei Feng, Zhiqing Sun et al. · 2019 · arXiv.org · preprint, non relu par les pairs · cité 2 fois · Pas encore résumé
Résumé scientifique
The ability of reasoning beyond data fitting is substantial to deep learning systems in order to make a leap forward towards artificial general intelligence. A lot of efforts have been made to model neural-based reasoning as an iterative decision-making process based on recurrent networks and reinforcement learning. Instead, inspired by the consciousness prior proposed by Yoshua Bengio, we explore reasoning with the notion of attentive awareness from a cognitive perspective, and formulate it in the form of attentive message passing on graphs, called neural consciousness flow (NeuCFlow). Aiming to bridge the gap between deep learning systems and reasoning, we propose an attentive computation …
-
Consciousness is entailed by compositional learning of new causal structures in deep predictive processing systemsVladimir Aksyuk · 2023 · cité 1 fois · Pas encore résumé
Résumé scientifique
Machine learning algorithms have achieved superhuman performance in specific complex domains. However, learning online from few examples and compositional learning for efficient generalization across domains remain elusive. In humans, such learning includes specific declarative memory formation and is closely associated with consciousness. Predictive processing has been advanced as a principled Bayesian framework for understanding the cortex as implementing deep generative models for both sensory perception and action control. However, predictive processing offers little direct insight into fast compositional learning or of the separation between conscious and unconscious contents. Here, pro…
-
The Backend as a Possible Functional Analogue of Consciousness: Redirecting Attention from the Language Model to the Orchestrating LayerPavel Straňák · 2026 · Philosophies · Pas encore résumé
Résumé scientifique
Discussion of consciousness and artificial intelligence has hitherto focused on the question of whether a large language model (LLM) exhibits signs of consciousness or understanding. This paper proposes to redirect attention elsewhere: not to the model itself, but to the orchestrating layer that governs the model—the backend, understood here as the collection of mechanisms (context management, retrieval, evaluation, planning, and tool-use control) that structure the model’s operation. We argue that the backend performs a function functionally analogous to the role of consciousness in the human brain: it stabilizes generative processes, directs attention, maintains context, and mitigates the …
-
Awareness as inference in a higher-order state spaceS. Fleming · 2019 · Neuroscience of Consciousness · cité 104 fois · Pas encore résumé
Résumé scientifique
Abstract Humans have the ability to report the contents of their subjective experience—we can say to each other, ‘I am aware of X’. The decision processes that support these reports about mental contents remain poorly understood. In this article, I propose a computational framework that characterizes awareness reports as metacognitive decisions (inference) about a generative model of perceptual content. This account is motivated from the perspective of how flexible hierarchical state spaces are built during learning and decision-making. Internal states supporting awareness reports, unlike those covarying with perceptual contents, are simple and abstract, varying along a 1D continuum from abs…
-
An artificial consciousness model and its relations with philosophy of mindE. Garrido-Merchán, M. Molina, Francisco M. Mendoza · 2020 · arXiv.org · preprint, non relu par les pairs · cité 4 fois · Pas encore résumé
Résumé scientifique
This work seeks to study the beneficial properties that an autonomous agent can obtain by implementing a cognitive architecture similar to the one of conscious beings. Along this document, a conscious model of autonomous agent based in a global workspace architecture is presented. We describe how this agent is viewed from different perspectives of philosophy of mind, being inspired by their ideas. The goal of this model is to create autonomous agents able to navigate within an environment composed of multiple independent magnitudes, adapting to its surroundings in order to find the best possible position in base of its inner preferences. The purpose of the model is to test the effectiveness …
-
The cognitive reality monitoring network and theories of consciousness.Aurelio Cortese, M. Kawato · 2024 · Neurosciences research · cité 4 fois · Pas encore résumé
Résumé scientifique
Theories of consciousness abound. However, it is difficult to arbitrate reliably among competing theories because they target different levels of neural and cognitive processing or anatomical loci, and only some were developed with computational models in mind. In particular, theories of consciousness need to fully address the three levels of understanding of the brain proposed by David Marr: computational theory, algorithms and hardware. Most major theories refer to only one or two levels, often indirectly. The cognitive reality monitoring network (CRMN) model is derived from computational theories of mixture-of-experts architecture, hierarchical reinforcement learning and generative/infere…
Les travaux antérieurs sur lesquels il s’appuie
-
What is consciousness, and could machines have it?S. Dehaene, Ha-Kwan C. Lau, S. Kouider · 2017 · Science · cité 654 fois · Pas encore résumé
Résumé scientifique
The controversial question of whether machines may ever be conscious must be based on a careful consideration of how consciousness arises in the only physical system that undoubtedly possesses it: the human brain. We suggest that the word “consciousness” conflates two different types of information-processing computations in the brain: the selection of information for global broadcasting, thus making it flexibly available for computation and report (C1, consciousness in the first sense), and the self-monitoring of those computations, leading to a subjective sense of certainty or error (C2, consciousness in the second sense). We argue that despite their recent successes, current machines are …
-
Learning Deep Architectures for AIYoshua Bengio · 2007 · Found. Trends Mach. Learn. · cité 8 945 fois · Pas encore résumé
-
Memory NetworksJ. Weston, S. Chopra, Antoine Bordes · 2014 · International Conference on Learning Representations · cité 1 956 fois · Pas encore résumé
Résumé scientifique
Abstract: We describe a new class of learning models called memory networks. Memory networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in the context of question answering (QA) where the long-term memory effectively acts as a (dynamic) knowledge base, and the output is a textual response. We evaluate them on a large-scale QA task, and a smaller, but more complex, toy task generated from a simulated world. In the latter, we show the reasoning power of such models by chaining multiple supporting sentences …
-
Show, Attend and Tell: Neural Image Caption Generation with Visual AttentionKe Xu, Jimmy Ba, R. Kiros et al. · 2015 · International Conference on Machine Learning · cité 11 166 fois · Pas encore résumé
Résumé scientifique
Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. We describe how we can train this model in a deterministic manner using standard backpropagation techniques and stochastically by maximizing a variational lower bound. We also show through visualization how the model is able to automatically learn to fix its gaze on salient objects while generating the corresponding words in the output sequence. We validate the use of attention with state-of-the-art performance on three benchmark datasets: Flickr9k, Flickr30k and MS COCO.
-
Auto-Encoding Variational BayesDiederik P. Kingma, M. Welling · 2013 · International Conference on Learning Representations · cité 18 688 fois · Pas encore résumé
Résumé scientifique
Abstract: How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case. Our contributions is two-fold. First, we show that a reparameterization of the variational lower bound yields a lower bound estimator that can be straightforwardly optimized using standard stochastic gradient methods. Second, we show that for i.i.d. datasets with continuous latent variables per datapoint, …
Les articles qui le citent
-
A Consciousness-Inspired Planning Agent for Model-Based Reinforcement LearningMingde Zhao, Zhen Liu, Si-Tao Luan et al. · 2021 · Neural Information Processing Systems · cité 41 fois · Pas encore résumé
Résumé scientifique
We present an end-to-end, model-based deep reinforcement learning agent which dynamically attends to relevant parts of its state during planning. The agent uses a bottleneck mechanism over a set-based representation to force the number of entities to which the agent attends at each planning step to be small. In experiments, we investigate the bottleneck mechanism with several sets of customized environments featuring different challenges. We consistently observe that the design allows the planning agents to generalize their learned task-solving abilities in compatible unseen environments by attending to the relevant objects, leading to better out-of-distribution generalization performance.
-
Causal representation learning through higher-level information extractionFrancisco Silva, Hélder P. Oliveira, Tânia Pereira · 2024 · ACM Computing Surveys · cité 7 fois · Pas encore résumé
Résumé scientifique
The large gap between the generalization level of state-of-the-art machine learning and human learning systems calls for the development of artificial intelligence (AI) models that are truly inspired by human cognition. In tasks related to image analysis, searching for pixel-level regularities has reached a power of information extraction still far from what humans capture with image-based observations. This leads to poor generalization when even small shifts occur at the level of the observations. We explore a perspective on this problem that is directed to learning the generative process with causality-related foundations, using models capable of combining symbolic manipulation, probabilis…
-
All in Good Time: Long-Lasting Postdictive Effects Reveal Discrete Perception.M. Herzog, Leila Drissi-Daoudi, Adrien Doerig · 2020 · Trends in Cognitive Sciences · cité 121 fois · Pas encore résumé
Résumé scientifique
Is consciousness a continuous stream of percepts or is it discrete, occurring only at certain moments in time? This question has puzzled philosophers, psychologists, and neuroscientists for centuries. Both hypotheses have fallen repeatedly in and out of favor. Here, we review recent studies exploring long-lasting postdictive effects and show that the results favor a two-stage discrete model, in which substantial periods of continuous unconscious processing precede discrete conscious percepts. We propose that such a model marries the advantages of both continuous and discrete models and resolves centuries old debates about perception and consciousness.
-
Conscious Empathic AI in ServiceH. Esmaeilzadeh, R. Vaezi · 2022 · Journal of services research · cité 49 fois · Pas encore résumé
Résumé scientifique
Recent advances in artificial intelligence (AI) have achieved human-scale speed and accuracy for classification tasks. Current systems do not need to be conscious to recognize patterns and classify them. However, for AI to advance to the next level, it needs to develop capabilities such as metathinking, creativity, and empathy. We contend that such a paradigm shift is possible through a fundamental change in the state of artificial intelligence toward consciousness, similar to what took place for humans through the process of natural selection and evolution. To that end, we propose that consciousness in AI is an emergent phenomenon that primordially appears when two machines cocreate their o…
-
Discrete and continuous representations and processing in deep learning: Looking forwardRuben Cartuyvels, Graham Spinks, Marie-Francine Moens · 2022 · AI Open · cité 34 fois · Pas encore résumé
Résumé scientifique
Abstract Discrete and continuous representations of content (e.g., of language or images) have interesting properties to be explored for the understanding of or reasoning with this content by machines. This position paper puts forward our opinion on the role of discrete and continuous representations and their processing in the deep learning field. Current neural network models compute continuous-valued data. Information is compressed into dense, distributed embeddings. By stark contrast, humans use discrete symbols in their communication with language. Such symbols represent a compressed version of the world that derives its meaning from shared contextual information. Additionally, human re…
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 3 novembre 2026.