Search Results for « Learning rates »

The accurate determination of the position-dependent energy fluence rate of scattered light (which is proportional to the energy density) is crucial to the understanding of transport in anisotropically scattering and absorbing samples, such as biological tissue, seawater, atmospheric turbulent layers, and light-emitting diodes. While Monte Carlo simulations are precise, their long computation time is not desirable. Common analytical approximations to the radiative transfer equation (RTE) fail to predict light transport and could even give unphysical results. Therefore, we experimentally probe the position-dependent energy fluence rate of light inside scattering samples where the widely used P1 and P3 approximations to the RTE fail. The samples are three-dimensional (3D) aqueous suspensions of anisotropically scattering and both absorbing and non-absorbing spherical scatterers, namely, microspheres (r = 0.5 um) with and without absorbing dye. To probe the energy fluence rate, we detect the emission of quantum-dot reporter particles that are excited by the incident light and that are contained in a thin capillary. By scanning the capillary through the sample, we access the position dependence. We present a comprehensive discussion of experimental limitations and of both random and systematic errors. Our observations agree well with the Monte Carlo simulations and the P3 approximation of the RTE with a correction for forward scattering. In contrast, the P1 and the P3 approximations deviate increasingly from our observations, ultimately even predicting unphysical negative energies.

Computational phenotyping is a powerful approach for characterizing individual variability across cognitive domains, with significant applications in computational psychiatry. Within the reinforcement learning framework, the learning rate is widely used to classify individuals based on learning patterns and deviations from optimal behavior. However, its interpretation depends on psychometric properties, which are rarely examined. To assess construct validity, we collected single-trial learning data (N = 126) from an uncertainty-augmented reward learning task to obtain participants' performance and learning rates. We additionally collected psychometric data across a battery of executive functioning tasks to obtain measures of cognitive constructs with high relevance to decision-making and reward learning. Individual differences in learning rates were not systematically explained by cognitive abilities, suggesting limited construct validity, despite moderate validity in learning performance. We identify low-reliability scores and lack of generalizability for the learning rate as a possible explanation for its low validity. These findings highlight the need to improve the psychometric properties of learning rates for studying individual differences, despite their success in capturing group-level learning behavior.

The increasing complexity of deep learning architectures is resulting in training time requiring weeks or even months. This slow training is due in part to vanishing gradients, in which the gradients used by back-propagation are extremely large for weights connecting deep layers (layers near the output layer), and extremely small for shallow layers (near the input layer); this results in slow learning in the shallow layers. Additionally, it has also been shown that in highly non-convex problems, such as deep neural networks, there is a proliferation of high-error low curvature saddle points, which slows down learning dramatically. In this paper, we attempt to overcome the two above problems by proposing an optimization method for training deep neural networks which uses learning rates which are both specific to each layer in the network and adaptive to the curvature of the function, increasing the learning rate at low curvature points. This enables us to speed up learning in the shallow layers of the network and quickly escape high-error low curvature saddle points. We test our method on standard image classification datasets such as MNIST, CIFAR10 and ImageNet, and demonstrate that our method increases accuracy as well as reduces the required training time over standard algorithms.

People often have to switch back and forth between different environments that come with different problems and volatilities. While volatile environments require fast learning (i.e., high learning rates), stable environments call for lower learning rates. Previous studies have shown that people adapt their learning rates, but it remains unclear whether they can also learn about environment-specific learning rates, and instantaneously retrieve them when revisiting environments. Here, using optimality simulations and hierarchical Bayesian analyses across three experiments, we show that people can learn to use different learning rates when switching back and forth between two different environments. We even observe a signature of these environment-specific learning rates when the volatility of both environments is suddenly the same. We conclude that humans can flexibly adapt and learn to associate different learning rates to different environments, offering important insights for developing theories of meta-learning and context-specific control.

Online active learning is a paradigm in machine learning that aims to select the most informative data points to label from a data stream. The problem of minimizing the cost associated with collecting labeled observations has gained a lot of attention in recent years, particularly in real-world applications where data is only available in an unlabeled form. Annotating each observation can be time-consuming and costly, making it difficult to obtain large amounts of labeled data. To overcome this issue, many active learning strategies have been proposed in the last decades, aiming to select the most informative observations for labeling in order to improve the performance of machine learning models. These approaches can be broadly divided into two categories: static pool-based and stream-based active learning. Pool-based active learning involves selecting a subset of observations from a closed pool of unlabeled data, and it has been the focus of many surveys and literature reviews. However, the growing availability of data streams has led to an increase in the number of approaches that focus on online active learning, which involves continuously selecting and labeling observations as they arrive in a stream. This work aims to provide an overview of the most recently proposed approaches for selecting the most informative observations from data streams in real time. We review the various techniques that have been proposed and discuss their strengths and limitations, as well as the challenges and opportunities that exist in this area of research.

Research on individual decision-making often finds a positivity bias, where people weight positive outcomes more strongly than negative ones during learning. This can be beneficial when rewards are rare, by amplifying relative value differences. Yet, we know very little about learning rate biases in social settings, where a key advantage is being able to vicariously learn from the negative experiences of others. This would imply a benefit for focusing on negative outcomes when learning socially, but is at odds with the seemingly inflexible positivity bias found in individual learning. Here, we examine learning rate biases across both individual and social settings, testing for adaptivity versus generally stable biases. Overall, participants appear more flexible in their learning rate biases when learning socially than when learning individually. This implies that human social learning may be more flexible and closer to normatively optimal behavior than individual learning.

Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks. The MAML algorithm performs well on few-shot learning problems in classification, regression, and fine-tuning of policy gradients in reinforcement learning, but comes with the need for costly hyperparameter tuning for training stability. We address this shortcoming by introducing an extension to MAML, called Alpha MAML, to incorporate an online hyperparameter adaptation scheme that eliminates the need to tune meta-learning and learning rates. Our results with the Omniglot database demonstrate a substantial reduction in the need to tune MAML training hyperparameters and improvement to training stability with less sensitivity to hyperparameter choice.

Research on individual decision-making often finds a positivity bias, where people weight positive outcomes more strongly than negative ones during learning. This can be beneficial when rewards are rare, by amplifying relative value differences. Yet, we know very little about learning rate biases in social settings, where a key advantage is being able to vicariously learn from the negative experiences of others. This would imply a benefit for focusing on negative outcomes when learning socially, but is at odds with the seemingly inflexible positivity bias found in individual learning. Here, we examine learning rate biases across both individual and social settings, testing for adaptivity versus generally stable biases. Overall, participants appear more flexible in their learning rate biases when learning socially than when learning individually. This implies that human social learning may be more flexible and closer to normatively optimal behavior than individual learning.

Contrastive learning operates on a simple yet effective principle: Embeddings of positive pairs are pulled together, while those of negative pairs are pushed apart. In this paper, we propose a unified framework for understanding contrastive learning through the lens of cosine similarity, and present two key theoretical insights derived from this framework. First, in full-batch settings, we show that perfect alignment of positive pairs is unattainable when negative-pair similarities fall below a threshold, and this misalignment can be mitigated by incorporating within-view negative pairs into the objective. Second, in mini-batch settings, smaller batch sizes induce stronger separation among negative pairs in the embedding space, i.e., higher variance in their similarities, which in turn degrades the quality of learned representations compared to full-batch settings. To address this, we propose an auxiliary loss that reduces the variance of negative-pair similarities in mini-batch settings. Empirical results show that incorporating the proposed loss improves performance in small-batch settings.

Research on individual decision-making often finds a positivity bias, where people weight positive outcomes more strongly than negative ones during learning. This can be beneficial when rewards are rare, by amplifying relative value differences. Yet, we know very little about learning rate biases in social settings, where a key advantage is being able to vicariously learn from the negative experiences of others. This would imply a benefit for focusing on negative outcomes when learning socially, but is at odds with the seemingly inflexible positivity bias found in individual learning. Here, we examine learning rate biases across both individual and social settings, testing for adaptivity versus generally stable biases. Overall, participants appear more flexible in their learning rate biases when learning socially than when learning individually. This implies that human social learning may be more flexible and closer to normatively optimal behavior than individual learning.