AI summary · arXiv: 2102.12141v1 · cs.RO
Learning to Shift Attention for Motion Generation
The essentials
- Generating motion trajectories accurately is a major challenge in robot learning from demonstration
- Prior approaches have failed to capture all modes or averaged them out, resulting in invalid trajectories
- Limited number of demonstrations available may not cover the entire working space
- Researchers propose a novel motion generation model with extrapolation ability that can shift attention between local frames during task execution
- The proposed model uses real-valued non-volume preserving (RealNVP) transformations to learn local latent representations of motion trajectories to address the problem of multiple modes
- The density estimation method provides powerful and stably invertible transformations that can capture all modes present in human demonstrations
- To improve extrapolation ability, the model shifts attention between different local frames during task execution
- Experiments on the docking problem showed that their proposed method outperformed other approaches and has potential applications in various domains such as robotics and automation.
Key points written by AI: they may contain errors, check them against the paper.
Ask questions about this paper to our AI assistant
How to chat with this paper? Illustrated guide
You can also chat with multiple papers at once here.
Keep this conversation — a free account saves your questions and answers, your collections of papers and your alerts.
Continue with Google Create a free accountTools for this paper
Advanced tools Premium
For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium
Papers around this one
See them on a mapRanked by how close their title and abstract are to this paper’s. Select some to read them together.
Closest papers
-
Guided Decoding for Robot On-line Motion Generation and AdaptionNutan Chen, Botond Cseke, Elie Aljalbout et al. · 2024 · IEEE-RAS International Conference on Humanoid Robots · cited 1 time · Not summarized yet
Abstract
We present a novel motion generation approach for robot arms, with high degrees of freedom, in complex settings that can adapt online to obstacles or new via points. Learning from Demonstration facilitates rapid adaptation to new tasks and optimizes the utilization of accumulated expertise by allowing robots to learn and generalize from demonstrated trajectories. We train a transformer architecture, based on conditional variational autoencoder, on a large dataset of simulated trajectories used as demonstrations. Our architecture learns essential motion generation skills from these demonstrations and is able to adapt them to meet auxiliary tasks. Additionally, our approach implements auto-reg…
-
Generalizing Robot Trajectories from Single-Context Human Demonstrations: A Probabilistic ApproachQian-Ying Lee, Suhas Raghavendra Kulkarni, Kenzhi Iskandar Wong et al. · 2025 · International Conference on Robotics, Control and Automation · cited 2 times · Not summarized yet
Abstract
Generalizing robot trajectories from human demonstrations to new contexts remains a key challenge in Learning from Demonstration (LfD), particularly when only low single-context demonstrations data are available. We present a novel Gaussian Mixture Model (GMM)-based approach that enables systematic generalization from single-context demonstrations to a wide range of unseen start and goal configurations. Our method performs component-level reparameterization of the GMM, adapting both mean vectors and covariance matrices, followed by Gaussian Mixture Regression (GMR) to generate smooth trajectories. We evaluate the approach on a dual-arm pick-and-place task with varying box placements, compari…
-
MKT-GMM: A Motion Knowledge Transferring Framework for Robot Trajectory Adaptation to Variable Via-PointsCongcong Ye, Chengxing Wu, M. Luo et al. · 2026 · Biomimetics · Not summarized yet
Abstract
Human motion provides a valuable source of information for robotic skill acquisition, and Learning from Demonstration (LfD) has been widely adopted as an intuitive paradigm for enabling robots to learn tasks from human demonstrations. However, the lack of an explicit representation of transferable motion knowledge significantly limits the adaptability of LfD when tasks involve varying spatial constraints or environmental configurations. To address this challenge, this paper proposes a motion representation framework based on two fundamental properties of motion and introduces a novel Motion Knowledge Transferring Gaussian Mixture Model (MKT-GMM) for trajectory generalization across related t…
-
Reactive Motion Generation on Learned Riemannian ManifoldsHadi Beik-Mohammadi, Søren Hauberg, Georgios Arvanitidis et al. · 2022 · Robotics · cited 21 times · AI summary available
Abstract
In recent decades, advancements in motion learning have enabled robots to acquire new skills and adapt to unseen conditions in both structured and unstructured environments. In practice, motion learning methods capture relevant patterns and adjust them to new conditions such as dynamic obstacle avoidance or variable targets. In this paper, we investigate the robot motion learning paradigm from a Riemannian manifold perspective. We argue that Riemannian manifolds may be learned via human demonstrations in which geodesics are natural motion skills. The geodesics are generated using a learned Riemannian metric produced by our novel variational autoencoder (VAE), which is especially intended to …
-
Constraint-Preserving Data Generation for Visuomotor Policy LearningKevin Lin, Varun Ragunath, Andrew McAlinden et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 12 times · Not summarized yet
Abstract
Large-scale demonstration data has powered key breakthroughs in robot manipulation, but collecting that data remains costly and time-consuming. We present Constraint-Preserving Data Generation (CP-Gen), a method that uses a single expert trajectory to generate robot demonstrations containing novel object geometries and poses. These generated demonstrations are used to train closed-loop visuomotor policies that transfer zero-shot to the real world and generalize across variations in object geometries and poses. Similar to prior work using pose variations for data generation, CP-Gen first decomposes expert demonstrations into free-space motions and robot skills. But unlike those works, we achi…
-
Learning from Few Demonstrations with Frame-Weighted Motion GenerationJianyong Sun, Ji-Hong Zhu, Jens Kober et al. · 2023 · International Symposium on Experimental Robotics · cited 4 times · Not summarized yet
Abstract
Learning from Demonstration (LfD) enables robots to acquire versatile skills by learning motion policies from human demonstrations. It endows users with an intuitive interface to transfer new skills to robots without the need for time-consuming robot programming and inefficient solution exploration. During task executions, the robot motion is usually influenced by constraints imposed by environments. In light of this, task-parameterized LfD (TP-LfD) encodes relevant contextual information into reference frames, enabling better skill generalization to new situations. However, most TP-LfD algorithms typically require multiple demonstrations across various environmental conditions to ensure suf…
-
Improving Task-Parameterised Movement Learning Generalisation with Frame-Weighted Trajectory GenerationAran Sena, B. Michael, M. Howard · 2019 · IEEE/RJS International Conference on Intelligent RObots and Systems · cited 22 times · Not summarized yet
Abstract
Learning from Demonstration depends on a robot learner generalising its learned model to unseen conditions. Generally, it is not feasible for a person to provide demonstrations that account for all possible variations in non-trivial tasks. While there are many learning methods that can handle interpolation of observed data effectively, extrapolation from observed data offers a much greater challenge. To address this problem, this paper proposes a modified Task-Parameterised Gaussian Mixture Regression method that considers the relevance of task parameters during trajectory generation, as determined by variance in the data. The benefits of the proposed method are first explored using a simula…
-
DiffGen: Robot Demonstration Generation via Differentiable Physics Simulation, Differentiable Rendering, and Vision-Language ModelYang Jin, Jun Lv, Shuqiang Jiang et al. · 2024 · Robotics · cited 5 times · AI summary available
Abstract
Generating robot demonstrations through simulation is widely recognized as an effective way to scale up robot data. Previous work often trained reinforcement learning agents to generate expert policies, but this approach lacks sample efficiency. Recently, a line of work has attempted to generate robot demonstrations via differentiable simulation, which is promising but heavily relies on reward design, a labor-intensive process. In this paper, we propose DiffGen, a novel framework that integrates differentiable physics simulation, differentiable rendering, and a vision-language model to enable automatic and efficient generation of robot demonstrations. Given a simulated robot manipulation sce…
Prior work it builds on
-
Movement Primitive Learning and Generalization: Using Mixture Density NetworksYou Zhou, Jianfeng Gao, T. Asfour · 2020 · IEEE robotics & automation magazine · cited 37 times · Not summarized yet
Abstract
Representing robot skills as movement primitives (MPs) that can be learned from human demonstration and adapted to new tasks and situations is a promising approach toward intuitive robot pro gramming. To allow such adaptation, mapping between task parameters and MP parameters is needed, and different approaches have been proposed in the literature to learn such mapping. In human demonstrations, however, multiple modes and models exist, and these should be taken into account when learning these mappings and generalized MP representations.
-
On improving the extrapolation capability of task-parameterized movement modelsS. Calinon, Tohid Alizadeh, D. Caldwell · 2013 · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · cited 106 times · Not summarized yet
-
A tutorial on task-parameterized movement learning and retrievalS. Calinon · 2015 · Intelligent Service Robotics · cited 546 times · Not summarized yet
-
Generalized Task-Parameterized Skill LearningYan-Long Huang, João Silvério, L. Rozo et al. · 2017 · IEEE International Conference on Robotics and Automation · cited 60 times · Not summarized yet
Abstract
Programming by demonstration has recently gained much attention due to its user-friendly and natural way to transfer human skills to robots. In order to facilitate the learning of multiple demonstrations and meanwhile generalize to new situations, a task-parameterized Gaussian mixture model (TP-GMM) has been recently developed. This model has achieved reliable performance in areas such as human-robot collaboration and dual-arm manipulation. However, the crucial task frames and associated parameters in this learning framework are often set by the human teacher, which renders three problems that have not been addressed yet: (i) task frames are treated equally, without considering their individ…
-
Density estimation using Real NVPLaurent Dinh, Jascha Narain Sohl-Dickstein, Samy Bengio · 2016 · International Conference on Learning Representations · cited 5,222 times · Not summarized yet
Abstract
Unsupervised learning of probabilistic models is a central yet challenging problem in machine learning. Specifically, designing models with tractable learning, sampling, inference and evaluation is crucial in solving this task. We extend the space of such models using real-valued non-volume preserving (real NVP) transformations, a set of powerful invertible and learnable transformations, resulting in an unsupervised learning algorithm with exact log-likelihood computation, exact sampling, exact inference of latent variables, and an interpretable latent space. We demonstrate its ability to model natural images on four datasets through sampling, log-likelihood evaluation and latent variable ma…
Go further
How are these papers found?
We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.
Updated on 6 November 2026.
The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.