# Learning to Plan Maneuverable and Agile Flight Trajectory with Optimization Embedded Networks

**Authors:** Zhichao Han, Long Xu, Fei Gao  
**arXiv:** [2405.07736](https://arxiv.org/abs/2405.07736)  
**Date:** 2024-05-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2405.07736v1/

## Key points

- Growing interest in using deep neural networks for end-to-end flight navigation
- Challenges of replacing original modules with neural networks in a black-box manner
- Introduction of optimization-embedded neural networks by Zhichao Han, Long Xu, and Fei Gao
- Leveraging traditional methods and neural networks to learn high-quality trajectories directly from visual inputs
- Utilization of deep neural network to extract environment safety regions and represent them as safety constraints in trajectory optimization
- Ability to robustly converge to feasible and optimal solutions that meet user-defined constraints through efficient optimization algorithms
- Training the optimization process as a layer within the neural network for direct interaction between perception and planning
- Enhancing quality and stability of generated trajectories by focusing on spatial regions where optimal solutions exist

## Summary

, , , , 

In recent years, there has been a growing interest among researchers in the use of deep neural networks for end-to-end flight navigation. This approach has become popular because it effectively addresses the disconnect between perception and planning present in traditional methods, thereby eliminating delays between different modules. However, a common issue with this approach is that replacing original modules with neural networks in a black-box manner can compromise the overall system's robustness and stability. This lack of transparency often leads to difficulties in consistently generating high-quality motion trajectories and properly accounting for the robot's kinematic constraints, resulting in trajectories that may not be executable satisfactorily. To overcome these challenges, a novel method known as optimization-embedded neural networks has been proposed by Zhichao Han, Long Xu, and Fei Gao. This innovative approach combines the strengths of traditional methods with those of neural networks to learn high-quality trajectories directly from visual inputs without requiring mapping while ensuring dynamic feasibility. The deep neural network is utilized to extract environment safety regions from depth images, which are then represented as safety constraints in trajectory optimization using a model-based approach. One key advantage of this method is its ability to leverage highly efficient optimization algorithms to robustly converge to feasible and optimal solutions that meet various user-defined constraints. Additionally, by training the optimization process as a layer within the neural network, direct interaction between perception and planning is facilitated. This enables the network to focus on spatial regions where optimal solutions exist, ultimately enhancing the quality and stability of generated trajectories. Overall, this research represents a significant advancement in flight trajectory planning by bridging the gap between traditional methods and deep learning techniques. By integrating optimization principles within neural networks, this approach offers a promising solution for maneuverable and agile flight navigation while ensuring safety and feasibility in complex environments.