# DizzyRNN: Reparameterizing Recurrent Neural Networks for Norm-Preserving Backpropagation

**Authors:** Victor Dorobantu, Per Andre Stromhaug, Jess Renteria  
**arXiv:** [1612.04035](https://arxiv.org/abs/1612.04035)  
**Date:** 2016-12-13  
**Summary page:** https://www.summarizepaper.com/en/paper/1612.04035v1/

## Key points

- Authors Victor Dorobantu, Per Andre Stromhaug, and Jess Renteria propose a reparameterization technique for standard RNNs using Givens rotations.
- The technique aims to address challenges of vanishing and exploding gradients by preserving signal norms during backpropagation.
- DizzyRNN utilizes absolute value function as an element-wise non-linearity to ensure norm preservation throughout the network.
- Experimental results demonstrate that DizzyRNN outperforms traditional RNN architectures and LSTM networks on tasks with long-range dependencies like the copy problem.
- This innovative approach not only addresses fundamental training issues in RNNs but also enhances performance on challenging sequential learning tasks.

## Summary

, , , , 

In their paper titled "DizzyRNN: Reparameterizing Recurrent Neural Networks for Norm-Preserving Backpropagation," authors Victor Dorobantu, Per Andre Stromhaug, and Jess Renteria propose a novel reparameterization technique for standard RNNs to address the challenges of vanishing and exploding gradients. This approach utilizes Givens rotations to update linear transformations in a way that preserves signal norms during backpropagation. Additionally, the use of absolute value function as an element-wise non-linearity ensures norm preservation throughout the network. The results from experiments show that DizzyRNN outperforms traditional RNN architectures and LSTM networks on tasks involving long-range dependencies such as the copy problem. This innovative approach not only tackles fundamental issues in training RNNs but also offers promising advancements in enhancing their performance on challenging sequential learning tasks.