# Perceiver: General Perception with Iterative Attention

**Authors:** Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, Joao Carreira  
**arXiv:** [2103.03206](https://arxiv.org/abs/2103.03206)  
**Date:** 2021-03-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2103.03206v2/

## Key points

- The Perceiver model addresses the challenge of processing high-dimensional inputs from diverse modalities such as vision, audition, touch, and proprioception.
- It leverages Transformers to make fewer architectural assumptions about input relationships compared to traditional deep learning perception models.
- The model is scalable to handle hundreds of thousands of inputs similar to Convolutional Neural Networks (ConvNets) through an asymmetric attention mechanism that iteratively distills inputs into a compact latent bottleneck.
- The Perceiver demonstrates competitive performance on classification tasks involving images, point clouds, audio, video, and combined audio-video data without being constrained by modality-specific priors.

## Summary

, , , , 

The Perceiver model, introduced in the paper "Perceiver: General Perception with Iterative Attention" by Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, and Joao Carreira, addresses the challenge of processing high-dimensional inputs from diverse modalities such as vision, audition, touch, and proprioception. The model leverages Transformers to make fewer architectural assumptions about input relationships compared to traditional deep learning perception models. One key feature is its scalability to handle hundreds of thousands of inputs similar to Convolutional Neural Networks (ConvNets) through an asymmetric attention mechanism that iteratively distills inputs into a compact latent bottleneck. This design allows for effective processing of large multi-modal inputs without being constrained by modality-specific priors. In evaluations, the Perceiver demonstrates competitive performance on classification tasks involving images, point clouds, audio, video and combined audio-video data. Notably achieving results comparable to ResNet-50 and Vision Transformer (ViT) on ImageNet without using 2D convolutions by directly attending to 50,000 pixels. Additionally showing competitiveness across all modalities in AudioSet datasets. Overall presenting a promising approach for general perception tasks through combining the flexibility of Transformers with scalable processing capabilities for multi-modal input processing without being limited by modality-specific assumptions.