A Runtime-Adaptive Transformer Neural Network Accelerator on FPGAs

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Not relevant

   HPTA: A High Performance Transformer Accelerator Based on FPGA

   Yuntao Han, Qiang Liu · 2023 · International Conference on Field-Programmable Logic and Applications · cited 32 times · Not summarized yet

   Abstract

   The transformer neural networks have achieved remarkable performance in both Natural Language Processing (NLP) and Computer Vision (CV) applications, with encoder-decoder architecture based on attention layers. However, implementing transformers on resource-constrained devices presents challenges due to the super-large network structures and nontrivial dataflows. Field-Programmable Gate Arrays (FPGA) have been a promising platform for Neural Network (NN) acceleration due to their design flexibility and customization. Existing FPGA-based implementations of transformers face efficiency and generality issues. This paper proposes HPTA, a high-performance accelerator for implementing transformers…

2. Very close Not relevant

   InTRRA: Inter-Task Resource-Repurposing Accelerator for Efficient Transformer Inference on FPGAs

   Zifan He, Hersh Gupta, Huifeng Ke et al. · 2025 · Symposium on Field Programmable Gate Arrays · cited 3 times · Not summarized yet

   Abstract

   The rise of deep neural networks (DNNs) has driven a boom in AI services, accompanied by increasing demands for computing power and memory. One of the most widely used DNNs is the transformer model. Unlike convolutional neural networks (CNNs), transformers involve more complex computations and generate larger intermediate data, which pose significant challenges for computation efficiency and off-chip memory access overhead in large language model (LLM) accelerator designs. Unfortunately, existing LLM accelerators have fixed execution patterns (either dataflow or sequential), resulting in either low computational efficiency due to pipeline stalls or frequent off-chip memory accesses to manage…

3. Very close Not relevant

   ProTEA: Programmable Transformer Encoder Acceleration on FPGA

   Ehsan Kabir, Jason D. Bakos, David Andrews et al. · 2024 · SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis · cited 8 times · Not summarized yet

   Abstract

   Transformer neural networks (TNN) have been widely utilized on a diverse range of applications, including natural language processing (NLP), machine translation, and computer vision (CV). Their widespread adoption has been primarily driven by the exceptional performance of their multi-head self-attention block used to extract key features from sequential data. The multi-head self-attention block is followed by feedforward neural networks, which play a crucial role in introducing non-linearity to assist the model in learning complex patterns. Despite the popularity of TNNs, there has been limited numbers of hardware accelerators targeting these two critical blocks. Most prior works have conce…

4. Very close Cited by this paper Not relevant

   TRAC: Compilation-Based Design of Transformer Accelerators for FPGAs

   Patrick Plagwitz, Frank Hannig, Jürgen Teich · 2022 · International Conference on Field-Programmable Logic and Applications · cited 12 times · Not summarized yet

   Abstract

   Transformer-type Neural Networks (NNs) have shown impressive accuracy numbers in Natural Language Processing (NLP) applications where Recurrent Neural Networks (RNNs) have been in use before, even surpassing them. However, differing considerably from common types of NNs, existing accelerator designs, particularly for Field-Programmable Gate Arrays (FPGAs), cannot be used to implement them. Previous research has shown FPGAs to be platforms superior to CPUs and even GPUs for accelerating NNs when it comes to energy efficiency. Following the development of automated compiler-based design flows for NNs, there is still a lack of such an approach for transformers and FPGA targets. In this realm, t…

5. Very close Cited by this paper Not relevant

   FTRANS: energy-efficient acceleration of transformers using FPGA

   Bingbing Li, Santosh Pandey, Haowen Fang et al. · 2020 · International Symposium on Low Power Electronics and Design · cited 245 times · Not summarized yet

   Abstract

   In natural language processing (NLP), the "Transformer" architecture was proposed as the first transduction model replying entirely on self-attention mechanisms without using sequence-aligned recurrent neural networks (RNNs) or convolution, and it achieved significant improvements for sequence to sequence tasks. The introduced intensive computation and storage of these pre-trained language representations has impeded their popularity into computation and memory constrained devices. The field-programmable gate array (FPGA) is widely used to accelerate deep learning algorithms for its high parallelism and low latency. However, the trained models are still too large to accommodate to an FPGA fa…

6. Very close Cited by this paper Not relevant

   Hardware Acceleration of Transformer Networks using FPGAs

   Georgios Tzanos, C. Kachris, D. Soudris · 2022 · 2022 Panhellenic Conference on Electronics & Telecommunications (PACET) · cited 14 times · Not summarized yet

   Abstract

   Natural Language Processing (NLP) allows program computers to process and analyze large amounts of natural language data. In the last few years, NLP has shown tremendous growth, with many organizations presenting models such as BERT (Bidirectional Encoder Representations from Transformers), GPT2 (Generative Pre-trained Transformer 2), GPT3 (Generative Pre-trained Transformer 3), etc. The cornerstone of these models and the reason for the growth of the NLP is mainly due to the Transformer networks. However, very few architectures have been presented, for the acceleration of the Transformer networks using FPGAs. In this paper, we propose a novel architecture for Transformer Networks optimized …

7. Very close Not relevant

   An Efficient Hardware Accelerator for Sparse Transformer Neural Networks

   Chao Fang, Shouliang Guo, Wei Wu et al. · 2022 · International Symposium on Circuits and Systems · cited 25 times · Not summarized yet

   Abstract

   Transformers have been an indispensable staple in deep learning. However, it is challenging to realize efficient deployment for Transformer-based model due to their substantial computation and memory demands. To address this issue, we present an efficient sparse Transformer accelerator on FPGA, namely STA, by exploiting N:M fine-grained structured sparsity. Our design features not only a unified computing engine capable of performing both sparse and dense matrix multiplications with high computational efficiency, but also a scalable softmax module eliminating the latency from intermediate off-chip data communication. Experimental results show that our implementation achieves the lowest laten…

8. Very close Cited by this paper Not relevant

   Accelerating Transformer Neural Networks on FPGAs for High Energy Physics Experiments

   Filip Wojcicki, Zhi-Qiang Que, A. Tapper et al. · 2022 · International Conference on Field-Programmable Technology · cited 13 times · Not summarized yet

   Abstract

   High Energy Physics studies the fundamental forces and elementary particles of the Universe. With the unprecedented scale of experiments comes the challenge of accurate, ultra-low latency decision-making. Transformer Neural Networks (TNNs) have been proven to accomplish cutting-edge accuracy in classification for hadronic jet tagging. Nevertheless, software-centered solutions targeting CPUs and GPUs lack the inference speed required for real-time particle triggers, most notably those at the CERN Large Hadron Collider. This paper proposes a novel TNN-based architecture, efficiently mapped to Field-Programmable Gate Arrays, that outperforms GPU inference capabilities involving state-of-the-art…

Prior work it builds on

1. Very close Not relevant

   A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining

   Hongwu Peng, Shaoyi Huang, Shi-Yang Chen et al. · 2022 · Design Automation Conference · cited 75 times · Not summarized yet

   Abstract

   Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable triumphs, the prolonged turnaround time of Transformer models is a widely recognized roadblock. The variety of sequence lengths imposes additional computing overhead where inputs need to be zero-padded to the maximum sentence length in the batch to accommodate the parallel computing platforms. This paper targets the field-programmable gate array (FPGA) and proposes a coherent sequence length adaptive algorithm-hardware co-design for Transformer accelerati…

2. Very close Not relevant

   Accelerating Transformer-based Deep Learning Models on FPGAs using Column Balanced Block Pruning

   Hongwu Peng, Shaoyi Huang, T. Geng et al. · 2021 · IEEE International Symposium on Quality Electronic Design · cited 116 times · Not summarized yet

   Abstract

   Although Transformer-based language representations achieve state-of-the-art accuracy on various natural language processing (NLP) tasks, the large model size has been challenging the resource constrained computing platforms. Weight pruning, as a popular and effective technique in reducing the number of weight parameters and accelerating the Transformer, has been investigated on GPUs. However, the Transformer acceleration using weight pruning on field-programmable gate array (FPGAs) remains unexplored. This paper investigates the column balanced block-wise pruning on Transformer and designs an FPGA acceleration engine to customize the balanced blockwise matrix multiplication. We implement th…

3. Very close Not relevant

   FET-OPU: A Flexible and Efficient FPGA-Based Overlay Processor for Transformer Networks

   Yueyin Bai, Hao Zhou, Keqing Zhao et al. · 2023 · 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD) · cited 19 times · Not summarized yet

   Abstract

   There are already some works on accelerating transformer networks with field-programmable gate array (FPGA). However, many accelerators focus only on attention computation or suffer from fixed data streams without flexibility. Moreover, their hardware performance is limited without schedule optimization and full use of hardware resources. In this article, we propose a flexible and efficient FPGA-based overlay processor, named FET-OPU. Specifically, we design an overlay architecture for general accelerations of transformer networks. We propose a unique matrix multiplication unit (MMU), which consists of a processing element (PE) array based on modified DSP-packing technology and a FIFO array …

4. Close Not relevant

   Accelerating Attention Mechanism on FPGAs based on Efficient Reconfigurable Systolic Array

   Wenhua Ye, Xu Zhou, Joey Tianyi Zhou et al. · 2022 · ACM Transactions on Embedded Computing Systems · cited 46 times · Not summarized yet

   Abstract

   Transformer model architectures have recently received great interest in natural language, machine translation, and computer vision, where attention mechanisms are their building blocks. However, the attention mechanism is expensive because of its intensive matrix computations and complicated data flow. The existing hardware architecture has some disadvantages for the computing structure of attention, such as inflexibility and low efficiency. Most of the existing papers accelerate attention by reducing the amount of computation through various pruning algorithms, which will affect the results to a certain extent with different sparsity. This paper proposes the hardware accelerator for the mu…

5. Close Not relevant

   Enhancing Long Sequence Input Processing in FPGA-Based Transformer Accelerators through Attention Fusion

   Yun-Ji Qin, Wenqi Lou, Chao Wang et al. · 2024 · ACM Great Lakes Symposium on VLSI · cited 16 times · Not summarized yet

   Abstract

   Attention-based transformers have achieved significant performance breakthroughs in natural language processing (NLP) and computer vision (CV) tasks. Meanwhile, the ever-increasing length of today’s input sequences puts much pressure on computing devices. FPGAs are widely used to accelerate Transformer inference due to their high energy efficiency and flexibility. However, most of the existing FPGA-based Transformer accelerators are oriented to small input lengths, making it hard to accelerate long input sequences. To this end, we design an efficient Transformer accelerator for FPGA and long-sequence input scenarios. We use the tiling softmax algorithm to fuse attention computation, eliminat…

Papers that cite it

1. Close Not relevant

   Recent developments in Transformer inference deployment on FPGA platforms: A survey

   Arjan Blankestijn, Uraz Odyurt, A. Yousefzadeh · 2026 · Journal of systems architecture · cited 2 times · Not summarized yet

   Abstract

   With the rapid and continuous growth in the incorporation of machine learning models based on the Transformer architecture, capable deployment is in high demand. In this context, capable deployment refers to operational performance aspects, e.g., throughput and latency, as well as efficiency aspects, e.g., energy consumption. When it comes to the task of inference using such models, purpose-built hardware accelerators provide a lucrative alternative to common deployment choices, such as Central Processing Units (CPUs) and Graphics Processing Units (GPUs). The Field Programmable Gate Array (FPGA) platforms category is an example of such alternative accelerators, promising implementation flexi…

2. Close Not relevant

   TrackCore-F: Deploying Transformer-Based Subatomic Particle Tracking on FPGAs

   Arjan Blankestijn, Uraz Odyurt, A. Yousefzadeh · 2025 · arXiv.org · preprint, not peer reviewed · Not summarized yet

   Abstract

   The Transformer Machine Learning (ML) architecture has been gaining considerable momentum in recent years. In particular, computational High-Energy Physics tasks such as jet tagging and particle track reconstruction (tracking), have either achieved proper solutions, or reached considerable milestones using Transformers. On the other hand, the use of specialised hardware accelerators, especially FPGAs, is an effective method to achieve online, or pseudo-online latencies. The development and integration of Transformer-based ML to FPGAs is still ongoing and the support from current tools is very limited or non-existent. Additionally, FPGA resources present a significant constraint. Considering …

3. Close Not relevant

   TriCCOT: Tri-part Convolutional Conformal Transformer for Onboard Space Object Detection

   A. Dorise, Marjorie Bellizzi, Julia F. Cohen et al. · 2026 · cited 2 times · Not summarized yet

   Abstract

   Onboard object detection in Earth observation is constrained by limited computational resources and the absence of fully corrected imagery. While convolutional detectors are hardware-efficient, they often struggle to extract robust representations from raw and noisy data. Conversely, transformer-based models provide stronger global reasoning capabilities but remain difficult to deploy on FPGA accelerators due to quadratic attention complexity and non-compatible operations. We introduce TriCCOT, a tri-part architecture for robust and deployable onboard object detection. TriCCOT combines a convolutional region proposal network, a conformal prediction stage, and Aper-GATES, our hardware-friendl…

4. Related* Not relevant

   FPGA-Based Hardware-Accelerated Sentiment Analysis using a Hybrid LLM Model with OpenVINO and Intel FPGA AI Suite Optimization

   Bonthu Chandra Praveen, S. K · 2026 · Results in Engineering · Not summarized yet
5. Related* Not relevant

   Constraint-guided generative AI for gate-level VLSI design with early physical awareness

   Ya-Di Lin, Wen-Di Lin · 2026 · Integration · Not summarized yet