Running summarizing tools on a new article
Check the paper below, then start its synthesis.
InTRRA: Inter-Task Resource-Repurposing Accelerator for Efficient Transformer Inference on FPGAs
Zifan He (University of California, Los Angeles), Hersh Gupta (University of California, Los Angeles), Hughes Ke (University of California, Los Angeles), Jason Cong (University of California, Los Angeles)
Abstract: The rise of deep neural networks (DNNs) has driven a boom in AI services, accompanied by increasing demands for computing power and memory. One of the most widely used DNNs is the transformer model. Unlike convolutional neural networks (CNNs), transformers involve more complex computations and generate larger intermediate data, which pose significant challenges for computation efficiency and off-chip memory access overhead in large language model (LLM) accelerator designs. Unfortunately, existing LLM accelerators have fixed execution patterns (either dataflow or sequential), resulting in either low computational efficiency due to pipeline stalls or frequent off-chip memory accesses to manage large intermediate results. This ultimately leads to high single-batch inference latency.