Ternary Spike: Learning Ternary Spikes for Spiking Neural Networks
AI-generated Key Points
- The Spiking Neural Network (SNN) is a biologically inspired neural network infrastructure.
- SNN utilizes binary spike activations for efficient energy usage.
- Binary spike activation map lacks sufficient information capacity, leading to a loss of accuracy.
- Researchers propose ternary spike neuron as an alternative method for transmitting information in SNNs.
- Ternary spike neuron retains advantages of event-driven and multiplication-free operations while enhancing information capacity.
- Trainable factor within ternary spike neuron learns suitable spike amplitudes, accommodating variations in membrane potential distributions.
- Trainable ternary spike SNN is converted back into standard SNN through re-parameterization techniques during inference for efficiency.
- Extensive experiments show that ternary spike consistently outperforms state-of-the-art methods on static and dynamic datasets with various network structures.
- Code for implementing the approach is open-sourced and available at https://github.com/yfguo91/Ternary-Spike.
- Study has been accepted by AAAI2024 conference for presentation.
Authors: Yufei Guo, Yuanpei Chen, Xiaode Liu, Weihang Peng, Yuhan Zhang, Xuhui Huang, Zhe Ma
Abstract: The Spiking Neural Network (SNN), as one of the biologically inspired neural network infrastructures, has drawn increasing attention recently. It adopts binary spike activations to transmit information, thus the multiplications of activations and weights can be substituted by additions, which brings high energy efficiency. However, in the paper, we theoretically and experimentally prove that the binary spike activation map cannot carry enough information, thus causing information loss and resulting in accuracy decreasing. To handle the problem, we propose a ternary spike neuron to transmit information. The ternary spike neuron can also enjoy the event-driven and multiplication-free operation advantages of the binary spike neuron but will boost the information capacity. Furthermore, we also embed a trainable factor in the ternary spike neuron to learn the suitable spike amplitude, thus our SNN will adopt different spike amplitudes along layers, which can better suit the phenomenon that the membrane potential distributions are different along layers. To retain the efficiency of the vanilla ternary spike, the trainable ternary spike SNN will be converted to a standard one again via a re-parameterization technique in the inference. Extensive experiments with several popular network structures over static and dynamic datasets show that the ternary spike can consistently outperform state-of-the-art methods. Our code is open-sourced at https://github.com/yfguo91/Ternary-Spike.
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
Assess the quality of the AI-generated content by voting
Score: -1
Why do we need votes?
Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.
The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.
Similar papers summarized with our AI tools
Navigate through even more similar papers through a
tree representationLook for similar papers (in beta version)
By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.