Journal club slides on “Mamba: Linear-Time Sequence Modeling with Selective State Spaces”, each point with its passage of the paper

A real example: what you get

15 slides for a 20-minute talk · “Mamba: Linear-Time Sequence Modeling with Selective State Spaces”

• Each point with its passage of the paper
• Earlier work and what came next, strengths and limits stated by the authors
• Where the paper stands: published, corrected, newer version
• 5 hard questions to prepare
• Presenter view, PowerPoint, Beamer and a link to share

The first results

1. Mamba: Linear-Time Sequence Modeling with Selective State Spaces
2. Context and central question Can recurrent context compression support content-dependent reasoning without attention’s sequence-length costs?
3. Selective dynamics Selection makes sequence-interaction parameters input-dependent.
4. Hardware and architecture Selective scan combines kernel fusion, parallel scan, and recomputation.
5. Mechanistic tests and ablations Induction-head experiments train two-layer models at length 256 with vocabulary size 16.
6. Language modeling and earlier work Language experiments evaluate Pile pretraining and downstream zero-shot tasks.
7. Long-context transfer across modalities DNA pretraining uses HG38, approximately 4.5 billion training tokens from one human genome.
8. Efficiency results Selective scan outperforms FlashAttention-2 beyond sequence length 2K in the reported operation benchmark.
9. What the comparisons control Pythia, RWKV, and Mamba comparisons share tokenizer, dataset, and 300-billion-token training length.
10. Take-home message Selective dynamics and hardware-aware execution jointly make recurrent sequence modeling competitive in these experiments.

5 more results hidden

See a complete example, explained step by step →