# MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoning

**Authors:** Yuxuan Luo, Yuhui Yuan, Junwen Chen, Haonan Cai, Ziyi Yue, Yuwei Yang, Fatima Zohra Daha, Ji Li, Zhouhui Lian  
**arXiv:** [2506.10963](https://arxiv.org/abs/2506.10963)  
**Date:** 2025-06-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2506.10963v2/

## Key points

- The Massive Multi-Discipline Multi-Tier Knowledge-Image Generation Benchmark (MMMG) is introduced to assess reasoning capabilities of image generation models in knowledge image generation.
- Knowledge images are essential for human civilization and learning processes, requiring complex multimodal reasoning to combine world knowledge with pixel-level grounding.
- MMMG consists of 4,456 expert-validated knowledge image-prompt pairs across different disciplines, educational levels, and formats, using a unified Knowledge Graph representation for evaluation complexity.
- The MMMG-Score metric is introduced to evaluate generated knowledge images based on factual fidelity and visual clarity.
- Evaluations of 16 state-of-the-art text-to-image generation models reveal significant reasoning deficits such as low entity fidelity and weak relations.
- The authors release FLUX-Reason as an open baseline model trained on 16,000 curated knowledge image-prompt pairs to drive further advancements in the field.
- Detailed analyses show varying levels of success in capturing essential entities and dependencies within tasks, emphasizing the importance of robust reasoning capabilities in knowledge image generation tasks.

## Summary

The authors present the Massive Multi-Discipline Multi-Tier Knowledge-Image Generation Benchmark (MMMG) to assess the reasoning capabilities of image generation models in the novel task of knowledge image generation. Knowledge images are crucial for human civilization and learning processes, and require complex multimodal reasoning to combine world knowledge with pixel-level grounding. MMMG consists of 4,456 expert-validated knowledge image-prompt pairs across various disciplines, educational levels, and formats. A unified Knowledge Graph representation is adopted for each pair to facilitate evaluation complexity. The authors also introduce the MMMG-Score metric to evaluate generated knowledge images based on factual fidelity and visual clarity. Evaluations of 16 state-of-the-art text-to-image generation models reveal significant reasoning deficits such as low entity fidelity and weak relations. To drive further advancements in this field, the authors release FLUX-Reason as an effective open baseline model trained on 16,000 curated knowledge image-prompt pairs. Detailed analyses showcase varying levels of success in capturing essential entities and dependencies within given tasks. This study highlights the importance of robust reasoning capabilities in knowledge image generation tasks and sets a high standard for future advancements in this domain.