Meta Module Network for Compositional Visual Reasoning

AI-generated keywords: Meta Module Network Neural Module Networks Scalability Generalizability WACV 21

AI-generated Key Points

The license of the paper does not allow us to build upon its content and the key points are generated using the paper metadata rather than the full article.

  • The Meta Module Network (MMN) is a new architecture designed to address scalability and generalizability issues in Neural Module Networks (NMNs) for compositional visual reasoning tasks.
  • NMNs have strong interpretability and compositionality, but their customized modules make them impractical when scaling up to larger sets of functions in complex tasks.
  • MMN uses a meta module that can take in function recipes and dynamically morph into diverse instance modules, which are woven into an execution graph for complex visual reasoning while inheriting the strong explainability and compositionality of NMN.
  • With this flexible instantiation mechanism, the parameters of instance modules are inherited from the central meta module, retaining the same model complexity as the function set grows, promising better scalability.
  • Functions are encoded into the embedding space, allowing unseen functions to be represented based on their structural similarity with previously observed ones ensuring better generalizability.
  • Experiments on GQA and CLEVR datasets validate the superiority of MMN over state-of-the-art NMN designs. Synthetic experiments on held-out unseen functions from GQA dataset also demonstrate the strong generalizability of MMN.
  • The authors Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Wang and Jingjing Liu have released their code and model on Github for practical use.
  • The paper has been accepted for oral presentation at WACV 21 conference.
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Wang, Jingjing Liu

Accepted to WACV 21 (Oral)

Abstract: Neural Module Network (NMN) exhibits strong interpretability and compositionality thanks to its handcrafted neural modules with explicit multi-hop reasoning capability. However, most NMNs suffer from two critical drawbacks: 1) scalability: customized module for specific function renders it impractical when scaling up to a larger set of functions in complex tasks; 2) generalizability: rigid pre-defined module inventory makes it difficult to generalize to unseen functions in new tasks/domains. To design a more powerful NMN architecture for practical use, we propose Meta Module Network (MMN) centered on a novel meta module, which can take in function recipes and morph into diverse instance modules dynamically. The instance modules are then woven into an execution graph for complex visual reasoning, inheriting the strong explainability and compositionality of NMN. With such a flexible instantiation mechanism, the parameters of instance modules are inherited from the central meta module, retaining the same model complexity as the function set grows, which promises better scalability. Meanwhile, as functions are encoded into the embedding space, unseen functions can be readily represented based on its structural similarity with previously observed ones, which ensures better generalizability. Experiments on GQA and CLEVR datasets validate the superiority of MMN over state-of-the-art NMN designs. Synthetic experiments on held-out unseen functions from GQA dataset also demonstrate the strong generalizability of MMN. Our code and model are released in Github https://github.com/wenhuchen/Meta-Module-Network.

Submitted to arXiv on 08 Oct. 2019

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

The license of the paper does not allow us to build upon its content and the AI assistant only knows about the paper metadata rather than the full article.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 1910.03230v5

This paper's license doesn't allow us to build upon its content and the summarizing process is here made with the paper's metadata rather than the article.

The Meta Module Network (MMN) is a novel architecture proposed to address the scalability and generalizability issues of Neural Module Networks (NMNs) in compositional visual reasoning tasks. While NMNs have strong interpretability and compositionality, their customized modules for specific functions make them impractical when scaling up to larger sets of functions in complex tasks. To overcome these limitations, MMN centers on a novel meta module that can take in function recipes and dynamically morph into diverse instance modules. The instance modules are then woven into an execution graph for complex visual reasoning, inheriting the strong explainability and compositionality of NMN. With such a flexible instantiation mechanism, the parameters of instance modules are inherited from the central meta module, retaining the same model complexity as the function set grows which promises better scalability. Furthermore, as functions are encoded into the embedding space, unseen functions can be readily represented based on their structural similarity with previously observed ones ensuring better generalizability. Experiments on GQA and CLEVR datasets validate the superiority of MMN over state-of-the-art NMN designs. Synthetic experiments on held-out unseen functions from GQA dataset also demonstrate the strong generalizability of MMN. The authors Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Wang and Jingjing Liu have released their code and model on Github for practical use. The paper has been accepted for oral presentation at WACV 21 conference.
Created on 26 Apr. 2023

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.