# Inferring Restaurant Styles by Mining Crowd Sourced Photos from User-Review Websites

**Authors:** Haofu Liao, Yuncheng Li, Tianran Hu, Jiebo Luo  
**arXiv:** [1611.06301](https://arxiv.org/abs/1611.06301)  
**Date:** 2016-11-19  
**Summary page:** https://www.summarizepaper.com/en/paper/1611.06301v3/

## Key points

- User-uploaded photos are crucial in online restaurant reviews for providing immediate and tangible impressions of a restaurant.
- Restaurant review websites enhance the search experience by offering insights into aspects such as ambiance, dish styles, and suitability for different occasions through user-contributed images.
- The paper introduces a novel approach to inferring restaurant types or styles from user-contributed photos on review websites using deep multi-instance multi-label learning (MIML) framework.
- The methodology involves collecting a unique dataset associating user photos with restaurant styles from TripAdvisor and utilizing a two-step bootstrap strategy to train a multi-label convolutional neural network (CNN) for classifying restaurant styles based on associated images.
- Experimental evaluation demonstrates the effectiveness of this crowd-sourcing approach in inferring restaurant styles when an adequate number of user-uploaded photos are available.

## Summary

In the realm of online restaurant reviews, user-uploaded photos play a crucial role in providing potential diners with immediate and tangible impressions of a restaurant. Leveraging these informative images, restaurant review websites enhance the search experience for users by offering insights into various aspects such as ambiance, dish styles, and suitability for different occasions. This paper introduces a novel approach to inferring restaurant types or styles from user-contributed photos on review websites. The methodology involves collecting a unique dataset associating user photos with restaurant styles from TripAdvisor and implementing a deep multi-instance multi-label learning (MIML) framework to address the classification task. The proposed framework utilizes a two-step bootstrap strategy to train a multi-label convolutional neural network (CNN) that computes confidence scores for restaurant styles based on associated images. These scores are then used to train final binary classifiers for each style tag, allowing for the profiling of restaurant styles through analysis of user photos. Experimental evaluation demonstrates the effectiveness of this crowd-sourcing approach in inferring restaurant styles when an adequate number of user-uploaded photos are available. The remainder of the paper is structured as follows: Section II delves into related work in social media and multiple instance multiple label learning, highlighting the uniqueness of this study's focus on abstract concepts like restaurant styles. Section III outlines the data collection procedure and statistics of the collected dataset, while Section IV details the proposed framework and algorithms employed. Performance evaluation is discussed in Section V, with conclusions presented in Section VI. Overall, this research offers valuable insights into leveraging user-generated content to classify restaurant styles accurately, paving the way for enhanced search experiences and recommendations for diners seeking specific dining preferences.