Results of the summarizing process for the arXiv paper: 2303.15467v1

The problem of open-set action recognition (OSAR) involves rejecting unknown human action cases that fall outside the distribution of the training set. Existing methods for OSAR focus on learning better uncertainty scores, but they often overlook the importance of feature representations. This paper proposes a novel Prototypical Similarity Learning (PSL) framework to enlarge instance-specific (IS) and class-specific (CS) information in feature representations for better OSAR performance. The authors begin by analyzing the behavior of feature representations in the open-set problem using the information bottleneck theory. They divide the information contained in features into IS and CS categories. CS information is used for inter-class recognition, while IS information is unique to each sample within a class. Both types of information are crucial for OSAR performance. To enlarge IS information, PSL encourages instances to have less than 1 similarity with their corresponding prototypes, retaining more IS information in learned feature representations. The authors also introduce video shuffling into PSL to alleviate misclassification issues caused by OoD videos that share similar appearances with InD videos. Shuffled videos are encouraged to have less than 1 similarity with original samples, allowing networks to extract distinct temporal information among them and enlarging CS information. Experiments demonstrate that PSL significantly boosts both open-set and closed-set performance on multiple benchmarks, achieving state-of-the-art results. The proposed framework provides a novel perspective on analyzing OSAR tasks based on the information bottleneck theory and highlights the importance of retaining both IS and CS information for optimal performance.

• Open-set action recognition (OSAR) involves rejecting unknown human action cases that fall outside the distribution of the training set.
• Existing methods for OSAR focus on learning better uncertainty scores, but they often overlook the importance of feature representations.
• The authors propose a novel Prototypical Similarity Learning (PSL) framework to enlarge instance-specific (IS) and class-specific (CS) information in feature representations for better OSAR performance.
• CS information is used for inter-class recognition, while IS information is unique to each sample within a class. Both types of information are crucial for OSAR performance.
• To enlarge IS information, PSL encourages instances to have less than 1 similarity with their corresponding prototypes, retaining more IS information in learned feature representations.
• Video shuffling is introduced into PSL to alleviate misclassification issues caused by OoD videos that share similar appearances with InD videos.
• Shuffled videos are encouraged to have less than 1 similarity with original samples, allowing networks to extract distinct temporal information among them and enlarging CS information.
• Experiments demonstrate that PSL significantly boosts both open-set and closed-set performance on multiple benchmarks, achieving state-of-the-art results.
• The proposed framework provides a novel perspective on analyzing OSAR tasks based on the information bottleneck theory and highlights the importance of retaining both IS and CS information for optimal performance.

The Problem of Open-Set Action Recognition (OSAR)

Open-set action recognition (OSAR) is a challenging problem in computer vision that involves recognizing human actions from videos while also rejecting unknown cases that fall outside the distribution of the training set. Existing methods for OSAR focus on learning better uncertainty scores, but they often overlook the importance of feature representations. This paper proposes a novel Prototypical Similarity Learning (PSL) framework to enlarge instance-specific (IS) and class-specific (CS) information in feature representations for better OSAR performance.

Analyzing Feature Representations with Information Bottleneck Theory

The authors begin by analyzing the behavior of feature representations in the open-set problem using the information bottleneck theory. They divide the information contained in features into IS and CS categories. CS information is used for inter-class recognition, while IS information is unique to each sample within a class. Both types of information are crucial for OSAR performance.

Enlarging Instance Specific Information

To enlarge IS information, PSL encourages instances to have less than 1 similarity with their corresponding prototypes, retaining more IS information in learned feature representations.

Video Shuffling

The authors also introduce video shuffling into PSL to alleviate misclassification issues caused by OoD videos that share similar appearances with InD videos. Shuffled videos are encouraged to have less than 1 similarity with original samples, allowing networks to extract distinct temporal information among them and enlarging CS information.

Experimental Results

Experiments demonstrate that PSL significantly boosts both open-set and closed-set performance on multiple benchmarks, achieving state-of-the-art results. The proposed framework provides a novel perspective on analyzing OSAR tasks based on the information bottleneck theory and highlights the importance of retaining both IS and CS information for optimal performance