Voint Cloud: Multi-View Point Cloud Representation for 3D Understanding

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Close Not relevant

   Multiresolution Tree Networks for 3D Point Cloud Processing

   Matheus Gadelha, Rui Wang, Subhransu Maji · 2018 · Computer Vision and Pattern Recognition · AI summary available

   Abstract

   We present multiresolution tree-structured networks to process point clouds for 3D shape understanding and generation tasks. Our network represents a 3D shape as a set of locality-preserving 1D ordered list of points at multiple resolutions. This allows efficient feed-forward processing through 1D convolutions, coarse-to-fine analysis through a multi-grid architecture, and it leads to faster convergence and small memory footprint during training. The proposed tree-structured encoders can be used to classify shapes and outperform existing point-based architectures on shape classification benchmarks, while tree-structured decoders can be used for generating point clouds directly and they outpe…

2. Close Not relevant

   CLIP$^2$: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data

   Yihan Zeng, Chenhan Jiang, Jiageng Mao et al. · 2023 · Computer Vision and Pattern Recognition · AI summary available

   Abstract

   Contrastive Language-Image Pre-training, benefiting from large-scale unlabeled text-image pairs, has demonstrated great performance in open-world vision understanding tasks. However, due to the limited Text-3D data pairs, adapting the success of 2D Vision-Language Models (VLM) to the 3D space remains an open problem. Existing works that leverage VLM for 3D understanding generally resort to constructing intermediate 2D representations for the 3D data, but at the cost of losing 3D geometry information. To take a step toward open-world 3D vision understanding, we propose Contrastive Language-Image-Point Cloud Pretraining (CLIP$^2$) to directly learn the transferable 3D point cloud representatio…

3. Close Not relevant

   PointCLIP: Point Cloud Understanding by CLIP

   Renrui Zhang, Ziyu Guo, Wei Zhang et al. · 2021 · Computer Vision and Pattern Recognition · AI summary available

   Abstract

   Recently, zero-shot and few-shot learning via Contrastive Vision-Language Pre-training (CLIP) have shown inspirational performance on 2D visual recognition, which learns to match images with their corresponding texts in open-vocabulary settings. However, it remains under explored that whether CLIP, pre-trained by large-scale image-text pairs in 2D, can be generalized to 3D recognition. In this paper, we identify such a setting is feasible by proposing PointCLIP, which conducts alignment between CLIP-encoded point cloud and 3D category texts. Specifically, we encode a point cloud by projecting it into multi-view depth maps without rendering, and aggregate the view-wise zero-shot prediction to…

4. Close Not relevant

   Deep Learning for 3D Point Clouds: A Survey

   Yulan Guo, Hanyun Wang, Qingyong Hu et al. · 2019 · Computer Vision and Pattern Recognition · Not summarized yet

   Abstract

   Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, deep learning has been successfully used to solve various 2D vision problems. However, deep learning on point clouds is still in its infancy due to the unique challenges faced by the processing of point clouds with deep neural networks. Recently, deep learning on point clouds has become even thriving, with numerous methods being proposed to address different problems in this area. To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods …

5. Close Not relevant

   Efficient Learning on Point Clouds with Basis Point Sets

   Sergey Prokudin, Christoph Lassner, Javier Romero · 2019 · Computer Vision and Pattern Recognition · AI summary available

   Abstract

   With the increased availability of 3D scanning technology, point clouds are moving into the focus of computer vision as a rich representation of everyday scenes. However, they are hard to handle for machine learning algorithms due to their unordered structure. One common approach is to apply occupancy grid mapping, which dramatically increases the amount of data stored and at the same time loses details through discretization. Recently, deep learning models were proposed to handle point clouds directly and achieve input permutation invariance. However, these architectures often use an increased number of parameters and are computationally inefficient. In this work, we propose basis point set…

6. Close Not relevant

   Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance

   Phuc D. A. Nguyen, Tuan Duc Ngo, Evangelos Kalogerakis et al. · 2023 · Computer Vision and Pattern Recognition · AI summary available

   Abstract

   We introduce Open3DIS, a novel solution designed to tackle the problem of Open-Vocabulary Instance Segmentation within 3D scenes. Objects within 3D environments exhibit diverse shapes, scales, and colors, making precise instance-level identification a challenging task. Recent advancements in Open-Vocabulary scene understanding have made significant strides in this area by employing class-agnostic 3D instance proposal networks for object localization and learning queryable features for each 3D mask. While these methods produce high-quality instance proposals, they struggle with identifying small-scale and geometrically ambiguous objects. The key idea of our method is a new module that aggrega…

7. Close Not relevant

   Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervision

   Zisong Chen, Chunyu Lin, Nie Lang et al. · 2023 · Computer Vision and Pattern Recognition · AI summary available

   Abstract

   Omnidirectional multi-view stereo (MVS) vision is attractive for its ultra-wide field-of-view (FoV), enabling machines to perceive 360{\deg} 3D surroundings. However, the existing solutions require expensive dense depth labels for supervision, making them impractical in real-world applications. In this paper, we propose the first unsupervised omnidirectional MVS framework based on multiple fisheye images. To this end, we project all images to a virtual view center and composite two panoramic images with spherical geometry from two pairs of back-to-back fisheye images. The two 360{\deg} images formulate a stereo pair with a special pose, and the photometric consistency is leveraged to establi…

8. Close Not relevant

   Volumetric and Multi-View CNNs for Object Classification on 3D Data

   Charles R. Qi, Hao Su, Matthias Niessner et al. · 2016 · Computer Vision and Pattern Recognition · AI summary available

   Abstract

   3D shape models are becoming widely available and easier to capture, making available 3D information crucial for progress in object classification. Current state-of-the-art methods rely on CNNs to address this problem. Recently, we witness two types of CNNs being developed: CNNs based upon volumetric representations versus CNNs based upon multi-view representations. Empirical results from these two types of CNNs exhibit a large gap, indicating that existing volumetric CNN architectures and approaches are unable to fully exploit the power of 3D representations. In this paper, we aim to improve both volumetric CNNs and multi-view CNNs according to extensive analysis of existing approaches. To …