Résultats de recherche pour « Neural Compression »

Neural compression algorithms are typically based on autoencoders that require specialized encoder and decoder architectures for different data modalities. In this paper, we propose COIN++, a neural compression framework that seamlessly handles a wide range of data modalities. Our approach is based on converting data to implicit neural representations, i.e. neural functions that map coordinates (such as pixel locations) to features (such as RGB values). Then, instead of storing the weights of the implicit neural representation directly, we store modulations applied to a meta-learned base network as a compressed code for the data. We further quantize and entropy code these modulations, leading to large compression gains while reducing encoding time by two orders of magnitude compared to baselines. We empirically demonstrate the feasibility of our method by compressing various data modalities, from images and audio to medical and climate data.

A popular approach to learning encoders for lossy compression is to use additive uniform noise during training as a differentiable approximation to test-time quantization. We demonstrate that a uniform noise channel can also be implemented at test time using universal quantization (Ziv, 1985). This allows us to eliminate the mismatch between training and test phases while maintaining a completely differentiable loss function. Implementing the uniform noise channel is a special case of the more general problem of communicating a sample, which we prove is computationally hard if we do not make assumptions about its distribution. However, the uniform special case is efficient as well as easy to implement and thus of great interest from a practical point of view. Finally, we show that quantization can be obtained as a limiting case of a soft quantizer applied to the uniform noise channel, bridging compression with and without quantization.

Gout is a common metabolic disease in which monosodium urate crystals called tophi develop. Spinal involvement in gout resulting in neural compression is unusual. We describe a case of a 64-year-old man with a history of gouty arthritis of the knee. The patient presented with thoracic myelopathy and radiculopathy. Imaging of the spine revealed an extradural mass lesion with bony erosion of the thoracic spine. A decompressive operation was performed, and a chalky white material was found. Histopathological examination confirmed a gouty tophus. The symptoms of spinal gout vary and its radiological features are not sufficiently specific to provide a definite diagnosis. Therefore, in patients with a history of gouty arthritis who present with neural compressive symptoms of the spine, spinal gout should be strongly suspected.

How to efficiently utilize the temporal features is crucial, yet challenging, for video restoration. The temporal features usually contain various noisy and uncorrelated information, and they may interfere with the restoration of the current frame. This paper proposes learning noiserobust feature representations to help video restoration. We are inspired by that the neural codec is a natural denoiser: In neural codec, the noisy and uncorrelated contents which are hard to predict but cost lots of bits are more inclined to be discarded for bitrate saving. Therefore, we design a neural compression module to filter the noise and keep the most useful information in features for video restoration. To achieve robustness to noise, our compression module adopts a spatial-channel-wise quantization mechanism to adaptively determine the quantization step size for each position in the latent. Experiments show that our method can significantly boost the performance on video denoising, where we obtain 0.13 dB improvement over BasicVSR++ with only 0.23x FLOPs. Meanwhile, our method also obtains SOTA results on video deraining and dehazing.

Learning-based denoising algorithms achieve state-of-the-art performance across various denoising tasks. However, training such models relies on access to large training datasets consisting of clean and noisy image pairs. On the other hand, in many imaging applications, such as microscopy, collecting ground truth images is often infeasible. To address this challenge, researchers have recently developed algorithms that can be trained without requiring access to ground truth data. However, training such models remains computationally challenging and still requires access to large noisy training samples. In this work, inspired by compression-based denoising and recent advances in neural compression, we propose a new compression-based denoising algorithm, which we name DeCompress, that i) does not require access to ground truth images, ii) does not require access to large training dataset - only a single noisy image is sufficient, iii) is robust to overfitting, and iv) achieves superior performance compared with zero-shot or unsupervised learning-based denoisers.

Over the past decades, there has been an explosion in the amount of available Earth Observation (EO) data. The unprecedented coverage of the Earth’s surface and atmosphere by satellite imagery has resulted in large volumes of data that must be transmitted to ground stations, stored in data centers, and distributed to end users. Modern Earth System Models (ESMs) face similar challenges, operating at high spatial and temporal resolutions, producing petabytes of data per simulated day. Data compression has gained relevance over the past decade, with neural compression (NC) emerging from deep learning and information theory, making EO data and ESM outputs ideal candidates due to their abundance of unlabeled data.In this review, we outline recent developments in NC applied to geospatial data. We introduce the fundamental concepts of NC including seminal works in its traditional applications to image and video compression domains with focus on lossy compression. We discuss the unique characteristics of EO and ESM data, contrasting them with “natural images”, and explain the additional challenges and opportunities they present. Additionally, we review current applications of NC across various EO modalities and explore the limited efforts in ESM compression to date. The advent of self-supervised learning (SSL) and foundation models (FM) has advanced methods to efficiently distill representations from vast unlabeled data. We connect these developments to NC for EO, highlighting the similarities between the two fields and elaborate on the potential of transferring compressed feature representations for machine–to–machine communication. Based on insights drawn from this review, we devise future directions relevant to applications in EO and ESM.

While neural compression has the potential to revolutionize image compression, recent studies have emphasized its ability to introduce subtle artifacts that could alter the image content.Concerned about the impact of such modifications on scientific images, this work explores the potential effects of neural compression on these images, focusing on two critical aspects: semantic understanding and forensic integrity.We use scientific image datasets to assess the performance of neural compression techniques on Visual Question Answering (VQA) and copy-move forgery detection tasks.Our findings indicate that the subtle changes introduced by neural compression do not significantly degrade the performance of state-ofthe-art solutions.In the experiments, neurally compressed images sufficiently preserved the original semantics and forensic traces.Moreover, compared to lossy techniques, e.g., JPEG compression, at similar bit-per-pixel (bpp) rates, neural compression demonstrates a superior ability to preserve both semantic content and forensic traces, even at high compression levels.Our results suggest that neural compression may provide a viable alternative to lossy compression for scientific images.

Dear Editor, We have read with great interest the paper of Yuh et al.[4] reporting a case of symptomatic hydrocephalus due to vertebrobasilar dolichoectasia (VBD) compressing the brainstem. Ectatic or pathologically enlarged vertebrobasilar arteries have been reported with increasing frequency and associated with several clinical syndromes. These include an assortment of cranial nerve syndromes, transient or permanent motor deficit, cerebellar dysfunction, central sleep apnea, ischemic stroke, hypertension, and hydrocephalus as reported in the paper by Yuh et al.[4] Others and we have also described pyramidal tract signs and cranial nerve nucleus dysfunction caused by vascular compression of the medulla oblongata by an elongated and ectatic vertebral artery and artery repositioning has proved to be an effective surgical treatment.[2,3] We agree with the authors that there is still uncertainty in the optimal treatment of VBD, since there has been no systematic review or long-term results from the different surgical interventions including microvascular decompression (MVD). However, to date, the MVD is the only interventional technique that is able to definitively treat such a pathological condition when associated with symptomatic neural compression. Neural compression by dolichoectatic vertebrobasilar arteries is an insidious clinical entity presenting several challenges to the neurosurgeon, because the operative procedure not only must be effective in relieving the symptoms, but also must preserve functioning neural and vascular structures. Therefore, the proper management should begin from the first observation of the patient affected by such an abnormality. A careful neurological examination is imperative to ensure that the patient's symptoms are attributable to vascular compression. Neuroradiological investigations are mandatory for diagnosis and surgical planning. With the advent of the magnetic resonance imaging technique, which, using specific three-dimensional sequences, has offered a good visualization of both cranial nerves and cerebral vessels, neurovascular compression disorders have been diagnosed with increasing frequency. In the clinical decision-making process, the neurosurgeon must determine whether the patient's symptoms are significant enough to warrant the surgical intervention. Finally, vessel repositioning must be performed carefully because of the elevated risk of injuring cranial nerves and the small perforating vessels of the vertebrobasilar system.[1] According to the literature and based on our experience, such a strategy can offer a complete clinical accomplishment with a functional respect of the neural structures.

The continuous advancement of photorealism in rendering is accompanied by a growth in texture data and, consequently, increasing storage and memory demands. To address this issue, we propose a novel neural compression technique specifically designed for material textures. We unlock two more levels of detail, i.e., 16× more texels, using low bitrate compression, with image quality that is better than advanced image compression techniques, such as AVIF and JPEG XL. At the same time, our method allows on-demand, real-time decompression with random access similar to block texture compression on GPUs, enabling compression on disk and memory. The key idea behind our approach is compressing multiple material textures and their mipmap chains together, and using a small neural network, that is optimized for each material, to decompress them. Finally, we use a custom training implementation to achieve practical compression speeds, whose performance surpasses that of general frameworks, like PyTorch, by an order of magnitude.

Deep learning has become a major field with many applications: from face recognition to generating images to compressing data. As a result, deep learning is becoming more and more integrated into our daily lives. We demonstrate how deep learning can be deployed for several applications in three different domains namely, improving business processes for agriculture, high-dimensional density estimation with generative models, and neural compression of data. The first domain aims to optimize the business process of a seed breeding company operating in agriculture. Therefore, we examine a dataset of white cabbage seedling images. The aim is to predict the (un)successfulness of the seedlings based on only an image. Since accurate and early predictions can terminate the seedlings stay in a growth chamber, which provides more space for other seeds to grow. Further, automating the process aids professionals. We show how a particular convolutional neural network, AlexNet, outperforms the other machine learning methods and that the model can accurately determine if a seedling is going to grow (un)successfully. Moreover, we observe that training AlexNet on earlier days generalizes to predictions on later days. The second domain concerns the utilization of generative modeling for high dimensional density estimation since this is an open problem in deep learning. We aid to close the gap in estimating the true data distribution that is modeled with generative models. More concretely, we improve model performance of a generative model, known as the normalizing flow. Therefore, we construct new methods and propose an activation function, which we call Concatenated LipSwish. The new architecture is known as i-DenseNet and outperforms its predecessor Residual Flow and other comparable flow-based models on generative and hybrid modeling performance. Finally, the third domain covers the neural compression process for images and videos. With the growing amount of data worldwide, compression, in general, has become a fundamental part of data storage and transmission. We first examine a neural image compression model, known as the mean-scale hyperprior. Even though these models are effective in practice, they do have limited capacity when it comes to optimization and generalization. Therefore, we introduce three new refinement methods that aids the compression performance and results in improved compression results per image. Additionally, we aim to optimize the latents of an already pre-trained image compression model, by keeping the networks weights fixed, and only further optimizing its latents with the refinement procedures. We show how the method can be extended to three-class rounding, outperforms the baselines, can be used to move partly along the rate-distortion curve and how it is robust to hyperparameter changes. Finally, we introduce a neural video compression model, based on scale-space flow, that allocates more bits to pre-specified regions-of-interest. We introduce two versions that are able to achieve this, namely, an implicit and a latent scaling model. In general, both models out-perform all baselines in terms of the rate-distortion performance in regions of interest and can generalize to different datasets at inference time. The latent scaling model has the best performance and can explicitly control the quantization binwidth of latent variables by only using a single model during evaluation. Further, we find that the models show a negligible performance gap when trained with synthetic region-of-interest masks, which do not correlate with the content of the video, compared to training with pixel-wise annotated masks.