# Three Variations on Variational Autoencoders

**Authors:** R. I. Cukier  
**arXiv:** [2212.04451](https://arxiv.org/abs/2212.04451)  
**Date:** 2022-12-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2212.04451v2/

## Key points

- Variational autoencoders (VAEs) are generative probabilistic latent-variable models used for inference based on known data.
- Three variations of VAEs were developed in the study by introducing additional parameterized encoder/decoder pairs and an extra fixed encoder.
- Parameters of the encoders/decoders were learned using neural networks, with the fixed encoder obtained through probabilistic PCA.
- One variation led to an Evidence Upper Bound (EUBO), which can be used alongside the Evidence Lower Bound (ELBO) to assess VAE model convergence.
- The refined version of VAE, VAE_A, incorporated two learnable encoders and a fixed encoder, requiring Probabilistic PCA for representation of the latent space.
- Mathematical formulations by Roweis connected latent and data space in P-PCA using descending-size-ordered eigenvalues lλ of the diagonal matrix L(NN×z).
- Means zμx and variances zΣx of the encoder Yϕz|x needed to be learned as parameters.
- These variations offer different approaches to enhancing inference based on known data, improving performance and convergence of VAE models for various machine learning and data analysis applications.

## Summary

Variational autoencoders (VAEs) are a class of generative probabilistic latent-variable models used for inference based on known data. In this study, three variations on VAEs were developed by introducing additional parameterized encoder/decoder pairs and an extra fixed encoder. The parameters of the encoders/decoders were learned using neural networks, with the fixed encoder obtained through probabilistic-PCA. These variations were compared to the Evidence Lower Bound (ELBO) approximation of the original VAE. One of the variations led to an Evidence Upper Bound (EUBO), which can be utilized alongside the ELBO to assess the convergence of the VAE model. The refined version of VAE, referred to as VAE_A, incorporated two learnable encoders (Yϕz|x) and a fixed encoder (Wz|x). To make this variation explicit, a model for W was required. Therefore, Probabilistic PCA (P-PCA) was employed as a generalization method for PCA to represent the latent space. The connection between latent and data space in P-PCA was established through mathematical formulations provided by Roweis. To obtain the desired PCA representation of RC, the descending-size-ordered eigenvalues lλ of the diagonal matrix L(NN×z) were used. The means zμx and variances zΣx of the encoder Yϕz|x were parameters that needed to be learned. In conclusion, these three variations on variational autoencoders offer different approaches to improving inference based on known data. By introducing additional encoders/decoders and utilizing methods like P-PCA, researchers can enhance the performance and convergence of VAE models for various applications in machine learning and data analysis.