Deep Energy-Based Generative Models

Energy-based generative modeling and learning have been successful in representing different types of data formats, such as images [1, 2, 3], videos [4, 5], 3D voxels [6, 7], and trajectories [8]. Their successes are attributed to the invention of the energy-based Generative ConvNet [1] framework initially proposed by Song-Chun Zhu’s group from UCLA in 2016. The Generative ConvNet parameterizes its energy function by a bottom-up deep neural network and bridges the gap between deep learning and energy-based learning.

The synthesized results by energy-based generative ConvNet family
The synthesized results by energy-based generative ConvNet family
Figure 1: The synthesized results generated by the energy-based Generative ConvNet family

Generative PointNet

Recently, UCLA and Baidu propose a novel energy-based representational model for unordered point clouds by designing an input-permutation-invariant energy function for the…

Generative Modeling of Data

Statistical modeling of high-dimensional signals, such as images, videos, and 3D shapes, is a very challenging task in computer vision and machine learning. Even though Generative Adversarial Net (GAN) and Variational Auto-Encoder (VAE) have become very popular in the task of data generation, they still have a few drawbacks such as mode collapse in GANs and posterior collapse in VAE. Most importantly, these models require an assisting network to train the generator, which makes the model less natural. Most importantly, GANs and VAEs are not able to provide explicit density functions of the data. In other words, their data likelihood…

I love EBM

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store