Less Suboptimal Learning and Control in Variational POMPDs

How to fix model-based RL by doing the obvious.

Baris Kayalibay
Model learning fails under strong partial-observability.

Partially-Observable Markov Decision Processes (POMDPs) are central to creating autonomous systems, but their use with deep variational state-space models fails if those models are not conditioned correctly. We show the problem and a possible solution.

A Tale of Gaps

Inference Suboptimalities in Amortised Variational Inference

Maximilian Soelch
A catastrophic example of the newly discovered conditioning gap.

With variational auto-encoders (VAEs), it has become popular to approximate Bayesian inference with neural networks. This scales Bayesian inference to large datasets and deep generative models at the cost of suboptimal inference. This post discusses the taxonomy researchers have developed over the past years and adds a piece to the puzzle that we recently discovered – the conditioning gap.

Richard Kurle
Online variational Bayes with Bayesian forgetting (left) and without adaptation (right).

This work addresses continual learning for non-stationary data, using Bayesian neural networks and memory-based online Variational Bayes. We represent the posterior approximation of the network weights by a diagonal Gaussian distribution and a complementary memory of raw data. This raw data corresponds to likelihood terms that cannot be well approximated by the Gaussian. We introduce a novel method for sequentially updating both components of the posterior approximation. Furthermore, we propose Bayesian forgetting and a Gaussian diffusion process for adapting to non-stationary data. The experimental results show that our update method improves on existing approaches for streaming data. Additionally, the adaptation methods lead to better predictive performance for non-stationary data.

Layerwise learning for quantum neural networks

Solving vanishing gradients in quantum neural networks

Andrea Skolik
Incrementally training layers solves vanishing gradients in quantum neural networks

We introduce a training strategy that addresses vanishing gradients in quantum neural networks, and makes better use of the resources provided by noisy intermediate-scale quantum (NISQ) devices.

Learning to Fly

Deep Model-Based Reinforcement Learning in the real world

A self-built drone controlled onboard by a learnt policy optimised in a learnt simulation

In this work we show how to learn neural network policy for thrust-attitude control of a self-built drone via model-based reinforcement learning and variational inference methods.

Learning Hierarchical Priors in VAEs

A constrained optimisation approach

Alexej Klushyn
Graph-based interpolation of human motion

We address the issue of learning informative latent representations of data. In the normal VAE, the latent space prior is a standard normal distribution. This over-regularises the posterior distribution, resulting in latent representations that do not represent well the structure of the data. This post, describing our 2019 NeurIPS publication, proposes and demonstrates a solution by using an hierarchical latent space prior.

How to Learn Functions on Sets with Neural Networks

And how to choose your aggregation

Maximilian Soelch
The basic Deep Sets architecture for set functions: embed, aggregate, process.

If you input a vector of data in a neural network, the order of the elements matters. But ssometimes the order doesn't carry any useful inforamtion: sometimes we are interested in working on sets of data. In this post, we will look into functions on sets, and how to learn them with the help of neural networks.

Approximate Geodesics for Deep Generative Models

How to efficiently find the shortest path in latent space

Nutan Chen
The graph of the fashion MNIST dataset in a 2D latent space, along with the magnification factor.

Neural samplers such as variational autoencoders (VAEs) or generative adversarial networks (GANs) approximate distributions by transforming samples from a simple random source—the latent space—to a more complex distribution—corresponding to the distribution from which is data is sampled.

Typically, the data set is sparse, while the latent space is compact. Consequently, points that are separated by low-density regions in observation space will be pushed together in latent space, and the spaces get distored. In effect, stationary distances in the latent space are poor proxies for similarity in the observation space. How can this be solved?

Georgi Dikov

Intuition and experience. Probably, that's the answer you would get if you happen to ask deep learning engineers how they chose the hyperparameters of a neural network. Depending on their familiarity with the problem, they might have done some good three to five full dataset runs until a satisfactory result popped up. Now, you might say, we surely could automate this, right, after all we do it implicitly in our heads? Well, yes, we definitely could, but should we?

Deep Variational Bayes Filter

DVBF: filter to learn what to filter

Maximilian Soelch
Learnt latent representation of a swinging pendulum

Machine-learning algorithms thrive in environments where data is abundant. In the land of scarce data, blessed are those who have simulators. The recent successes in Go or Atari games would be much harder to achieve without the ability to parallelise millions of perfect game simulations.


Machine Learning Research Lab, Volkswagen Group

The front view of the lab building

Established in 2016, Volkswagen Group Machine Learning Research Lab, located in Munich, was set up as a fundamental research lab on topics close to what we think artificial intelligence is about: machine learning and probabilistic inference, efficient exploration, and optimal control.