FNN-VAE for noisy time series forecasting
In the last part of this mini-series on forecasting with false nearest neighbors (FNN) loss, we replace the LSTM autoencoder from the previous post by a convolutional VAE, resulting in equivalent prediction performance but significantly lower training time. In addition, we find that FNN regularization is of great help when an underlying deterministic process is obscured by substantial noise.
Sigrid Keydana - July 30, 2020
This post did not end up quite the way I’d imagined. A quick follow-up on the recent Time series prediction with FNN-LSTM, it was supposed to demonstrate how noisy time series (so common in practice) could profit from a change in architecture: Instead of FNN-LSTM, an LSTM autoencoder regularized by false nearest neighbors (FNN) loss, use FNN-VAE, a variational autoencoder constrained by the same. However, FNN-VAE did not seem to handle noise better than FNN-LSTM. No plot, no post, then?
On the other hand – this is not a scientific study, with hypothesis and experimental setup all preregistered; all that really matters is if there’s something useful to report. And it looks like there is.
Firstly, FNN-VAE, while on par performance-wise with FNN-LSTM, is far superior in that other meaning of “performance”: Training goes a lot faster for FNN-VAE.
Secondly, while we don’t see much difference between FNN-LSTM and FNN-VAE, we do see a clear impact of using FNN loss. Adding in FNN loss strongly reduces mean squared error with respect to the underlying (denoised) series – especially in the case of VAE, but for LSTM as well. This is of particular interest with VAE, as it comes with a regularizer out-of-the-box – namely, Kullback-Leibler (KL) divergence.
Of course, we don’t claim that similar results will always be obtained on other noisy series; nor did we tune any of the models “to death”. For what could be the intent of such a post but to show our readers interesting (and promising) ideas to pursue in their own experimentation?
Read more at Posit AI Blog: FNN-VAE for noisy time series forecasting