Latent Space COWBOYS: a VAE-BO method I can actually buy into!

I am generally pessimistic about BO (Bayesian optimization) methods which use VAE embeddings as part of the model. Mostly this is because distance in a VAE's latent space has no reason to correlate with distances in property space (unless trained for it), and because training with labels is basically deep kernel learning which usually overfits¹. However, I recently came across the paper "Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spaces"² and quite liked the idea, ending my long-standing streak of disliking every VAE-BO paper I read.

The key idea is basically to use the VAE as a tool for acquisition function optimization only: actual BO modelling is handled by an external model. This allows (some) continuous optimization methods to be used without requiring that the VAE's latent space is a key feature of the model. They use the VAE + GP combo essentially as a likelihood function on the latent variable $z$³, and apply MCMC methods normally meant for posterior inference in Bayesian models.

Overall the idea is sound and I appreciated the experiments. That being said, I would have liked it if Tanimoto GP + Graph GA optimizer was included as a baseline to compare latent space vs non-latent space methods, but that doesn't fundamentally change my evaluation of the paper.