Notes from Oxford BO mini-conference
machine learning
bayesian optimization
🤖 AI-assisted ✏️ Quick post 🪞 Low originality 🌱 New
Last week (2026-09-29) I had the pleasure of attending a mini-conference in Oxford on Bayesian optimization (BO), focusing mostly on high dimensional BO and the recent paper proposing linear kernels on transformed inputs (Doumont et al. 2026). I gave a talk on how I thought high-dimensional BO is the wrong problem to study (post here).
Here are my main takeaways:
- Overall I think the field is moving towards studying more fundamental mechanisms (the main recommendation of my talk).
- Numerically tiny gradients make a lot of empirical BO very difficult, and confound past results. In particular, it is very easy to stop acquisition function optimization too early and pick a sub-optimal point. This is hard to avoid unless you are deliberately careful, so I suspect some (maybe much) of the poor high-dimensional BO performance reported in the past comes from this.
- People are still very interested in latent space BO methods. I’m not a believer myself, and I should write a post explaining why.
- BO with linear kernels (in a transformed space) opens the possibility for semi-analytic optimization methods exploiting the known structure of the problem (e.g. for most standard acquisition functions, all maximizers will be on the Pareto front of high predicted mean vs. high predicted variance).
Overall it was a nice conference to attend and a good community of people to have a pint with 😊🍻
References
Doumont, Colin, Donney Fan, Natalie Maus, Jacob R. Gardner, Henry Moss, and Geoff Pleiss. 2026. “We Still Don’t Understand High-Dimensional Bayesian Optimization.” In Proceedings of the 29th International Conference on Artificial Intelligence and Statistics, edited by Emtiyaz Khan, Yingzhen Li, Arno Solin, and Aaditya Ramdas, vol. 300. Proceedings of Machine Learning Research. PMLR. https://proceedings.mlr.press/v300/doumont26a.html.