Austin Tripp's website (Posts about machine learning)https://austintripp.ca/enContents © 2024 <a href="mailto:austin.james.tripp[at]gmail.com">Austin Tripp</a> MIT LicenseMon, 14 Oct 2024 09:25:10 GMTNikola (getnikola.com)http://blogs.law.harvard.edu/tech/rss- Thoughts on Google Vizierhttps://austintripp.ca/blog/2024-10-13-vizier-bayesopt/Austin Tripp<div><p>Vizier,
described in a <a href="http://arxiv.org/abs/2408.11527">recent paper from Google</a>,
is a black-box optimization algorithm deployed
for "numerous research and production systems at Google".
Allegedly, this one algorithm works well on a wide range of settings
(something which the "no-free-lunch-theorem" suggests might not be possible).
In this post I provide my thoughts on what key design decisions likely make this
algorithm work well.</p>
<p><a href="https://austintripp.ca/blog/2024-10-13-vizier-bayesopt/">Read more…</a> (7 min remaining to read)</p></div>bayesian optimizationmachine learninghttps://austintripp.ca/blog/2024-10-13-vizier-bayesopt/Sat, 12 Oct 2024 23:00:00 GMT
- Problems with the top-k diversity metric for diverse optimizationhttps://austintripp.ca/blog/2024-10-07-problems-with-topk-diversity/Austin Tripp<div class="cell border-box-sizing text_cell rendered" id="cell-id=b4485680-e872-4498-b53b-cfc0bcde2315"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<div class="alert alert-block alert-info">
<b>NOTE</b>
this blog post can be run as a jupyter notebook. I re-ordered the cells to make it easier to read; to re-produce all the plots see instructions at the end of the post.
</div>
<h3 id="Background">Background<a class="anchor-link" href="https://austintripp.ca/blog/2024-10-07-problems-with-topk-diversity/#Background">¶</a></h3><p><em>"Diverse optimization"</em> has been a popular topic in machine learning conferences
for a few years now, particularly in the "AI for drug discovery" sub-field.
In this context, the goal of "optimization" algorithms is to suggest
promising drug candidates,
where "promising" means maximizing one (or more) objective functions.
An example of an objective function could be a docking score
(an approximate simulation of the interactions between a protein and a molecule).
"Diverse" optimization further requires that an algorithm produce multiple
distinct candidate solutions.
This is typically desired when the objective functions don't fully capture
everything we want (for example, a drug candidate also having low toxicity).
The hope is that a diverse set of candidates will have a higher chance of
one useful candidate compared to a non-diverse sets.</p>
<p><a href="https://austintripp.ca/blog/2024-10-07-problems-with-topk-diversity/">Read more…</a> (7 min remaining to read)</p></div></div></div>machine learningstatisticshttps://austintripp.ca/blog/2024-10-07-problems-with-topk-diversity/Sun, 06 Oct 2024 23:00:00 GMT
- Sparse Matrices: 5 Tips and Trickshttps://austintripp.ca/blog/2018/09/12/sparse-matrices-tips1/Austin Tripp<div><p>Over the course of my internship at the online shopping company <a href="https://www.wish.com/">Wish</a>, I have dealt a lot with a lot of data in the form of sparse matrices, specificaly in the form of item interaction matrices for customer data. In doing so, I have made heavy use of <a href="https://docs.scipy.org/doc/scipy/reference/sparse.html">scipy's sparse matrices library</a>.
Here are 5 tricks that I have learned.</p>
<p><a href="https://austintripp.ca/blog/2018/09/12/sparse-matrices-tips1/">Read more…</a> (2 min remaining to read)</p></div>frontpagemachine learninghttps://austintripp.ca/blog/2018/09/12/sparse-matrices-tips1/Tue, 11 Sep 2018 23:00:00 GMT
- An Overview of Gradient Boosting and Popular Libraries for it.https://austintripp.ca/blog/2018/07/17/gradient-boosting/Austin Tripp<div><p>Everybody doing machine learning wants the best models possible.
The aim of this blog article is the following:</p>
<ol>
<li>To provide an introduction to the machine learning technique known as <em>boosting</em>, and specifically <em>gradient boosting</em>.</li>
<li>To compare/contrast boosting with other ensemble methods, such as <em>bagging</em></li>
<li>To explain and compare several popular gradient boosting frameworks, specifically <em>XGBoost</em>, <em>CatBoost</em>, and <em>LightGBM</em>.</li>
</ol>
<p><a href="https://austintripp.ca/blog/2018/07/17/gradient-boosting/">Read more…</a> (7 min remaining to read)</p></div>frontpagemachine learninghttps://austintripp.ca/blog/2018/07/17/gradient-boosting/Mon, 16 Jul 2018 23:00:00 GMT
- Turning Adam Optimization into SGDhttps://austintripp.ca/blog/2018/07/02/adam-to-sgd/Austin Tripp<div><h3 id="motivation">Motivation</h3>
<p>This strange question came up when working on a machine learning project to generate embeddings.
Working with the version of Pytorch available on our DGX (similar to version 0.3.1), I found there was an optimizer called <em>SparseAdam</em> but not one called <em>SparseSGD</em>.
Since what I really wanted to do was use SGD, I wondered: could I turn the Adam optimizer into an SGD optimizer by setting the hyperparameters \(\beta_1\), \(\beta_2\), and \(\epsilon\)?</p>
<p><a href="https://austintripp.ca/blog/2018/07/02/adam-to-sgd/">Read more…</a> (1 min remaining to read)</p></div>frontpagemachine learninghttps://austintripp.ca/blog/2018/07/02/adam-to-sgd/Sun, 01 Jul 2018 23:00:00 GMT