Week 3 into GSoC¶
Today (June 21, 2020) marks the ending of week 3 of GSoC coding period. Let me summarize what I did for this week. Following up the work done in week 2, I went on exploring
ArviZ InferenceData and
tfp.vi module to learn how to deal with shapes, convergence checks and optimizers.
- Adding a new axis to samples resolved the shape issues. By this,
number of chainsparameter is set to 1 and all the priors are handled perfectly.
- The convergence checks did not seem to work properly. (As pointed out by my mentor, we need to adjust window size).
- The default tensorflow optimizers also did not lead to convergence really well. Trying out the default values taken from PyMC3, I got really good results. This motivated me to write
updatesmodule for Variational Inference Interface.
- I applied inverse of bijectors to transformed parameters to match support in bounded space. But this approach is wrong. I need to handle this using deterministics callback which I will do in coming week.
- I had also written tests during this interval.
- From last week, I did not get quite good results while experimenting with Mean Field ADVI in TFP, PyMC3 and PyMC4. As suggested by my mentor, setting a common random seed and same optimizer leads to very good results . (gist)
Whatever experiments I perform, I polish them out and share through GitHub gists. I do not why but I started loving to share code through GitHub gists rather than Colab or GitHub repo.
Here is the notebook -
Tasks for week 4¶
Phase 1 Evaluations are coming up. So, I need to sync work with my proposed timeline and spend time summarizing all the results in a single notebook. My tasks for week 4 -
- Write tests for conjugate normal models with known mean/variance.
np.testing.assert_allclose. (I misunderstood how this parameter works)
- Complete docs for optimizers by adding
**kwargsoption and writing corresponding maths equations.
- Properly configure convergence checks and add an example to quickstart notebook.
- Configure autobatching. (I need to understand how this works for mcmc)
- Integrate Deterministics callbacks.
- Complete Full Rank/Low Rank Approximation. (This will take time)
- Configure Minibatches.
- Update quick_start notebook with respect to all changes above.
If I will be able to complete above mentioned tasks in time, I would love to -
- Have another implementation of Mean Field ADVI using
- Play around with Mean Field ADVI on Eight Schools notebook.
- Resolve a warning from
tfp.vimodule regarding repetitive use of
tf.functionin a loop. Maybe using
tf.while_loopwill solve this but I am not sure.
- Experiment with MeanField/ FullRank/ LowRank ADVI in TFP, PyMC3, PyMC4 inspired from ColCarroll's notebook. I am already getting excited to play around with this after having all APIs set correctly.
I look forward to learn how Variational AutoEncoders are implemented in PyMC3 and try implementing that in PyMC4.
I am thankful to my mentor for his constant guidance and pymc-devs for being such a supportive community.
Thank you for reading!
With , Sayam