Week 5 into GSoC¶
Today (July 12, 2020) marks mid-way between phase 1 and phase 2 of GSoC coding period. Some good news, I have cleared Phase 1 evaluations. My mentor Maxim Kochurov is amazing. Not only we explore different ideas related to the project but also we have great conversations not related to GSoC.
Most of the time spent this week was into resolving issues with Full Rank ADVI PR #289. Here is the long story short -
- Mean Field ADVI PR uses JointDistributionSequential and on the same paths, I added an interface of Full Rank ADVI.
- That approach was wrong because I misunderstood model flattening as variable flattening. And this leads to all sort of shape issues.
- After learning about model flattening, I integrated it into the PR.
- ADVI worked but losses contains lots of nans. As suspected, it was due to missing transformations.
- It took time to figure out how to setup transformations but later selected manually doing it.
- I was unable to make transformations generalized enough. Conditional statements and
tf.conddoes not seem to work either because at hand, we do not have values to transform inside bijectors. Later I plan to write a proposal mentioning the issue to PyMC-devs.
- Then after transformations, the major issue was of cholesky decomposition failure. And this comment is a life saviour one.
- It took time to experiment with each option with park_bias_model and the last one worked. As stated, cholesky decomposition is unstable to
tf.float32. So casting all the parameters to
tf.float64resolved the issue. I also want to thank Tirth Patel for helping me on the way to resolve dtype issues.
- Finally, I am all set to see the Full Rank ADVI PR getting merged.
Flattening and Full Rank ADVI in PyMC4
Testing issues with transformations
Failed at an attempt to generalize transformations
Good news regarding the progress bars but I cannot do
num_steps = 10_000 def trace_fn(trace): tf.cond( tf.math.mod(trace.step, 100) == 0, lambda: tf.print(trace.step, "/", num_steps, "Loss:", trace.loss), lambda: tf.print("", end="\r") ) return trace.loss # Pass this trace_fn to pm.fit()
I am planning to add convergence checks and progress bars for next week. Stay tuned.
I am thankful to my mentor for his constant guidance and pymc-devs for being such a supportive community.
Thank you for reading!
With , Sayam