Markov Chain Monte-Carlo
Markov chain Monte Carlo (MCMC) is a technique for estimating by simulation the expectation of a statistic in a complex model.
Two good blog articles:
- https://jeremykun.com/2015/04/06/markov-chain-monte-carlo-without-all-the-bullshit/
- https://machinelearningmastery.com/markov-chain-monte-carlo-for-probability/
This is one of the most influential ideas of the 20th century.
https://www.youtube.com/watch?v=12eZWG0Z5gY&t=12s&ab_channel=mathematicalmonk Goal: Sample from , or approximate
I found this 50 page explanation: https://arxiv.org/pdf/1909.12313.pdf
This is a good blog: https://jeremykun.com/2015/04/06/markov-chain-monte-carlo-without-all-the-bullshit/ We can use MCMC to approximate some some sort of unknown distribution, by modelling it as some sort of Random Walk on a Graph, where we can apply the Metropolis-Hastings Algorithm?
How is this different from purely monte-carlo method?
Referenced from the original article for Reinforcement Learning: https://lilianweng.github.io/posts/2018-02-19-rl-overview/ Referenced from the original article for Reinforcement Learning:
Mentioned also in the original GAN paper.
Flavors of MCMC
Replica-Exchange MCMC / Parallel Tempering
This “is a standard approach to maintaining a population of configurations” from FormulaZero.
Hit-and-Run Algorithm
Hit-and-run is a Markov chain Monte Carlo (MCMC) sampling technique that iteratively generates a sequence of points in a set by taking steps of random length in randomly generated directions. Hit-and-run can be applied to virtually any bounded region in {\Re^n} , and has nice convergence properties.Jan 23, 2016