A list of Python-based MCMC packages. Also here's a nice list of MCMC algorithms.
A Python Approximate Bayesian Computing (ABC) Population Monte Carlo (PMC) implementation based on Sequential Monte Carlo (SMC) with Particle Filtering techniques.
Features:
- Entirely implemented in Python and easy to extend
- Follows Beaumont et al. 2009 PMC algorithm
- Parallelized with muliprocessing or message passing interface (MPI)
- Extendable with k-nearest neighbour (KNN) or optimal local covariance matrix (OLCM) pertubation kernels (Fillipi et al. 2012)
- Detailed examples in IPython notebooks
astroABC is a Python implementation of an Approximate Bayesian Computation Sequential Monte Carlo (ABC SMC) sampler for parameter estimation.
Key features
- Parallel sampling using MPI or multiprocessing
- MPI communicator can be split so both the sampler, and simulation launched by each particle, can run in parallel
- A Sequential Monte Carlo sampler (see e.g. Toni et al. 2009, Beaumont et al. 2009, Sisson & Fan 2010)
- A method for iterative adapting tolerance levels using the qth quantile of the distance for t iterations (Turner & Van Zandt (2012))
- Scikit-learn covariance matrix estimation using Ledoit-Wolf shrinkage for singular matrices
- A module for specifying particle covariance using method proposed by Turner & Van Zandt (2012), optimal covariance matrix for a multivariate normal perturbation kernel, local covariance estimate using scikit-learn KDTree method for nearest neighbours (Filippi et al 2013) and a weighted covariance (Beaumont et al 2009)
- Restart files output frequently so an interrupted run can be resumed at any iteration
- Output and restart files are backed up every iteration
- User defined distance metric and simulation methods
- A class for specifying heterogeneous parameter priors
- Methods for drawing from any non-standard prior PDF e.g using Planck/WMAP chains
- A module for specifying a constant, linear, log or exponential tolerance level
- Well-documented examples and sample scripts
bmcmc is a general purpose mcmc package which should be useful for Bayesian data analysis. It uses an adaptive scheme for automatic tuning of proposal distributions. It can also handle hierarchical Bayesian models via Metropolis-Within-Gibbs scheme.
Package which enables parameter inference using an Approximate Bayesian Computation (ABC) algorithm. The code was originally designed for cosmological parameter inference from galaxy clusters number counts based on Sunyaev-Zel’dovich measurements. In this context, the cosmological simulations were performed using the NumCosmo library.
Parallel nested sampling in python.
A Dynamic Nested Sampling package for computing Bayesian posteriors and evidences.
Edward2 is a probabilistic programming language in TensorFlow and Python. It extends the TensorFlow ecosystem so that one can declare models as probabilistic programs and manipulate a model's computation for flexible training, latent variable inference, and predictions.
- Original project: Edward (Tran et al. (2016))
ELFI is a statistical software package written in Python for likelihood-free inference (LFI) such as Approximate Bayesian Computation (ABC). The term LFI refers to a family of inference methods that replace the use of the likelihood function with a data generating simulator function. ELFI features an easy to use generative modeling syntax and supports parallelized inference out of the box.
emcee is an MIT licensed pure-Python implementation of Goodman & Weare’s Affine Invariant Markov chain Monte Carlo (MCMC) Ensemble sampler. It's designed for Bayesian parameter estimation and it's really sweet!
A simple Hamiltonian MCMC sampler.
An adaptive basin-hopping Markov-chain Monte Carlo algorithm for Bayesian optimisation. Python implementation of the hoppMCMC algorithm aiming to identify and sample from the high-probability regions of a posterior distribution. The algorithm combines three strategies: (i) parallel MCMC, (ii) adaptive Gibbs sampling and (iii) simulated annealing. Overall, hoppMCMC resembles the basin-hopping algorithm implemented in the optimize module of scipy, but it is developed for a wide range of modelling approaches including stochastic models with or without time-delay.
kombine is an ensemble sampler built for efficiently exploring multimodal distributions. By using estimates of ensemble’s instantaneous distribution as a proposal, it achieves very fast burnin, followed by sampling with very short autocorrelation times.
Multi-Core Markov-Chain Monte Carlo (MC3) is a powerful Bayesian-statistics tool that offers:
- Levenberg-Marquardt least-squares optimization.
- Markov-chain Monte Carlo (MCMC) posterior-distribution sampling following the:
- Metropolis-Hastings algorithm with Gaussian proposal distribution,
- Differential-Evolution MCMC (DEMC), or
- DEMCzs (Snooker).
Flexible and efficient Python implementation of the nested sampling algorithm. This implementation is geared towards allowing statistical physicists to use this method for thermodynamic analysis but is also being used by astrophysicists.
This implementation uses the language of statistical mechanics (partition function, phase space, configurations, energy, density of states) rather than the language of Bayesian sampling (likelihood, prior, evidence). This is simply for convenience, the method is the same.
The package goes beyond the bare implementation of the method providing:
- built-in parallelisation on single computing node (max total number of cpu threads on a single machine)
- built-in Pyro4-based parallelisation by distributed computing, ideal to run calculations on a cluster or across a network
- ability to save and restart from checkpoint binary files, ideal for very long calculations
- scripts to compute heat capacities and perform error analysis integration with the MCpele package to implement efficient Monte Carlo walkers.
Pure Python, MIT-licensed implementation of nested sampling algorithms. Nested Sampling is a computational approach for integrating posterior probability in order to compare models in Bayesian statistics. It is similar to Markov Chain Monte Carlo (MCMC) in that it generates samples that can be used to estimate the posterior probability distribution. Unlike MCMC, the nature of the sampling also allows one to calculate the integral of the distribution. It also happens to be a pretty good method for robustly finding global maxima.
No-U-Turn Sampler (NUTS) for python This package implements the No-U-Turn Sampler (NUTS) algorithm 6 from the NUTS paper (Hoffman & Gelman, 2011).
Python library for working with Probabilistic Graphical Models.
Fork of Daniel Foreman-Mackey's emcee to implement parallel tempering more robustly. As far as possible, it is designed as a drop-in replacement for emcee. If you're trying to characterise awkward, multi-modal probability distributions, then ptemcee is your friend.
MPI enabled Parallel Tempering MCMC code written in Python.
Python class that coordinates an MPI implementation of parallel tempering. Supports a fully parallelised implementation of parallel tempering using mpi4py (message passing interface for python). Each replica runs as a separate parallel process and they communicate via an mpi4py object. To minimise message passing the replicas stay in place and only the temperatures are exchanged between the processes. It is this exchange of temperatures that ptmpi handles.
pyABC is a framework for distributed, likelihood-free inference. That means, if you have a model and some data and want to know the posterior distribution over the model parameters, i.e. you want to know with which probability which parameters explain the observed data, then pyABC might be for you.
All you need is some way to numerically draw samples from the model, given the model parameters. pyABC “inverts” the model for you and tells you which parameters were well matching and which ones not. You do not need to analytically calculate the likelihood function.
pyABC runs efficiently on multi-core machines and distributed cluster setups. It is easy to use and flexibly extensible.
A Python implementation of the MT-DREAM(ZS) algorithm.
This package is a straight-forward port of the functions
hmc2.mandhmc2_opt.mfrom the MCMCstuff matlab toolbox written by Aki Vehtari. The code is originally based on the functions hmc.m from the netlab toolbox written by Ian T Nabney. The portion of algorithm involving "windows" is derived from the C code for this function included in the Software for Flexible Bayesian Modeling written by Radford Neal.
PyJAGS provides a Python interface to JAGS, a program for analysis of Bayesian hierarchical models using Markov Chain Monte Carlo (MCMC) simulation.
PyMC3 is a probabilistic programming module for Python that allows users to fit Bayesian models using a variety of numerical methods, most notably Markov chain Monte Carlo (MCMC) and variational inference (VI). Its flexibility and extensibility make it applicable to a large suite of problems. Along with core model specification and fitting functionality, PyMC3 includes functionality for summarizing output and for model diagnostics.
- Tutorials:
- [Bayesian Modelling in Python](https://github.com/markdregan/ Bayesian-Modelling-in-Python)
- Using PyMC3
- [Tutorial 5a: Parameter estimation with Markov chain Monte Carlo](http:// bebi103.caltech.edu.s3-website-us-east-1.amazonaws.com/2017/tutorials/ t5a_mcmc.html)
- Books:
- [Bayesian Methods for Hackers](https://github.com/CamDavidsonPilon/ Probabilistic-Programming-and-Bayesian-Methods-for-Hackers)
- [Statistical Rethinking with Python and PyMC3](https://github.com/ aloctavodia/Statistical-Rethinking-with-Python-and-PyMC3)
Simple implementation of the Metropolis-Hastings algorithm for Markov Chain Monte Carlo sampling of multidimensional spaces. The implementation is minimalistic. All that is required is a funtion which accepts an iterable of parameter values, and returns the positive log likelihood at that point.
A simple MCMC framework for training Gaussian processes adding functionality to GPy.
MultiNest is a program and a sampling technique. As a Bayesian inference technique, it allows parameter estimation and model selection. Recently, MultiNest added Importance Nested Sampling which is now also supported. The efficient Monte Carlo algorithm for sampling the parameter space is based on nested sampling and the idea of disjoint multi-dimensional ellipse sampling. For the scientific community, where Python is becoming the new lingua franca (luckily), I provide an interface to MultiNest.
pysmc is a Python package for sampling complicated probability densities using the celebrated Sequential Monte Carlo method.
PyStan provides an interface to Stan, a package for Bayesian inference using the No-U-Turn sampler, a variant of Hamiltonian Monte Carlo.
Sampyl is a Python library implementing Markov Chain Monte Carlo (MCMC) samplers in Python. It’s designed for use in Bayesian parameter estimation and provides a collection of distribution log-likelihoods for use in constructing models.
A Python package for Approximate Bayesian Computation.
A Statistical Parameter Optimization Tool for Python. SPOTPY is a Python framework that enables the use of Computational optimization techniques for calibration, uncertainty and sensitivity analysis techniques of almost every (environmental-) model.
Nested Sampling Development Framework & UltraNest A Pythonic implementation of the Nested Sampling integration algorithm for Bayesian model comparison and parameter estimation. This package provides multiple implementations for constrained drawing functions, and a test suite to evaluate the correctness, accuracy and efficiency of various implementations. The source code consists of three components:
- A Modular framework for nested sampling algorithms (nested_sampling) and their development
- Test framework to evaluate the performance and accuracy of algorithms (testsuite)
- UltraNest - A fast C implementation of a mixed RadFriends/MCMC nested sampling algorithm
zeus is a pure-Python implementation of the Ensemble Slice Sampling method.
- Fast & Robust Bayesian Inference,
- No hand-tuning,
- Excellent performance in terms of autocorrelation time and convergence rate,
- Scale to multiple CPUs without any extra effort.