pysgmcmc Documentation - Read the Docs...pysgmcmc Documentation 3.2Getting started 3.2.1Samplers on simple energy functions In this notebook, we ﬁt a simple Banana energy function

pysgmcmc Documentation

Moritz Freidank

Jul 30, 2018

Contents:

1 PYSGMCMC 3

2 Features 5

3 Install 73.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.1.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.1.2 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.2 Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.2.1 Samplers on simple energy functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.2.2 Simple sinc example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.3 API Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3.1 Samplers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3.2 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.3.3 Mixin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3.4 SGHMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3.5 SGLD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3.6 Data Batches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3.7 Progressbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3.8 Torch Utils . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 Indices and tables 19

Python Module Index 21

i

ii


This package provides out-of-the-box implementations of various state-of-the-art Stochastic Gradient Markov ChainMonte Carlo sampling methods for pytorch.

Contents: 1


2 Contents:

CHAPTER 1

PYSGMCMC

PYSGMCMC is a Python framework for Bayesian Deep Learning that focuses on Stochastic Gradient Markov ChainMonte Carlo methods.

3

https://travis-ci.org/MFreidank/pysgmcmc

https://coveralls.io/github/MFreidank/pysgmcmc?branch=pytorch

https://www.codacy.com/app/MFreidank/pysgmcmc?utm_source=github.com&utm_medium=referral&utm_content=MFreidank/pysgmcmc&utm_campaign=Badge_Grade


4 Chapter 1. PYSGMCMC

CHAPTER 2

Features

• Complex samplers as black boxes, computing the next sample with corresponding costs of any MCMC sampleris as easy as:

sample, cost = next(sampler)

• Based on pytorch that provides:

– efficient numerical computation via data flow graphs

– flexible computation environments (CPU/GPU support, desktop/server/mobile device support)

– Linear algebra operations

5

https://www.pytorch.org/


6 Chapter 2. Features

CHAPTER 3

Install

The quick way:

pip3 install git+https://github.com/MFreidank/pysgmcmc@pytorch

3.1 Introduction

3.1.1 Purpose

PySGMCMC is a Python package that enables users to fit Bayesian models using Markov chain Monte Charlo(MCMC) sampling methods in settings where only noisy gradient information is available.

Due to the stochastic nature of the gradient, these methods are also called Stochastic Gradient Markov Chain MonteCarlo (SGMCMC) methods.

One particular target audience for our samplers are Bayesian Deep Learning practitioners. In Bayesian Deep Learningdatasets quickly become large, which makes it intractable to compute the gradient of a model on the whole dataset. Acommon remedy for this is to sub-sample the dataset into (mini-) batches.

3.1.2 Features

• Modern MCMC solutions applicable when fitting Bayesian models to sub-sampled datasets.

• Keras (Tensorflow) as the computational backend, which allows for efficient numeric calculation, possibly onGPUs and automatic gradient calculation.

• Flexible: painless application of any of our samplers to your estimation problem.

7


3.2 Getting started

3.2.1 Samplers on simple energy functions

In this notebook, we fit a simple Banana energy function using our sampler interface. We plot resulting traces for bothSGHMC and SGLD directly on top of the contours of our Banana function to visualize our sampling process.

In [78]: % matplotlib notebookfrom itertools import isliceimport syssys.path.insert(0, "../../../")

import matplotlib.pyplot as pltimport numpy as npimport torch

from pysgmcmc.samplers.sghmc import SGHMCfrom pysgmcmc.samplers.sgld import SGLD

def banana_nll(x):ll = -(

0.5 * (0.01 * x[0] ** 2 +(x[1] + 0.1 * x[0] ** 2 - 10) ** 2)

)return -ll

def plot_banana(grid=(np.arange(-25, 25, 0.05),np.arange(-25, 25, 0.05)),):

x, y = gridxx, yy = np.meshgrid(x, y, sparse=True)densities = np.asarray([np.exp(-banana_nll((x, y))) for x in xx for y in yy])plt.figure()plt.contour(x, y, densities, 1)plt.xlim(xmin=-30, xmax=30)plt.ylim(ymin=-40, ymax=20)plt.show()

def plot_samples(sampler_cls=SGHMC, lr=0.1,num_burn_in_steps=3000,num_samples=1000,color="b"):

sampler = sampler_cls(params=(torch.rand(1, requires_grad=True),torch.rand(1, requires_grad=True)),

lr=lr, negative_log_likelihood=banana_nll)

# skip burn-in samples_ = [sample for sample, _ in islice(sampler, num_burn_in_steps)]

samples = np.asarray([sample for sample, _ in islice(sampler, num_samples)

])plt.scatter(samples[:, 0], samples[:, 1], color=color, label=sampler_cls.__name__)plt.legend()

In [79]: plot_banana()plot_samples(SGHMC, lr=0.5)

8 Chapter 3. Install


<IPython.core.display.Javascript object>

<IPython.core.display.HTML object>

In [80]: plot_banana()plot_samples(SGLD, lr=0.5, color="g")



3.2.2 Simple sinc example

This notebook shows how to fit a simple 𝑠𝑖𝑛𝑐 variant with our BNN and visualize the results.

In [12]: % matplotlib notebookimport syssys.path.insert(0, "../../../")

import numpy as npimport matplotlib.pyplot as plt

from pysgmcmc.models.architectures import simple_tanh_networkfrom pysgmcmc.models.bayesian_neural_network import BayesianNeuralNetworkfrom pysgmcmc.optimizers.sgld import SGLDfrom pysgmcmc.optimizers.sghmc import SGHMC

def fit_bnn(optimizer=SGHMC, num_training_datapoints=20,lr=1e-2, num_burn_in_steps=3000,num_steps=13000, keep_every=100,network_architecture=simple_tanh_network):

input_dimensionality = 1x_train = np.array([

np.random.uniform(np.zeros(1), np.ones(1), input_dimensionality)for _ in range(num_training_datapoints)

])y_train = np.sinc(x_train * 10 - 5).sum(axis=1)

x_test = np.linspace(0, 1, 100)[:, None]y_test = np.sinc(x_test * 10 - 5).sum(axis=1)

bnn = BayesianNeuralNetwork(optimizer=optimizer, lr=lr)

prediction, variance_prediction = bnn.train(x_train, y_train).predict(x_test)

prediction_std = np.sqrt(variance_prediction)

plt.figure()plt.grid()

plt.plot(x_test[:, 0], y_test, label="true", color="black")plt.plot(x_train[:, 0], y_train, "ro")

plt.plot(x_test[:, 0], prediction, label=optimizer.__name__, color="blue")plt.fill_between(x_test[:, 0], prediction + prediction_std, prediction - prediction_std, alpha=0.2, color="indianred")plt.legend()

3.2. Getting started 9


In [13]: fit_bnn(optimizer=SGHMC, lr=1e-2)

INFO:root:Performing 13000 iterationsINFO:root:Progress bar enabled. To disable pass `logging_configuration={level: debug.WARN}`.13000/13000[] - 00:00 - , NLL: 294.4358825683594 - MSELoss: 0.20250172913074493



In [15]: fit_bnn(optimizer=SGLD, lr=1e-3)

INFO:root:Performing 13000 iterationsINFO:root:Progress bar enabled. To disable pass `logging_configuration={level: debug.WARN}`.13000/13000[] - 00:00 - , NLL: 76.88319396972656 - MSELoss: 0.005371000152081251



3.3 API Reference

3.3.1 Samplers

SGHMC

class pysgmcmc.optimizers.sghmc.SGHMC(params, lr: float = 0.01, num_burn_in_steps: int= 3000, noise: float = 0.0, mdecay: float = 0.05,scale_grad: float = 1.0)

Stochastic Gradient Hamiltonian Monte-Carlo Sampler that uses a burn-in procedure to adapt its own hyperpa-rameters during the initial stages of sampling.

See [1] for more details on this burn-in procedure.

See [2] for more details on Stochastic Gradient Hamiltonian Monte-Carlo.

[1] J. T. Springenberg, A. Klein, S. Falkner, F. Hutter In Advances in Neural Information Processing Sys-tems 29 (2016).

Bayesian Optimization with Robust Bayesian Neural Networks.

[2] T. Chen, E. B. Fox, C. Guestrin In Proceedings of Machine Learning Research 32 (2014).

Stochastic Gradient Hamiltonian Monte Carlo

__init__(params, lr: float = 0.01, num_burn_in_steps: int = 3000, noise: float = 0.0, mdecay: float= 0.05, scale_grad: float = 1.0)→ None

Set up a SGHMC Optimizer.

Parameters

• params (iterable) – Parameters serving as optimization variable.

• lr (float, optional) – Base learning rate for this optimizer. Must be tuned to thespecific function being minimized. Default: 1e-2.

• num_burn_in_steps (int, optional) – Number of burn-in steps to perform. Ineach burn-in step, this sampler will adapt its own internal parameters to decrease its error.Set to 0 to turn scale adaption off. Default: 3000.

• noise (float, optional) – (Constant) per-parameter noise level. Default: 0..


http://aad.informatik.uni-freiburg.de/papers/16-NIPS-BOHamiANN.pdf

https://arxiv.org/pdf/1402.4102.pdf

https://docs.python.org/3/library/functions.html#float

https://docs.python.org/3/library/functions.html#int



• mdecay (float, optional) – (Constant) momentum decay per time-step. Default:0.05.

• scale_grad (float, optional) – Value that is used to scale the magnitude of thenoise used during sampling. In a typical batches-of-data setting this usually correspondsto the number of examples in the entire dataset. Default: 1.0.

step(closure=None)Performs a single optimization step (parameter update).

Parameters closure (callable) – A closure that reevaluates the model and returns the loss.Optional for most optimizers.

SGLD

class pysgmcmc.optimizers.sgld.SGLD(params, lr=0.01, precondition_decay_rate=0.95,num_pseudo_batches=1, num_burn_in_steps=3000,diagonal_bias=1e-08)

Stochastic Gradient Langevin Dynamics Sampler with preconditioning. Optimization variable is viewed as aposterior sample under Stochastic Gradient Langevin Dynamics with noise rescaled in eaach dimension accord-ing to RMSProp.

__init__(params, lr=0.01, precondition_decay_rate=0.95, num_pseudo_batches=1,num_burn_in_steps=3000, diagonal_bias=1e-08)→ None

Set up a SGLD Optimizer.

Parameters

• params (iterable) – Parameters serving as optimization variable.

• lr (float, optional) – Base learning rate for this optimizer. Must be tuned to thespecific function being minimized. Default: 1e-2.

• precondition_decay_rate (float, optional) – Exponential decay rate ofthe rescaling of the preconditioner (RMSprop). Should be smaller than but nearly 1 toapproximate sampling from the posterior. Default: 0.95

• num_pseudo_batches (int, optional) – Effective number of minibatches in thedata set. Trades off noise and prior with the SGD likelihood term. Note: Assumes loss istaken as mean over a minibatch. Otherwise, if the sum was taken, divide this number bythe batch size. Default: 1.

• num_burn_in_steps (int, optional) – Number of iterations to collect gradientstatistics to update the preconditioner before starting to draw noisy samples. Default:3000.

• diagonal_bias (float, optional) – Term added to the diagonal of the precon-ditioner to prevent it from degenerating. Default: 1e-8.

step(closure=None)Performs a single optimization step (parameter update).

Parameters closure (callable) – A closure that reevaluates the model and returns the loss.Optional for most optimizers.

3.3. API Reference 11









3.3.2 Models

Bayesian Neural Network

class pysgmcmc.models.bayesian_neural_network.BayesianNeuralNetwork(network_architecture=<functionsim-ple_tanh_network>,batch_size=20,normal-ize_input:bool =True,normal-ize_output:bool =True,num_steps:int =13000,burn_in_steps:int =3000,keep_every:int= 100,loss=<class’pysgm-cmc.models.losses.NegativeLogLikelihood’>,met-rics=(<class’torch.nn.modules.loss.MSELoss’>,), log-ging_configuration:Dict[str,Any] ={’datefmt’:’y/m/d’,’level’:20},opti-mizer=<class’pysgm-cmc.optimizers.sghmc.SGHMC’>,**opti-mizer_kwargs)

__init__(network_architecture=<function simple_tanh_network>, batch_size=20, nor-malize_input: bool = True, normalize_output: bool = True, num_steps:int = 13000, burn_in_steps: int = 3000, keep_every: int = 100,loss=<class ’pysgmcmc.models.losses.NegativeLogLikelihood’>, metrics=(<class’torch.nn.modules.loss.MSELoss’>, ), logging_configuration: Dict[str, Any] = {’datefmt’:’y/m/d’, ’level’: 20}, optimizer=<class ’pysgmcmc.optimizers.sghmc.SGHMC’>, **opti-mizer_kwargs)→ None

Bayesian Neural Network for regression problems.



Bayesian Neural Networks use Bayesian methods to estimate the posterior distribution of a neural net-work’s weights. This allows to also predict uncertainties for test points and thus makes Bayesian NeuralNetworks suitable for Bayesian optimization. This module uses stochastic gradient MCMC methods tosample from the posterior distribution.

See [1] for more details.

[1] J. T. Springenberg, A. Klein, S. Falkner, F. Hutter Bayesian Optimization with Robust BayesianNeural Networks. In Advances in Neural Information Processing Systems 29 (2016).

Parameters

• network_architecture (pysgmcmc.torch_typing.NetworkFactory,optional) – Function mapping integer input dimensionality to an (initialized)torch.nn.Module.

• normalize_input (bool, optional) – Specifies if inputs should be normalizedto zero mean and unit variance.

• normalize_output (bool, optional) – Specifies whether outputs should be un-normalized.

• num_steps (int, optional) – Number of sampling steps to perform after burn-inis finished. In total, num_steps // keep_every network weights will be sampled. Defaultsto 10000.

• burn_in_steps (int, optional) – Number of burn-in steps to perform. Thisvalue is passed to the given optimizer if it supports special burn-in specific behavior. Net-works sampled during burn-in are discarded. Defaults to 3000.

• keep_every (int, optional) – Number of sampling steps (after burn-in) to per-form before keeping a sample. In total, num_steps // keep_every network weights will besampled. Defaults to 100.

• loss (pysgmcmc.torch_typing.TorchLoss, optional) – Loss to use. De-fault: pysgmcmc.models.losses.NegativeLogLikelihood

• logging_configuration (typing.Dict[str, typing.Any],optional) – Configuration for pythons logging module to use. Specifying “level” aslogging.INFO or lower in this dictionary enables displaying a progressbar for training. Ifno “level” is specified, logging.INFO is assumed as default choice. Defaults to {“level”:logging.INFO, “datefmt”: “y/m/d”}.

• optimizer (torch.optim.Optimizer, optional) – Function that re-turns a torch.optim.optimizer.Optimizer subclass. Defaults to pysgm-cmc.optimizers.sghmc.SGHMC.

__weakref__list of weak references to the object (if defined)

_keep_sample(step: int)→ bool

Determine if the network weight sample recorded at step should be stored. Samples are recorded af-ter burn-in (step > self.num_burn_in_steps), and only every self.keep_every th step.

Parameters step (int) – Current iteration count.

Returns should_keep – Sentinel that is True if and only if network weights should be stored atstep.

Return type bool


https://docs.python.org/3/library/functions.html#bool





https://docs.python.org/3/library/typing.html#typing.Dict

https://docs.python.org/3/library/stdtypes.html#str




network_weightsExtract current network weight values as np.ndarray.

Returns weight_values – Numpy array containing current network weight values.

Return type np.ndarray

train(x_train: numpy.ndarray, y_train: numpy.ndarray)Train a BNN using input datapoints x_train with corresponding labels y_train. :param x_train: Inputtraining datapoints. :type x_train: numpy.ndarray (N, D) :param y_train: Input training labels. :typey_train: numpy.ndarray (N,)

Losses

class pysgmcmc.models.losses.NegativeLogLikelihood(parameters: Iter-able[torch.Tensor],num_datapoints:int, variance_prior:Callable[[torch.Tensor],torch.Tensor] = <func-tion log_variance_prior>,weight_prior:Callable[[Iterable[torch.Tensor]],torch.Tensor] = <functionweight_prior>, size_average:bool = True, reduce: bool =False)

Impementation of BNN negative log likelihood for regression problems.

__init__(parameters: Iterable[torch.Tensor], num_datapoints: int, variance_prior:Callable[[torch.Tensor], torch.Tensor] = <function log_variance_prior>, weight_prior:Callable[[Iterable[torch.Tensor]], torch.Tensor] = <function weight_prior>, size_average:bool = True, reduce: bool = False)→ None

Instantiate a loss object for given network parameters. Requires num_datapoints of the entire regres-sion dataset for proper scaling.

Parameters

• parameters (typing.Iterable[torch.Tensor]) – Pytorch variables of BNNparameters.

• num_datapoints (int) – Total number of datapoints of the entire regression datasetto process.

• variance_prior (pysgmcmc.torch_typing.VariancePrior,optional) – Prior for BNN variance. Default: pysgm-cmc.models.priors.log_variance_prior.

• weight_prior (pysgmcmc.torch_typing.WeightPrior, optional) –Prior for BNN weights. Default: pysgmcmc.models.priors.weight_prior.

forward(input: Iterable[torch.Tensor], target: Iterable[torch.Tensor])→ torch.TensorCompute NLL for 2d-network predictions input and (batch) labels target.

Parameters

• input (pysgmcmc.torch_typing.Predictions) – Network predictions.


https://docs.python.org/3/library/typing.html#typing.Iterable



• target (pysgmcmc.torch_typing.Targets) – Labels for each datapoint in thecurrent batch.

Returns nll – Scalar value. NLL of BNN predictions given as input with respect to labels target.

Return type torch.Tensor

pysgmcmc.models.losses.get_loss(loss_cls: Union[Callable[[Iterable[torch.Tensor], int],Callable[[Iterable[torch.Tensor], Iterable[torch.Tensor]],torch.Tensor]], Callable[[], Callable[[Iterable[torch.Tensor],Iterable[torch.Tensor]], torch.Tensor]]], **loss_kwargs)→ Callable[[Iterable[torch.Tensor], Iterable[torch.Tensor]],torch.Tensor]

Wrapper to use NegativeLogLikelihood interchangeably with other pytorch losses. loss_kwargs is ex-pected to be a dict with key parameters mapped to network parameters and key num_datapoints mappedto an integer representing the amount of datapoints in the entire regression dataset.

Parameters

• loss_cls (pysgmcmc.torch_typing.TorchLoss) – Class type of a loss, e.g. pys-gmcmc.models.losses.NegativeLogLikelihood.

• loss_kwargs (dict) – Keyword arguments to be passed to loss_cls. Must contain keysparameters for BNN parameters and num_datapoints for the amount of datapoints in theentire regression dataset.

Returns loss_instance – Instance of loss_cls.

Return type pysgmcmc.torch_typing.TorchLossFunction

pysgmcmc.models.losses.to_bayesian_loss(torch_loss)

Wrapper to make pytorch losses compatible with our BNN predictions. BNN predictions are 2-d, with thesecond dimension representing model variance. This wrapper essentially passes only the network meanprediction into torch_loss, which allows us to evaluate torch_loss on our network predictions as normally.

Parameters torch_loss (pysgmcmc.torch_typing.TochLoss) – Class type of a py-torch loss to evaluate on our BNN, e.g. torch.nn.MSELoss.

Returns Class type that behaves like torch_loss but assumes inputs coming from a BNN. It willevaluate torch_loss on the BNN predictions first dimension, on the mean prediction, only.

Return type torch_loss_changed

Architectures

Priors

3.3.3 Mixin

class pysgmcmc.samplers.mixin.SamplerMixin(negative_log_likelihood, params, *args,**kwargs)

Mixin class that turns a torch.nn.optim.Optimizer into a MCMC sampler.

__init__(negative_log_likelihood, params, *args, **kwargs)

Instantiate a sampler object. (Initial) parameters are passed as iterable params, negative_log_likelihoodis a function mapping parameters to a NLL value and *args and **kwargs allow specifying additionalarguments to pass to a sampler, e.g. lr or mdecay.


https://docs.python.org/3/library/stdtypes.html#dict


Parameters

• negative_log_likelihood (typing.Callable[[typing.Iterable[torch.Tensor]], torch.Tensor]) – Callable mapping parametersto a NLL value.

• params (iterable) – Iterable of parameters used to construct samples.

See also:

pysgmcmc.samplers.sghmc.SGHMC() SGHMC sampler that uses this mixin.

__next__()

Perform a step of this sampler and return parameters with costs. Together with __iter__, this allowsusing samplers as iterables.

Returns

• parameters (typing.Tuple[numpy.ndarray, . . . ]) – Current parameters.

• cost (torch.Tensor) – NLL value associated with parameters.


parametersReturn last sample as tuple of numpy arrays.

Returns current_parameters – Tuple of numpy arrays containing last sampled values.

Return type typing.Tuple[numpy.ndarray, ..]

sample_step()Perform a single step with the sampler.

Returns

• parameters (typing.Tuple[numpy.ndarray, . . . ]) – Current parameters.

• cost (torch.Tensor) – NLL value associated with parameters.

• next_parameters (typing.Tuple[numpy.ndarray, . . . ]) – Parameters to evaluate on a sub-sequent call.

3.3.4 SGHMC

class pysgmcmc.samplers.sghmc.SGHMC(negative_log_likelihood, params, *args, **kwargs)SGHMC Sampler that performs updates like pysgmcmc.optimizers.sghmc.SGHMCOptimizer.

3.3.5 SGLD

class pysgmcmc.samplers.sgld.SGLD(negative_log_likelihood, params, *args, **kwargs)SGLD Sampler that performs his updates like pysgmcmc.optimizers.sgld.SGLDOptimizer.





3.3.6 Data Batches

pysgmcmc.data.utils.infinite_dataloader(dataloader)Yield an unbounded amount of batches from a torch.utils.data.DataLoader.

Parameters dataloader (torch.utils.data.DataLoader) – Iterable yielding batchesof data from a dataset of interest.

3.3.7 Progressbar

class pysgmcmc.progressbar.TrainingProgressbar(iterable, *args, **kwargs)Dummy progressbar that displays nothing and simply iterates the iterable. Used as placeholder if tqdm is notinstalled.

__init__(iterable, *args, **kwargs)Initialize self. See help(type(self)) for accurate signature.


3.3.8 Torch Utils

pysgmcmc.torch_utils.get_name(object_: Any)→ str

Get a string representation of the name of object_. Defaults to object_.__name__ for most objects. Forclasses in module pysgmcmc.optimizers and pysgmcmc.models.losses this returns an abbreviated nameidentifying the loss or optimizer.

Parameters object (typing.Any) – Any python object. Must have a __name__ attribute.

Returns name – String represenation of the name of object_.

Return type str

Examples

For most objects, this function simply returns their __name__ attribute:

>>> from torch.optim import Adam>>> get_name(Adam) == Adam.__name__True

If an object sets a name attribute, that is used instead:

>>> from pysgmcmc.models.losses import NegativeLogLikelihood>>> get_name(NegativeLogLikelihood) == "NLL" != NegativeLogLikelihood.__name__True


https://docs.python.org/3/library/stdtypes.html#str



CHAPTER 4

Indices and tables

• genindex

• modindex

• search

19


20 Chapter 4. Indices and tables

Python Module Index

ppysgmcmc.data.utils, 17pysgmcmc.models, 12pysgmcmc.models.architectures, 15pysgmcmc.models.losses, 14pysgmcmc.models.priors, 15pysgmcmc.optimizers, 10pysgmcmc.optimizers.sghmc, 10pysgmcmc.optimizers.sgld, 11pysgmcmc.progressbar, 17pysgmcmc.samplers, 15pysgmcmc.samplers.mixin, 15pysgmcmc.samplers.sghmc, 16pysgmcmc.samplers.sgld, 16pysgmcmc.torch_utils, 17

21


22 Python Module Index

Index

Symbols__init__() (pysgmcmc.models.bayesian_neural_network.BayesianNeuralNetwork

method), 12__init__() (pysgmcmc.models.losses.NegativeLogLikelihood

method), 14__init__() (pysgmcmc.optimizers.sghmc.SGHMC

method), 10__init__() (pysgmcmc.optimizers.sgld.SGLD method),

11__init__() (pysgmcmc.progressbar.TrainingProgressbar

method), 17__init__() (pysgmcmc.samplers.mixin.SamplerMixin

method), 15__next__() (pysgmcmc.samplers.mixin.SamplerMixin

method), 16__weakref__ (pysgmcmc.models.bayesian_neural_network.BayesianNeuralNetwork

attribute), 13__weakref__ (pysgmcmc.progressbar.TrainingProgressbar

attribute), 17__weakref__ (pysgmcmc.samplers.mixin.SamplerMixin

attribute), 16_keep_sample() (pysgm-

cmc.models.bayesian_neural_network.BayesianNeuralNetworkmethod), 13

BBayesianNeuralNetwork (class in pysgm-

cmc.models.bayesian_neural_network), 12

Fforward() (pysgmcmc.models.losses.NegativeLogLikelihood

method), 14

Gget_loss() (in module pysgmcmc.models.losses), 15get_name() (in module pysgmcmc.torch_utils), 17

Iinfinite_dataloader() (in module pysgmcmc.data.utils), 17

NNegativeLogLikelihood (class in pysgm-

cmc.models.losses), 14network_weights (pysgm-

cmc.models.bayesian_neural_network.BayesianNeuralNetworkattribute), 13

Pparameters (pysgmcmc.samplers.mixin.SamplerMixin at-

tribute), 16pysgmcmc.data.utils (module), 17pysgmcmc.models (module), 12pysgmcmc.models.architectures (module), 15pysgmcmc.models.losses (module), 14pysgmcmc.models.priors (module), 15pysgmcmc.optimizers (module), 10pysgmcmc.optimizers.sghmc (module), 10pysgmcmc.optimizers.sgld (module), 11pysgmcmc.progressbar (module), 17pysgmcmc.samplers (module), 15pysgmcmc.samplers.mixin (module), 15pysgmcmc.samplers.sghmc (module), 16pysgmcmc.samplers.sgld (module), 16pysgmcmc.torch_utils (module), 17

Ssample_step() (pysgmcmc.samplers.mixin.SamplerMixin

method), 16SamplerMixin (class in pysgmcmc.samplers.mixin), 15SGHMC (class in pysgmcmc.optimizers.sghmc), 10SGHMC (class in pysgmcmc.samplers.sghmc), 16SGLD (class in pysgmcmc.optimizers.sgld), 11SGLD (class in pysgmcmc.samplers.sgld), 16step() (pysgmcmc.optimizers.sghmc.SGHMC method),

11step() (pysgmcmc.optimizers.sgld.SGLD method), 11

Tto_bayesian_loss() (in module pysgmcmc.models.losses),

15

23


train() (pysgmcmc.models.bayesian_neural_network.BayesianNeuralNetworkmethod), 14

TrainingProgressbar (class in pysgmcmc.progressbar), 17

24 Index

Documents

pysgmcmc Documentation - Read the Docs...pysgmcmc Documentation 3.2Getting started 3.2.1Samplers on simple energy functions In this notebook, we ﬁt a simple Banana energy function