Metadata-Version: 2.1
Name: torchflows
Version: 1.0.2
Summary: Normalizing flows in PyTorch
Home-page: https://github.com/davidnabergoj/normalizing-flows
Author: David Nabergoj
Author-email: David Nabergoj <david.nabergoj@fri.uni-lj.si>
Project-URL: Homepage, https://github.com/davidnabergoj/torchflows
Project-URL: Issues, https://github.com/davidnabergoj/torchflows/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7, <4
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.1
Requires-Dist: numpy<2,>=1.26.0
Requires-Dist: torchdiffeq>=0.2.3
Requires-Dist: tqdm>=4.65.0

# Torchflows: normalizing flows in PyTorch

Torchflows is a library for generative modeling and density estimation using normalizing flows.
It implements many normalizing flow architectures and their building blocks for:

* easy use of normalizing flows as trainable distributions;
* easy implementation of new normalizing flows.

Example use:

```python
import torch
from torchflows.flows import Flow
from torchflows.architectures import RealNVP

torch.manual_seed(0)

n_data = 1000
n_dim = 3

x = torch.randn(n_data, n_dim)  # Generate some training data
bijection = RealNVP(n_dim)  # Create the bijection
flow = Flow(bijection)  # Create the normalizing flow

flow.fit(x)  # Fit the normalizing flow to training data
log_prob = flow.log_prob(x)  # Compute the log probability of training data
x_new = flow.sample(50)  # Sample 50 new data points

print(log_prob.shape)  # (100,)
print(x_new.shape)  # (50, 3)
```

We provide more examples [here](examples/).

## Installing

Install via pip:
```
pip install torchflows
```

Install the package directly from Github:

```
pip install git+https://github.com/davidnabergoj/torchflows.git
```

Setup for development:

```
git clone https://github.com/davidnabergoj/torchflows.git
cd torchflows
pip install -r requirements.txt
```

We support Python versions 3.7 and upwards.

## Brief background

A normalizing flow (NF) is a flexible trainable distribution.
It is defined as a bijective transformation of a simple distribution, such as a standard Gaussian.
The bijection is typically an invertible neural network.
Training a NF using a dataset means optimizing the bijection's parameters to make the dataset likely under the NF.
We can use a NF to compute the probability of a data point or to independently sample data from the process that
generated our dataset.

The density of a NF $q(x)$ with the bijection $f(z) = x$ and base distribution $p(z)$ is defined as:
$$\log q(x) = \log p(f^{-1}(x)) + \log\left|\det J_{f^{-1}}(x)\right|.$$
Sampling from a NF means sampling from the simple distribution and transforming the sample using the bijection.

## Supported architectures

We list supported NF architectures below.
We classify architectures as either autoregressive, residual, or continuous; as defined
by [Papamakarios et al. (2021)](https://arxiv.org/abs/1912.02762).
We specify whether the forward and inverse passes are exact; otherwise they are numerical or not implemented (Planar,
Radial, and Sylvester flows).
An exact forward pass guarantees exact density estimation, whereas an exact inverse pass guarantees exact sampling.
Note that the directions can always be reversed, which enables exact computation for the opposite task.
We also specify whether the logarithm of the Jacobian determinant of the transformation is exact or computed numerically.

| Architecture                                                           	 | Bijection type           	 | Exact forward 	 | Exact inverse | Exact log determinant |
|--------------------------------------------------------------------------|:--------------------------:|:---------------:|:-------------:|:---------------------:|
| [NICE](http://arxiv.org/abs/1410.8516)                              	    |      Autoregressive 	      |     ✔     	     |       ✔       |           ✔           |
| [Real NVP](http://arxiv.org/abs/1605.08803)                         	    |      Autoregressive 	      |     ✔     	     |       ✔       |           ✔           |
| [MAF](http://arxiv.org/abs/1705.07057)                              	    |      Autoregressive 	      |     ✔     	     |       ✔       |           ✔           |
| [IAF](http://arxiv.org/abs/1606.04934)                              	    |      Autoregressive 	      |     ✔     	     |       ✔       |           ✔           |
| [Rational quadratic NSF](http://arxiv.org/abs/1906.04032)           	    |      Autoregressive 	      |     ✔     	     |       ✔       |           ✔           |
| [Linear rational NSF](http://arxiv.org/abs/2001.05168)              	    |      Autoregressive 	      |     ✔     	     |       ✔       |           ✔           |
| [NAF](http://arxiv.org/abs/1804.00779)                              	    |      Autoregressive 	      |     ✔     	     |       ✗       |           ✔           |
| [UMNN](http://arxiv.org/abs/1908.05164)                             	    |      Autoregressive 	      |     ✗     	     |       ✗       |           ✔           |
| [Planar](https://onlinelibrary.wiley.com/doi/abs/10.1002/cpa.21423) 	    |      Residual       	      |     ✔     	     |       ✗       |           ✔           | 
| [Radial](https://proceedings.mlr.press/v37/rezende15.html)          	    |      Residual       	      |     ✔     	     |       ✗       |           ✔           |
| [Sylvester](http://arxiv.org/abs/1803.05649)                        	    |      Residual       	      |     ✔     	     |       ✗       |           ✔           |
| [Invertible ResNet](http://arxiv.org/abs/1811.00995)                	    |      Residual       	      |     ✔     	     |       ✗       |           ✗           |
| [ResFlow](http://arxiv.org/abs/1906.02735)                          	    |      Residual       	      |     ✔     	     |       ✗       |           ✗           |
| [Proximal ResFlow](http://arxiv.org/abs/2211.17158)                 	    |      Residual       	      |     ✔     	     |       ✗       |           ✗           |
| [FFJORD](http://arxiv.org/abs/1810.01367)                           	    |      Continuous     	      |     ✗     	     |       ✗       |           ✗           |
| [RNODE](http://arxiv.org/abs/2002.02798)                            	    |      Continuous     	      |     ✗     	     |       ✗       |           ✗           |
| [DDNF](http://arxiv.org/abs/1810.03256)                             	    |      Continuous     	      |     ✗     	     |       ✗       |           ✗           |
| [OT flow](http://arxiv.org/abs/2006.00104)                          	    |      Continuous     	      |     ✗     	     |       ✗       |           ✗           |


We also support simple bijections (all with exact forward passes, inverse passes, and log determinants):

* Permutation
* Elementwise translation (shift vector)
* Elementwise scaling (diagonal matrix)
* Rotation (orthogonal matrix)
* Triangular matrix
* Dense matrix (using the QR or LU decomposition)
