Metadata-Version: 2.1
Name: omegaconf-argparse
Version: 0.2.0
Summary: Integration between Omegaconf and argparse for mixed config file and CLI arguments
License: Apache-2.0
Author: Giorgos Paraskevopoulos
Author-email: georgepar.91@gmail.com
Requires-Python: >=3.7,<4.0
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Requires-Dist: omegaconf (>=2.0.6,<3.0.0)
Description-Content-Type: text/markdown

# omegaconf-argparse

Integration between Omegaconf and argparse for mixed config file and CLI arguments

Flexible configuration management is important during experimentation, e.g. when training machine learning models.

Ideally, we want both a configuration file to hold more "stable" hyperparameter values and the flexibility to
change values through command line arguments for rapid experimentation.

## How it works

This package provides a barebones solution based on the excellent [OmegaConf](https://github.com/omry/omegaconf) package.

Specifically, we extend the `OmegaConf` class with a static `from_argparse` method to parse arguments provided using argparse,
and provide utility functions to merge the default CLI values, YAML configuration file values and user provided CLI arguments.

# Installation

Install package from PyPi:

```
pip install omegaconf-argparse
```

## Usage

We provide a high-level utility function `parse_config` that merges a YAML configuration file with an `argparse.ArgumentParser`:

```
import io
from omegacli import parse_config
mock_config_file = io.StringIO('''
model:
  hidden: 100
''')
parser = argparse.ArgumentParser("My cool model")
parser.add_argument("--hidden", dest="model.hidden", type=int, default=20)
cfg = parse_config(parser, mock_config_file)
>>> {'model': {'hidden': 100}}
type(cfg)
>>> <class 'omegaconf.dictconfig.DictConfig'>
cfg = parse_config(parser, mock_config_file, args=["--hidden", "200"])
>>> {'model': {'hidden': 200}}
mock_config_file = io.StringIO('''
random_value: hello
''')
cfg = parse_config(parser, mock_config_file)
>>> {'model': {'hidden': 20}, 'random_value': 'hello'}
```

You can also use the patched `OmegaConf` class directly:

```
import argparse
from omegacli import OmegaConf
parser = argparse.ArgumentParser("My cool model")
parser.add_argument("--hidden", dest="model.hidden", type=int, default=20)
user_provided_args, default_args = OmegaConf.from_argparse(parser, args=["--hidden", "100"])
user_provided_args
>>> {'model': {'hidden': 100}}
default_args
>>> {}
user_provided_args, default_args = OmegaConf.from_argparse(parser)
user_provided_args
>>> {}
default_args
>>> {'model': {'hidden': 20}}
```

**NOTE**: the `from_argparse` method calls the `parser.parse_args()`.

## Merging of provided values

The precedence for merging is as follows

- default cli args values < config file values < user provided cli args

E.g.:

- if you don't include a value in your configuration it will take the default value from the argparse arguments
- if you provide a cli arg (e.g. run the script with --bsz 64) it will override the value in the config file

### Conventions

To create a nested configuration structure and match with the argparse provided CLI args,
we use the `dest` kwarg when adding an argument with `parser.add_argument`.
Specifically, we follow the convention that the destination is a string, delimited by `.` to indicate nested structure.

For example:

```
parser.add_argument("--hidden", dest="model_hidden", type=int, default=20)
```

will create a top-level element in the configuration:

```
user_provided_args, default_args = OmegaConf.from_argparse(parser, args=["--hidden", "100"])
user_provided_args
>>> {'model_hidden': 100}
```

This will match with the following entry in the YAML file:

```
model_hidden: 100
```

The following:

```
parser.add_argument("--hidden", dest="model.hidden", type=int, default=20)
```

will create a nested structure in the configuration:

```
user_provided_args, default_args = OmegaConf.from_argparse(parser, args=["--hidden", "100"])
user_provided_args
>>> {'model': {'hidden': 100}}
```

and will match with the following YAML entry:

```
model:
  hidden: 100
```

The parsing is recursive, so you can go as deep as you want.

## Generate a configuration file based on an argument parser

Run:

```
from omegacli import generate_config_template
generate_config_template(parser, "example-config.yaml")
```

This will initialize a configuration file, that is consistent with the argument parser.
You can use this as a starting point for saving and editing your configuration.

## Similar solutions

- [Hydra](https://hydra.cc/docs/intro/): A more feature-rich, but more complex solution. If you are willing to introduce the dependency you can use it
- [Pytorch-Lightning](https://pytorch-lightning.readthedocs.io/en/1.6.2/common/lightning_cli.html): PL introduced a similar functionality. You can use it if you are in the PL ecosystem.

## Why create a separate package?

`OmegaConf` plans to remain agnostic to the command line argument parser, therefore we cannot merge this solution upstream. [See related issue](https://github.com/omry/omegaconf/issues/569).

