Metadata-Version: 2.1
Name: mlem
Version: 0.2.7
Summary: Version and deploy your models following GitOps principles
Home-page: https://mlem.ai
Author: Mikhail Sveshnikov
Author-email: mike0sv@iterative.ai
Maintainer: Iterative
Maintainer-email: support@mlem.ai
License: Apache License 2.0
Download-URL: https://github.com/iterative/mlem
Keywords: data-science data-version-control machine-learning git mlops developer-tools reproducibility collaboration ai
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Requires-Python: >=3.6
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: dill
Requires-Dist: requests
Requires-Dist: isort (>=5.10)
Requires-Dist: pydantic (<2,>=1.9.0)
Requires-Dist: typer
Requires-Dist: click (<8.2)
Requires-Dist: rich
Requires-Dist: aiohttp (<4)
Requires-Dist: aiohttp-swagger (<2)
Requires-Dist: Jinja2 (>=3)
Requires-Dist: fsspec (>=2021.7.0)
Requires-Dist: pyparsing (<4)
Requires-Dist: cached-property
Requires-Dist: entrypoints
Requires-Dist: gitpython
Requires-Dist: python-gitlab
Requires-Dist: flatdict
Requires-Dist: iterative-telemetry
Provides-Extra: all
Requires-Dist: pytest ; extra == 'all'
Requires-Dist: pytest-cov ; extra == 'all'
Requires-Dist: pytest-lazy-fixture (==0.6.3) ; extra == 'all'
Requires-Dist: pytest-mock ; extra == 'all'
Requires-Dist: pylint (<2.14) ; extra == 'all'
Requires-Dist: pylint-pytest ; extra == 'all'
Requires-Dist: pylint-plugin-utils ; extra == 'all'
Requires-Dist: flaky ; extra == 'all'
Requires-Dist: s3fs ; extra == 'all'
Requires-Dist: boto3 ; extra == 'all'
Requires-Dist: botocore ; extra == 'all'
Requires-Dist: adlfs ; extra == 'all'
Requires-Dist: gcsfs ; extra == 'all'
Requires-Dist: testcontainers ; extra == 'all'
Requires-Dist: emoji ; extra == 'all'
Requires-Dist: lxml ; extra == 'all'
Requires-Dist: openpyxl ; extra == 'all'
Requires-Dist: xlrd ; extra == 'all'
Requires-Dist: tables ; extra == 'all'
Requires-Dist: pyarrow ; extra == 'all'
Requires-Dist: skl2onnx ; extra == 'all'
Requires-Dist: pandas ; extra == 'all'
Requires-Dist: numpy ; extra == 'all'
Requires-Dist: scikit-learn ; extra == 'all'
Requires-Dist: onnx ; extra == 'all'
Requires-Dist: protobuf (==3.20.1) ; extra == 'all'
Requires-Dist: onnxruntime ; extra == 'all'
Requires-Dist: catboost ; extra == 'all'
Requires-Dist: xgboost ; extra == 'all'
Requires-Dist: lightgbm ; extra == 'all'
Requires-Dist: uvicorn ; extra == 'all'
Requires-Dist: fastapi ; extra == 'all'
Requires-Dist: torch ; extra == 'all'
Requires-Dist: tensorflow ; extra == 'all'
Requires-Dist: adlfs (>=2021.10.0) ; extra == 'all'
Requires-Dist: azure-identity (>=1.4.0) ; extra == 'all'
Requires-Dist: knack ; extra == 'all'
Requires-Dist: gcsfs (>=2021.11.1) ; extra == 'all'
Requires-Dist: pyarrow (>=1) ; extra == 'all'
Requires-Dist: fsspec[arrow] ; extra == 'all'
Requires-Dist: s3fs[boto3] (>=2021.11.1) ; extra == 'all'
Requires-Dist: aiobotocore[boto3] (>2) ; extra == 'all'
Requires-Dist: bcrypt ; extra == 'all'
Requires-Dist: sshfs[bcrypt] (>=2021.11.2) ; extra == 'all'
Requires-Dist: pika ; extra == 'all'
Requires-Dist: docker ; extra == 'all'
Requires-Dist: dvc (~=2.0) ; extra == 'all'
Requires-Dist: dvc[azure] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[gdrive] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[gs] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[hdfs] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[oss] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[s3] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[ssh] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[ssh_gssapi] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[webdav] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[webhdfs] (~=2.0) ; extra == 'all'
Requires-Dist: dvc[webdhfs_kerberos] (~=2.0) ; extra == 'all'
Provides-Extra: azure
Requires-Dist: adlfs (>=2021.10.0) ; extra == 'azure'
Requires-Dist: azure-identity (>=1.4.0) ; extra == 'azure'
Requires-Dist: knack ; extra == 'azure'
Provides-Extra: catboost
Requires-Dist: catboost ; extra == 'catboost'
Provides-Extra: docker
Requires-Dist: docker ; extra == 'docker'
Provides-Extra: dvc
Requires-Dist: dvc (~=2.0) ; extra == 'dvc'
Provides-Extra: dvc-azure
Requires-Dist: dvc[azure] (~=2.0) ; extra == 'dvc-azure'
Provides-Extra: dvc-gdrive
Requires-Dist: dvc[gdrive] (~=2.0) ; extra == 'dvc-gdrive'
Provides-Extra: dvc-gs
Requires-Dist: dvc[gs] (~=2.0) ; extra == 'dvc-gs'
Provides-Extra: dvc-hdfs
Requires-Dist: dvc[hdfs] (~=2.0) ; extra == 'dvc-hdfs'
Provides-Extra: dvc-oss
Requires-Dist: dvc[oss] (~=2.0) ; extra == 'dvc-oss'
Provides-Extra: dvc-s3
Requires-Dist: dvc[s3] (~=2.0) ; extra == 'dvc-s3'
Provides-Extra: dvc-ssh
Requires-Dist: dvc[ssh] (~=2.0) ; extra == 'dvc-ssh'
Provides-Extra: dvc-ssh_gssapi
Requires-Dist: dvc[ssh_gssapi] (~=2.0) ; extra == 'dvc-ssh_gssapi'
Provides-Extra: dvc-webdav
Requires-Dist: dvc[webdav] (~=2.0) ; extra == 'dvc-webdav'
Provides-Extra: dvc-webdhfs_kerberos
Requires-Dist: dvc[webdhfs_kerberos] (~=2.0) ; extra == 'dvc-webdhfs_kerberos'
Provides-Extra: dvc-webhdfs
Requires-Dist: dvc[webhdfs] (~=2.0) ; extra == 'dvc-webhdfs'
Provides-Extra: fastapi
Requires-Dist: uvicorn ; extra == 'fastapi'
Requires-Dist: fastapi ; extra == 'fastapi'
Provides-Extra: gs
Requires-Dist: gcsfs (>=2021.11.1) ; extra == 'gs'
Provides-Extra: hdfs
Requires-Dist: pyarrow (>=1) ; extra == 'hdfs'
Requires-Dist: fsspec[arrow] ; extra == 'hdfs'
Provides-Extra: heroku
Requires-Dist: docker ; extra == 'heroku'
Provides-Extra: lightgbm
Requires-Dist: lightgbm ; extra == 'lightgbm'
Provides-Extra: numpy
Requires-Dist: numpy ; extra == 'numpy'
Provides-Extra: onnx
Requires-Dist: onnx ; extra == 'onnx'
Provides-Extra: onnxruntime
Requires-Dist: protobuf (==3.20.1) ; extra == 'onnxruntime'
Requires-Dist: onnxruntime ; extra == 'onnxruntime'
Provides-Extra: pandas
Requires-Dist: pandas ; extra == 'pandas'
Provides-Extra: rmq
Requires-Dist: pika ; extra == 'rmq'
Provides-Extra: s3
Requires-Dist: s3fs[boto3] (>=2021.11.1) ; extra == 's3'
Requires-Dist: aiobotocore[boto3] (>2) ; extra == 's3'
Provides-Extra: sklearn
Requires-Dist: scikit-learn ; extra == 'sklearn'
Provides-Extra: ssh
Requires-Dist: bcrypt ; extra == 'ssh'
Requires-Dist: sshfs[bcrypt] (>=2021.11.2) ; extra == 'ssh'
Provides-Extra: tensorflow
Requires-Dist: tensorflow ; extra == 'tensorflow'
Provides-Extra: tests
Requires-Dist: pytest ; extra == 'tests'
Requires-Dist: pytest-cov ; extra == 'tests'
Requires-Dist: pytest-lazy-fixture (==0.6.3) ; extra == 'tests'
Requires-Dist: pytest-mock ; extra == 'tests'
Requires-Dist: pylint (<2.14) ; extra == 'tests'
Requires-Dist: pylint-pytest ; extra == 'tests'
Requires-Dist: pylint-plugin-utils ; extra == 'tests'
Requires-Dist: flaky ; extra == 'tests'
Requires-Dist: s3fs ; extra == 'tests'
Requires-Dist: boto3 ; extra == 'tests'
Requires-Dist: botocore ; extra == 'tests'
Requires-Dist: adlfs ; extra == 'tests'
Requires-Dist: gcsfs ; extra == 'tests'
Requires-Dist: testcontainers ; extra == 'tests'
Requires-Dist: emoji ; extra == 'tests'
Requires-Dist: lxml ; extra == 'tests'
Requires-Dist: openpyxl ; extra == 'tests'
Requires-Dist: xlrd ; extra == 'tests'
Requires-Dist: tables ; extra == 'tests'
Requires-Dist: pyarrow ; extra == 'tests'
Requires-Dist: skl2onnx ; extra == 'tests'
Requires-Dist: pandas ; extra == 'tests'
Requires-Dist: numpy ; extra == 'tests'
Requires-Dist: scikit-learn ; extra == 'tests'
Requires-Dist: onnx ; extra == 'tests'
Requires-Dist: protobuf (==3.20.1) ; extra == 'tests'
Requires-Dist: onnxruntime ; extra == 'tests'
Requires-Dist: catboost ; extra == 'tests'
Requires-Dist: xgboost ; extra == 'tests'
Requires-Dist: lightgbm ; extra == 'tests'
Requires-Dist: uvicorn ; extra == 'tests'
Requires-Dist: fastapi ; extra == 'tests'
Requires-Dist: torch ; extra == 'tests'
Requires-Dist: tensorflow ; extra == 'tests'
Requires-Dist: adlfs (>=2021.10.0) ; extra == 'tests'
Requires-Dist: azure-identity (>=1.4.0) ; extra == 'tests'
Requires-Dist: knack ; extra == 'tests'
Requires-Dist: gcsfs (>=2021.11.1) ; extra == 'tests'
Requires-Dist: pyarrow (>=1) ; extra == 'tests'
Requires-Dist: fsspec[arrow] ; extra == 'tests'
Requires-Dist: s3fs[boto3] (>=2021.11.1) ; extra == 'tests'
Requires-Dist: aiobotocore[boto3] (>2) ; extra == 'tests'
Requires-Dist: bcrypt ; extra == 'tests'
Requires-Dist: sshfs[bcrypt] (>=2021.11.2) ; extra == 'tests'
Requires-Dist: pika ; extra == 'tests'
Requires-Dist: docker ; extra == 'tests'
Requires-Dist: dvc (~=2.0) ; extra == 'tests'
Provides-Extra: torch
Requires-Dist: torch ; extra == 'torch'
Provides-Extra: xgboost
Requires-Dist: xgboost ; extra == 'xgboost'

![image](https://user-images.githubusercontent.com/6797716/165590476-994d4d93-8e98-4afb-b5f8-6f42b9d56efc.png)


[![Check, test and release](https://github.com/iterative/mlem/actions/workflows/check-test-release.yml/badge.svg)](https://github.com/iterative/mlem/actions/workflows/check-test-release.yml)
[![codecov](https://codecov.io/gh/iterative/mlem/branch/main/graph/badge.svg?token=WHU4OAB6O2)](https://codecov.io/gh/iterative/mlem)
[![PyPi](https://img.shields.io/pypi/v/mlem.svg?label=pip&logo=PyPI&logoColor=white)](https://pypi.org/project/mlem)
[![License: Apache 2.0](https://img.shields.io/github/license/iterative/mlem)](https://github.com/iterative/mlem/blob/master/LICENSE)
<!-- [![Maintainability](https://codeclimate.com/github/iterative/mlem/badges/gpa.svg)](https://codeclimate.com/github/iterative/mlem) -->

MLEM helps you package and deploy machine learning models.
It saves ML models in a standard format that can be used in a variety of production scenarios such as real-time REST serving or batch processing.

- **Run your ML models anywhere:**
  Wrap models as a Python package or Docker Image, or deploy them to Heroku (SageMaker, Kubernetes, and more platforms coming soon).
  Switch between platforms transparently, with a single command.

- **Model metadata into YAML automatically:**
  Automatically include Python requirements and input data needs into a human-readable, deployment-ready format.
  Use the same metafile on any ML framework.

- **Stick to your training workflow:**
  MLEM doesn't ask you to rewrite model training code.
  Add just two lines around your Python code: one to import the library and one to save the model.

- **Developer-first experience:**
  Use the CLI when you feel like DevOps, or the API if you feel like a developer.

## Why is MLEM special?

The main reason to use MLEM instead of other tools is to adopt a **GitOps approach** to manage model lifecycles.

- **Git as a single source of truth:**
  MLEM writes model metadata to a plain text file that can be versioned in Git along with code.
  This enables GitFlow and other software engineering best practices.

- **Unify model and software deployment:**
  Release models using the same processes used for software updates (branching, pull requests, etc.).

- **Reuse existing Git infrastructure:**
  Use familiar hosting like Github or Gitlab for model management, instead of having separate services.

- **UNIX philosophy:**
  MLEM is a modular tool that solves one problem very well.
  It integrates well into a larger toolset from Iterative.ai, such as [DVC](https://dvc.org/) and [CML](https://cml.dev/).

## Usage

This a quick walkthrough showcasing deployment and export functionality of MLEM.

Please read [Get Started guide](https://mlem.ai/doc/get-started) for a full version.

### Installation

MLEM requires Python 3.

```console
$ python -m pip install mlem
```

> To install the pre-release version:
>
> ```console
> $ python -m pip install git+https://github.com/iterative/mlem
> ```

### Saving the model

```python
# train.py
from mlem.api import save
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris

def main():
    data, y = load_iris(return_X_y=True, as_frame=True)
    rf = RandomForestClassifier(
        n_jobs=2,
        random_state=42,
    )
    rf.fit(data, y)

    save(
        rf,
        "rf",
        sample_data=data,
        labels=["random-forest", "classifier"],
        description="Random Forest Classifier",
    )

if __name__ == "__main__":
    main()
```

Check out what we have:

```shell
$ ls
rf
rf.mlem
$ cat rf.mlem
```
<details>
  <summary> Click to show `cat` output</summary>

```yaml
artifacts:
  data:
    hash: ea4f1bf769414fdacc2075ef9de73be5
    size: 163651
    uri: rf
description: Random Forest Classifier
labels:
- random-forest
- classifier
model_type:
  methods:
    predict:
      args:
      - name: data
        type_:
          columns:
          - sepal length (cm)
          - sepal width (cm)
          - petal length (cm)
          - petal width (cm)
          dtypes:
          - float64
          - float64
          - float64
          - float64
          index_cols: []
          type: dataframe
      name: predict
      returns:
        dtype: int64
        shape:
        - null
        type: ndarray
    predict_proba:
      args:
      - name: data
        type_:
          columns:
          - sepal length (cm)
          - sepal width (cm)
          - petal length (cm)
          - petal width (cm)
          dtypes:
          - float64
          - float64
          - float64
          - float64
          index_cols: []
          type: dataframe
      name: predict_proba
      returns:
        dtype: float64
        shape:
        - null
        - 3
        type: ndarray
    sklearn_predict:
      args:
      - name: X
        type_:
          columns:
          - sepal length (cm)
          - sepal width (cm)
          - petal length (cm)
          - petal width (cm)
          dtypes:
          - float64
          - float64
          - float64
          - float64
          index_cols: []
          type: dataframe
      name: predict
      returns:
        dtype: int64
        shape:
        - null
        type: ndarray
    sklearn_predict_proba:
      args:
      - name: X
        type_:
          columns:
          - sepal length (cm)
          - sepal width (cm)
          - petal length (cm)
          - petal width (cm)
          dtypes:
          - float64
          - float64
          - float64
          - float64
          index_cols: []
          type: dataframe
      name: predict_proba
      returns:
        dtype: float64
        shape:
        - null
        - 3
        type: ndarray
  type: sklearn
object_type: model
requirements:
- module: sklearn
  version: 1.0.2
- module: pandas
  version: 1.4.1
- module: numpy
  version: 1.22.3
```
</details>

### Deploying the model

If you want to follow this Quick Start, you'll need to sign up on https://heroku.com,
create an API_KEY and populate `HEROKU_API_KEY` env var.

First, create an environment to deploy your model:

```shell
$ mlem declare env heroku staging
💾 Saving env to staging.mlem
```

Now we can [deploy the model with `mlem deploy`](https://mlem.ai/doc/get-started/deploying)
(you need to use different `app_name`, since it's going to be published on https://herokuapp.com):

```shell
$ mlem deployment run mydeploy -m rf -t staging -c app_name=mlem-quick-start
⏳️ Loading deployment from .mlem/deployment/myservice.mlem
🔗 Loading link to .mlem/env/staging.mlem
🔗 Loading link to .mlem/model/rf.mlem
💾 Updating deployment at .mlem/deployment/myservice.mlem
🏛 Creating Heroku App example-mlem-get-started
💾 Updating deployment at .mlem/deployment/myservice.mlem
🛠 Creating docker image for heroku
  💼 Adding model files...
  🛠 Generating dockerfile...
  💼 Adding sources...
  💼 Generating requirements file...
  🛠 Building docker image registry.heroku.com/example-mlem-get-started/web...
  ✅  Built docker image registry.heroku.com/example-mlem-get-started/web
  🔼 Pushed image registry.heroku.com/example-mlem-get-started/web to remote registry at host registry.heroku.com
💾 Updating deployment at .mlem/deployment/myservice.mlem
🛠 Releasing app my-mlem-service formation
💾 Updating deployment at .mlem/deployment/myservice.mlem
✅  Service example-mlem-get-started is up. You can check it out at https://mlem-quick-start.herokuapp.com/
```

### Exporting the model

You could easily [export the model to a different format using `mlem build`](https://mlem.ai/doc/get-started/building):

```
$ mlem build rf docker -c server.type=fastapi -c image.name=sklearn-model
⏳️ Loading model from rf.mlem
🛠 Building MLEM wheel file...
💼 Adding model files...
🛠 Generating dockerfile...
💼 Adding sources...
💼 Generating requirements file...
🛠 Building docker image sklearn-model:latest...
✅  Built docker image sklearn-model:latest
```

## Contributing

Contributions are welcome! Please see our [Contributing Guide](https://mlem.ai/doc/contributing/core)
for more details. Thanks to all our contributors!

## Copyright

This project is distributed under the Apache license version 2.0 (see the LICENSE file in the project root).

By submitting a pull request to this project, you agree to license your contribution under the Apache license version 2.0 to this project.


