Metadata-Version: 2.1
Name: osml-model-runner-test
Version: 2.0.2
Summary: Package to run tests against osml-model-runner deployments.
Author: Amazon Web Services
Author-email: aws-osml-admin@amazon.com
License: 
        MIT No Attribution
        
        Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
        
        Permission is hereby granted, free of charge, to any person obtaining a copy of
        this software and associated documentation files (the "Software"), to deal in
        the Software without restriction, including without limitation the rights to
        use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
        the Software, and to permit persons to whom the Software is furnished to do so.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
        FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
        COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
        IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
        CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Project-URL: Source, https://github.com/aws-solutions-library-samples/osml-model-runner-test
Project-URL: Tracker, https://github.com/aws-solutions-library-samples/osml-model-runner-test/issues
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: boto3==1.28.57
Requires-Dist: botocore==1.31.57
Requires-Dist: geojson==3.0.1
Requires-Dist: pytest==7.3.1
Requires-Dist: setuptools==68.0.0
Requires-Dist: toml==0.10.2
Provides-Extra: gdal
Requires-Dist: gdal>=3.7.0; extra == "gdal"
Provides-Extra: test
Requires-Dist: tox; extra == "test"

# OSML Model Runner Test

This package contains the integration tests for OSML application

### Table of Contents
* [Getting Started](#getting-started)
    * [Prerequisites](#prerequisites)
    * [Installation Guide](#installation-guide)
    * [Documentation](#documentation)
    * [Build and Local Testing](#build-and-local-testing)
    * [Running LoadTest](#running-loadtest)
* [Support & Feedback](#support--feedback)
* [Security](#security)
* [License](#license)


## Getting Started
### Prerequisites

First, ensure you have installed the following tools locally

1. [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
2. [tox](https://tox.wiki/en/latest/installation.html)

### Installation Guide

1. Clone `osml-model-runner-test` package into your desktop

```sh
git clone https://github.com/aws-solutions-library-samples/osml-model-runner-test.git
```

1. Run `tox` to create a virtual environment

```sh
cd osml-model-runner-test
tox
```

### Documentation

You can find documentation for this library in the `./doc` directory. Sphinx is used to construct a searchable HTML
version of the API documents.

```shell
tox -e docs
```

### Local Testing


####  Credentials

Credentials from the user's account are volume mounted into the container's root directory.

**Processing an image:**

You can run the integration tests against your dev account by exporting the required parameters and using the pytest CLI by
using the python script ``bin/process_image.py``. Remember to load up your AWS credentials into your terminal, please follow this [guide](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) on how to load your aws credentials.


```
python bin/process_image.py --image <image type> --model <model type>
```

**Examples:**

```
python3 bin/process_image.py --image small --model centerpoint

python3 bin/process_image.py --image meta --model centerpoint

python3 bin/process_image.py --image large --model flood

python3 bin/process_image.py --image tile_tif --model aircraft
```

To print out the usage for python script, execute:
```
python3 bin/process_image.py --help
```

To execute the integration test, exclude `--skip_integ` from the command line interface. It is essential that the images and models listed in the table below are aligned accurately for the test to succeed. Conversely, by adding `--skip_integ` to the CLI, all comparison checks will be bypassed, rendering the table irrelevant for testing purposes.

| image                       | model       |
|-----------------------------|-------------|
| small                       | centerpoint |
| meta                        | centerpoint |
| sicd_capella_chip_ntf       | centerpoint |
| sicd_umbra_chip_ntf         | centerpoint |
| sicd_interferometric_hh_ntf | centerpoint |
| wbid                        | centerpoint |
| large                       | flood       |
| tile_tif                    | aircraft    |
| tile_ntf                    | aircraft    |
| tile_jpeg                   | aircraft    |
| tile_png                    | aircraft    |

### Running LoadTest

You can run the load test against your dev account and be able to determine the cost and the performance. **Please advise** it can potentially rack up your AWS bills!

**Examples:**
```
python3 bin/run_load_test.py --periodic_sleep 60 --processing_window 1
```

To print out the usage for this load test script, execute:

```
python3 bin/run_load_test.py --help
```

## Support & Feedback

To post feedback, submit feature ideas, or report bugs, please use the [Issues](https://github.com/aws-solutions-library-samples/osml-model-runner-test/issues) section of this GitHub repo.

If you are interested in contributing to OversightML Model Runner, see the [CONTRIBUTING](https://github.com/aws-solutions-library-samples/osml-model-runner-test/tree/main/CONTRIBUTING.md) guide.

## Security

See [CONTRIBUTING](https://github.com/aws-solutions-library-samples/osml-model-runner-test/tree/main/CONTRIBUTING.md) for more information.

## License

MIT No Attribution Licensed. See [LICENSE](https://github.com/aws-solutions-library-samples/osml-model-runner-test/tree/main/LICENSE).
