Metadata-Version: 2.1
Name: osml-model-runner-test
Version: 2.0.0
Summary: Package to run tests against osml-model-runner deployments.
Author: Amazon Web Services
Author-email: aws-osml-admin@amazon.com
License: 
        MIT No Attribution
        
        Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
        
        Permission is hereby granted, free of charge, to any person obtaining a copy of
        this software and associated documentation files (the "Software"), to deal in
        the Software without restriction, including without limitation the rights to
        use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
        the Software, and to permit persons to whom the Software is furnished to do so.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
        FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
        COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
        IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
        CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Project-URL: Source, https://github.com/aws-solutions-library-samples/osml-model-runner-test
Project-URL: Tracker, https://github.com/aws-solutions-library-samples/osml-model-runner-test/issues
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: boto3==1.28.57
Requires-Dist: botocore==1.31.57
Requires-Dist: geojson==3.0.1
Requires-Dist: pytest==7.3.1
Requires-Dist: setuptools==68.0.0
Requires-Dist: toml==0.10.2
Provides-Extra: gdal
Requires-Dist: gdal>=3.7.0; extra == "gdal"
Provides-Extra: test
Requires-Dist: tox; extra == "test"

# OSML Model Runner Test

This package contains the integration tests for OSML application

### Table of Contents
* [Getting Started](#getting-started)
    * [Prerequisites](#prerequisites)
    * [Installation Guide](#installation-guide)
    * [Documentation](#documentation)
    * [Build and Local Testing](#build-and-local-testing)
    * [Running Tests in Docker](#running-tests-in-docker)
    * [Running LoadTest](#running-loadtest)
* [Support & Feedback](#support--feedback)
* [Security](#security)
* [License](#license)


## Getting Started
### Prerequisites

First, ensure you have installed the following tools locally

1. [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
2. [docker](https://nodejs.org/en)
3. [tox](https://tox.wiki/en/latest/installation.html)

### Installation Guide

1. Clone `osml-model-runner-test` package into your desktop

```sh
git clone https://github.com/aws-solutions-library-samples/osml-model-runner-test.git
```

1. Run `tox` to create a virtual environment

```sh
cd osml-model-runner-test
tox
```

### Documentation

You can find documentation for this library in the `./doc` directory. Sphinx is used to construct a searchable HTML
version of the API documents.

```shell
tox -e docs
```

### Build and Local Testing

You can run the hydra tests against your dev account by exporting the required parameters and using the pytest CLI by
using the demo utility ``bin/run_test.sh``. Do not forget to load up your AWS credentials into your terminal, please follow this [guide](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) on how to load your aws credentials.


```
export ACCOUNT="INSERT YOUR ACCOUNT"
export IMAGE_TYPE="INSERT YOUR IMAGE TYPE" # small, medium, large, or tile.<tif|ntf|jpeg|png>
export MODEL_TYPE="INSERT YOUR MODEL TYPE" # centerpoint, flood, or aircraft

./bin/run_test.sh ${IMAGE_TYPE} ${MODEL_TYPE} NITF JPEG us-west-2 $ACCOUNT_NUMBER
```

**Examples:**

```
./bin/run_test.sh small centerpoint NITF JPEG us-west-2 $ACCOUNT_NUMBER

./bin/run_test.sh medium flood NITF JPEG us-west-2 $ACCOUNT_NUMBER

./bin/run_test.sh large centerpoint NITF JPEG us-west-2 $ACCOUNT_NUMBER

./bin/run_test.sh tile.tif aircraft NITF JPEG us-west-2 $ACCOUNT_NUMBER
```

If you prefer, you can execute the python script:

```
python3 bin/process_image.py --image small --model centerpoint
```

To print out the usage for python script, execute:
```
python3 bin/process_image.py --help
```

### Running Tests in Docker
Arguments can be passed in to docker run in the same way that they are provided to the shell script.
For example, the container can be built and run as follows:

```
docker build . -t model-runner-hydra-test:latest
docker run -v ~/.aws:/root/.aws model-runner-hydra-test:latest small centerpoint NITF JPEG us-west-2 $ACCOUNT_NUMBER
```

Credentials from the user's account are volume mounted into the container's root directory.

### Running LoadTest

You can run the load test against your dev account and be able to determine the cost and the performance. **Please advise** it can potentially rack up your AWS bills!

**Examples:**
```
python3 bin/run_load_test.py --periodic_sleep 60 --processing_window 1
```

To print out the usage for this load test script, execute:

```
python3 bin/run_load_test.py --help
```

## Support & Feedback

To post feedback, submit feature ideas, or report bugs, please use the [Issues](https://github.com/aws-solutions-library-samples/osml-model-runner-test/issues) section of this GitHub repo.

If you are interested in contributing to OversightML Model Runner, see the [CONTRIBUTING](CONTRIBUTING.md) guide.

## Security

See [CONTRIBUTING](CONTRIBUTING.md) for more information.

## License

MIT No Attribution Licensed. See [LICENSE](LICENSE).
