Metadata-Version: 2.1
Name: dagster-scaleway
Version: 0.1.0
Summary: 
License: Apache-2.0
Author: Andy Méry
Author-email: amery@scaleway.com
Requires-Python: >=3.11,<4.0
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: dagster (>=1.6.0,<2.0.0)
Requires-Dist: scaleway (>=1.4.1,<2.0.0)
Description-Content-Type: text/markdown

# Dagster Scaleway (WIP)

This repository contains a [Dagster](https://dagster.io) integration for [Scaleway](https://www.scaleway.com/en/).

It allows you to run Dagster pipelines on [Scaleway Serverless Jobs](https://www.scaleway.com/en/docs/serverless/jobs/quickstart/).

## Basic usage

Build a docker image containing your Dagster code and push it to the Scaleway Registry (or any other registry of your choice):

```bash
dagster project scaffold --name my-dagster-project
cd my-dagster-project

echo<<EOF > Dockerfile
FROM python:3.12-slim-bookworm
WORKDIR /app
COPY . .
RUN pip install .
# Install the Dagster Scaleway module. You can also specify it in your "setup.py" file
RUN pip install dagster_scaleway
EOF
```

Build and push the image:

```bash
docker build -t rg.fr-par.scw.cloud/<your-namespace>/dagster-scaleway-example:latest .
docker push rg.fr-par.scw.cloud/<your-namespace>/dagster-scaleway-example:latest
```

Then, configure the `dagster.yaml` file to use this image:

```yaml
run_launcher:
  module: dagster_scaleway
  class: ScalewayServerlessJobRunLauncher
  config:
    docker_image: rg.fr-par.scw.cloud/<your-namespace>/dagster-scaleway-example:latest
```

Run Dagster locally:

```bash
pip install -e ".[dev]" "dagster-scaleway"
dagster dev
```

Your Dagster ops will be run as Scaleway Serverless Jobs! :tada:

See the [Dagster documentation](https://docs.dagster.io/getting-started/create-new-project#step-4-development) for more information on how to get started with Dagster.

## Examples

See the [examples](./examples) folder for examples of how to use this integration.

