Metadata-Version: 2.1
Name: shub-workflow
Version: 1.13.0.1
Summary: Workflow manager for Zyte ScrapyCloud tasks.
Home-page: https://github.com/scrapinghub/shub-workflow
Maintainer: Scrapinghub
License: BSD
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml (>=3.12)
Requires-Dist: scrapinghub[msgpack] (>=2.3.1)
Requires-Dist: jinja2 (>=2.7.3)
Requires-Dist: sqlitedict (==1.6.0)
Requires-Dist: boto3 (>=1.9.92)
Requires-Dist: bloom-filter2 (==2.0.0)
Requires-Dist: collection-scanner
Requires-Dist: tenacity
Requires-Dist: typing-extensions
Requires-Dist: scrapy
Provides-Extra: with-gcs-tools
Requires-Dist: google-cloud-storage (>=1.38.0) ; extra == 'with-gcs-tools'
Provides-Extra: with-s3-tools
Requires-Dist: s3fs (>=0.4.0) ; extra == 'with-s3-tools'

A set of tools for controlling processing workflow with spiders and script running in scrapinghub ScrapyCloud.

# Installation

```
pip install shub-workflow
```

If you want to support s3 tools:

```
pip install shub-workflow[with-s3-tools]
```

For google cloud storage tools support:

```
pip install shub-workflow[with-gcs-tools]
```

# Usage

Check [Project Wiki](https://github.com/scrapinghub/shub-workflow/wiki) for documentation. You can also see code tests for lots of examples of usage.

# Note

The requirements for this library are defined in setup.py as usual. The Pipfile files in the repository don't define dependencies. It is only used
for setting up a development environment for shub-workflow library development and testing.
