Metadata-Version: 2.1
Name: datapipe-core
Version: 0.13.13
Summary: `datapipe` is a realtime incremental ETL library for Python application
Home-page: https://github.com/epoch8/datapipe
Author: Andrey Tatarinov
Author-email: a@tatarinov.co
Requires-Python: >3.8,<3.12
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Provides-Extra: docs
Provides-Extra: excel
Provides-Extra: gcp
Provides-Extra: gcsfs
Provides-Extra: milvus
Provides-Extra: qdrant
Provides-Extra: ray
Provides-Extra: redis
Provides-Extra: s3fs
Provides-Extra: sqlite
Requires-Dist: Pillow (>=10.0.0,<11.0.0)
Requires-Dist: PyYAML (>=5.3.1)
Requires-Dist: SQLAlchemy (>=1.4.25,<3.0.0)
Requires-Dist: cityhash (>=0.4.2,<0.5.0)
Requires-Dist: click (>=7.1.2)
Requires-Dist: cloudpickle (>=1.6.0)
Requires-Dist: fsspec (>=2021.11.1)
Requires-Dist: gcsfs (>=2021.11.1) ; extra == "gcsfs"
Requires-Dist: iteration-utilities (>=0.11.0)
Requires-Dist: myst-parser (>=0.17.2,<1.1.0) ; extra == "docs"
Requires-Dist: numpy (>=1.21.0,<2.0)
Requires-Dist: openpyxl (>=3.0.7) ; extra == "excel"
Requires-Dist: opentelemetry-api (>=1.8.0,<2.0.0)
Requires-Dist: opentelemetry-exporter-gcp-trace ; extra == "gcp"
Requires-Dist: opentelemetry-instrumentation-sqlalchemy
Requires-Dist: opentelemetry-sdk (>=1.8.0,<2.0.0)
Requires-Dist: pandas (>=1.2.0)
Requires-Dist: psycopg2_binary (>=2.8.4)
Requires-Dist: pymilvus (>=2.0.2,<3.0.0) ; extra == "milvus"
Requires-Dist: pysqlite3-binary (>=0.5.0,<0.6.0) ; (sys_platform != "darwin") and (extra == "sqlite")
Requires-Dist: qdrant-client (>=1.1.7,<2.0.0) ; extra == "qdrant"
Requires-Dist: ray[default] (>=2.5.0,<3.0.0) ; extra == "ray"
Requires-Dist: redis (>=4.3.4,<5.0.0) ; extra == "redis"
Requires-Dist: rich (>=13.3.2,<14.0.0)
Requires-Dist: s3fs (>=2021.11.1) ; extra == "s3fs"
Requires-Dist: sphinx (>=4.5,<6.0) ; extra == "docs"
Requires-Dist: sphinx-rtd-theme (>=2.0.0,<3.0.0) ; extra == "docs"
Requires-Dist: sqlalchemy-pysqlite3-binary (>=0.0.4,<0.0.5) ; (sys_platform != "darwin") and (extra == "sqlite")
Requires-Dist: tqdm-loggable (>=0.2,<0.3)
Requires-Dist: traceback-with-variables (>=2.0.4,<3.0.0)
Requires-Dist: xlrd (>=2.0.1) ; extra == "excel"
Project-URL: Repository, https://github.com/epoch8/datapipe
Description-Content-Type: text/markdown

# Datapipe

[Datapipe](https://datapipe.dev/) is a real-time, incremental ETL library for Python with record-level dependency tracking.

The library is designed for describing data processing pipelines and is capable
of tracking dependencies for each record in the pipeline. This ensures that
tasks within the pipeline receive only the data that has been modified, thereby
improving the overall efficiency of data handling.

https://datapipe.dev/

# Development

At the moment these branches are active:

* `master` - current development state, will be promoted to `0.13.x` series
  release once ready
* `v0.13` - current stable version
* `v0.11` - legacy stable version (`v0.12` was skipped due reasons)

# Version compatibility

At the moment, the datapipe library is under active development. Versions:
`v0.*.*`

It should be expected that each minor version is not backward compatible with
the previous one. That is, `v0.7.0` is not compatible with `v0.6.1`. Dependencies
should be fixed to the exact minor version.

After stabilization and transition to the major version `v1.*.*`, the common
rules will apply: all versions with the same major component are compatible.

