Metadata-Version: 2.1
Name: proxystore
Version: 0.7.0
Summary: Lazy object proxy interface for distributed stores.
Author: Globus Labs
Author-email: Greg Pauloski <jgpauloski@uchicago.edu>
Maintainer-email: Greg Pauloski <jgpauloski@uchicago.edu>, Valerie Hayot-Sasson <vhayot@uchicago.edu>
License: MIT
Project-URL: Homepage, https://proxystore.dev
Project-URL: Documentation, https://docs.proxystore.dev
Project-URL: Source, https://github.com/proxystore/proxystore
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: Implementation :: CPython
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click!=8.1.4
Requires-Dist: cloudpickle>=1.6.0
Requires-Dist: cryptography>=39.0.1
Requires-Dist: globus-sdk>=3.3.0
Requires-Dist: pydantic>=2
Requires-Dist: tomli; python_version < "3.11"
Requires-Dist: tomli-w
Requires-Dist: typing-extensions>=4.3.0
Provides-Extra: all
Requires-Dist: proxystore[endpoints,extensions,kafka,redis,zmq]; extra == "all"
Provides-Extra: endpoints
Requires-Dist: aiortc>=1.3.2; extra == "endpoints"
Requires-Dist: aiosqlite; extra == "endpoints"
Requires-Dist: uvicorn[standard]; extra == "endpoints"
Requires-Dist: psutil; extra == "endpoints"
Requires-Dist: pystun3; extra == "endpoints"
Requires-Dist: python-daemon; extra == "endpoints"
Requires-Dist: quart>=0.18.0; extra == "endpoints"
Requires-Dist: requests>=2.27.1; extra == "endpoints"
Requires-Dist: websockets>=10.0; extra == "endpoints"
Provides-Extra: extensions
Requires-Dist: proxystore-ex; extra == "extensions"
Provides-Extra: kafka
Requires-Dist: confluent-kafka; extra == "kafka"
Provides-Extra: redis
Requires-Dist: redis>=3.4; extra == "redis"
Provides-Extra: zmq
Requires-Dist: pyzmq; extra == "zmq"
Provides-Extra: dev
Requires-Dist: covdefaults>=2.2; extra == "dev"
Requires-Dist: coverage; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: numpy; extra == "dev"
Requires-Dist: pandas; extra == "dev"
Requires-Dist: polars; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23.2; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-timeout; extra == "dev"
Requires-Dist: ruff>=0.2.0; extra == "dev"
Requires-Dist: tox; extra == "dev"
Requires-Dist: types-psutil; extra == "dev"
Requires-Dist: types-redis; extra == "dev"
Requires-Dist: types-requests; extra == "dev"
Requires-Dist: virtualenv; extra == "dev"
Provides-Extra: docs
Requires-Dist: black; extra == "docs"
Requires-Dist: mkdocs-click; extra == "docs"
Requires-Dist: mkdocs-gen-files; extra == "docs"
Requires-Dist: mkdocs-literate-nav; extra == "docs"
Requires-Dist: mkdocs-material==9.4.7; extra == "docs"
Requires-Dist: mkdocs-section-index; extra == "docs"
Requires-Dist: mkdocstrings==0.23.0; extra == "docs"
Requires-Dist: mkdocstrings-python==1.8.0; extra == "docs"
Requires-Dist: mike; extra == "docs"
Requires-Dist: proxystore[all]; extra == "docs"

# ProxyStore

![PyPI - Version](https://img.shields.io/pypi/v/proxystore)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/proxystore)
![GitHub License](https://img.shields.io/github/license/proxystore/proxystore)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.8077899.svg)](https://doi.org/10.5281/zenodo.8077899)

[![docs](https://github.com/proxystore/proxystore/actions/workflows/docs.yml/badge.svg)](https://github.com/proxystore/proxystore/actions/workflows/docs.yml)
[![tests](https://github.com/proxystore/proxystore/actions/workflows/tests.yml/badge.svg?label=tests)](https://github.com/proxystore/proxystore/actions)
[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/proxystore/proxystore/main.svg)](https://results.pre-commit.ci/latest/github/proxystore/proxystore/main)

ProxyStore is a library that facilitates efficient data management in distributed Python applications.
At the core of ProxyStore is the [*proxy*](https://docs.proxystore.dev/latest/concepts/proxy/) object which acts as a transparent reference to an object living in a global object store.
This pass-by-reference interface with just-in-time object resolution works across processes, machines, and sites, and enables data producers to change the low-level communication method dynamically without altering application code or behavior.

ProxyStore accelerates the development of dynamic task-based workflows, serverless applications, and more by (1) decoupling control flow from data flow, (2) abstracting low-level communication mechanisms, and (3) eliminating the need for shims, wrapper functions, and boilerplate code.

ProxyStore supports a diverse set of programming paradigms:

* [Task-based Workflows](https://arxiv.org/abs/2303.08803)
* [Function-as-a-Service/Serverless Applications](https://docs.proxystore.dev/latest/guides/globus-compute/)
* [Distributed Futures](https://docs.proxystore.dev/latest/guides/proxy-futures/)
* [Bulk Data Streaming](https://docs.proxystore.dev/latest/guides/streaming/)
* and more!

ProxyStore provides support for many third-party mediated communication methods
out-of-the-box including
[DAOS](https://docs.daos.io/v2.4/),
[Globus Transfer](https://www.globus.org/data-transfer),
[Kafka](https://kafka.apache.org/),
[KeyDB](https://docs.keydb.dev/), and
[Redis](https://redis.io/).
Custom communication methods built on
[Mochi](https://mochi.readthedocs.io/en/latest/margo.html),
[UCX](https://openucx.org/),
[WebRTC](https://webrtc.org/), and
[ZeroMQ](https://zeromq.org/)
are provided for high-performance and peer-to-peer applications.

Read more about ProxyStore's concepts [here](https://docs.proxystore.dev/latest/concepts/).
Complete documentation for ProxyStore is available at
[docs.proxystore.dev](https://docs.proxystore.dev).

## Installation

The base ProxyStore package can be installed with [`pip`](https://pip.pypa.io/en/stable/).
```bash
pip install proxystore
```

Many features require dependencies that are not installed by default but can
be enabled via extras installation options such as `endpoints`, `kafka`, or `redis`.
*All* optional dependencies can be installed with:
```bash
pip install proxystore[all]
```
This will also install the [`proxystore-ex`](https://extensions.proxystore.dev/)
package which contains extension and experimental features.
The extensions package can also be installed with `pip` using
`proxystore[extensions]` or `proxystore-ex`.

See the [Installation](https://docs.proxystore.dev/latest/installation) guide
for more information about the available extras installation options.
See the [Contributing](https://docs.proxystore.dev/latest/contributing) guide
to get started for local development.

## Example

Getting started with ProxyStore requires a few lines of code.

```python
from proxystore.connectors.redis import RedisConnector
from proxystore.proxy import Proxy
from proxystore.store import register_store
from proxystore.store import Store

store = Store('my-store', RedisConnector('localhost', 6379))

# Store the object and get a proxy. The proxy acts
# like a reference to the object.
data = MyDataType(...)
proxy = store.proxy(data)
assert isinstance(proxy, Proxy)

def my_function(x: MyDataType) -> ...:
    # x is resolved my-store on first use transparently to the
    # function. Then x behaves as an instance of MyDataType.
    assert isinstance(x, MyDataType)

my_function(proxy)  # Succeeds
```

Check out the [Get Started](https://docs.proxystore.dev/latest/get-started)
guide to learn more!

## Citation

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.8077899.svg)](https://doi.org/10.5281/zenodo.8077899)

If you use ProxyStore or any of this code in your work, please cite our ProxyStore ([SC '23](https://dl.acm.org/doi/10.1145/3581784.3607047)) and Proxy Patterns ([arXiv preprint](https://arxiv.org/abs/2407.01764)) papers.
```bib
@inproceedings{pauloski2023proxystore,
    author = {Pauloski, J. Gregory and Hayot-Sasson, Valerie and Ward, Logan and Hudson, Nathaniel and Sabino, Charlie and Baughman, Matt and Chard, Kyle and Foster, Ian},
    title = {{Accelerating Communications in Federated Applications with Transparent Object Proxies}},
    address = {New York, NY, USA},
    articleno = {59},
    booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis},
    doi = {10.1145/3581784.3607047},
    isbn = {9798400701092},
    location = {Denver, CO, USA},
    numpages = {15},
    publisher = {Association for Computing Machinery},
    series = {SC '23},
    url = {https://doi.org/10.1145/3581784.3607047},
    year = {2023}
}

@misc{pauloski2024proxystore,
    author = {J. Gregory Pauloski and Valerie Hayot-Sasson and Logan Ward and Alexander Brace and André Bauer and Kyle Chard and Ian Foster},
    title = {{Object Proxy Patterns for Accelerating Distributed Applications}},
    archiveprefix = {arXiv},
    eprint = {2407.01764},
    primaryclass = {cs.DC},
    url = {https://arxiv.org/abs/2407.01764},
    year = {2024}
}
```
