Metadata-Version: 2.1
Name: prediction-market-agent-tooling
Version: 0.11.0
Summary: Tools to benchmark, deploy and monitor prediction market agents.
Author: Gnosis
Requires-Python: >=3.10,<3.12
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Provides-Extra: google
Provides-Extra: langchain
Requires-Dist: autoflake (>=2.2.1,<3.0.0)
Requires-Dist: cron-validator (>=1.0.8,<2.0.0)
Requires-Dist: eth-typing (>=4.0.0,<5.0.0)
Requires-Dist: functions-framework (>=3.5.0,<4.0.0)
Requires-Dist: google-api-python-client (==2.95.0) ; extra == "google"
Requires-Dist: google-cloud-functions (>=1.16.0,<2.0.0)
Requires-Dist: google-cloud-resource-manager (>=1.12.0,<2.0.0)
Requires-Dist: google-cloud-secret-manager (>=2.18.2,<3.0.0)
Requires-Dist: isort (>=5.13.2,<6.0.0)
Requires-Dist: langchain (>=0.1.9,<0.2.0) ; extra == "langchain"
Requires-Dist: langchain-community (>=0.0.19)
Requires-Dist: langchain-openai (>=0.0.8,<0.0.9) ; extra == "langchain"
Requires-Dist: loguru (>=0.7.2,<0.8.0)
Requires-Dist: numpy (>=1.26.4,<2.0.0)
Requires-Dist: pydantic (>=2.6.1,<3.0.0)
Requires-Dist: pydantic-settings (>=2.1.0,<3.0.0)
Requires-Dist: scikit-learn (>=1.4.0,<2.0.0)
Requires-Dist: streamlit (>=1.31.0,<2.0.0)
Requires-Dist: subgrounds (>=1.8.1,<2.0.0)
Requires-Dist: tabulate (>=0.9.0,<0.10.0)
Requires-Dist: tqdm (>=4.66.2,<5.0.0)
Requires-Dist: typer (>=0.9.0,<0.10.0)
Requires-Dist: types-pytz (>=2024.1.0.20240203,<2025.0.0.0)
Requires-Dist: types-requests (>=2.31.0.20240106,<3.0.0.0)
Requires-Dist: web3 (>=6.15.1,<7.0.0)
Description-Content-Type: text/markdown

# Prediction Market Agent Tooling

Tooling for benchmarking, deploying and monitoring agents for prediction market applications.

## Setup

Install the project dependencies with `poetry`, using Python >=3.10:

```bash
python3.10 -m pip install poetry
python3.10 -m poetry install
python3.10 -m poetry shell
```

Create a `.env` file in the root of the repo with the following variables:

Deploying and monitoring agents using GCP requires that you set up the gcloud CLI (see [here](https://cloud.google.com/sdk/docs/install) for installation instructions, and use `gcloud auth login` to authorize.)

```bash
MANIFOLD_API_KEY=...
BET_FROM_ADDRESS=...
BET_FROM_PRIVATE_KEY=...
OPENAI_API_KEY=...
```

## Benchmarking

Create a benchmarkable agent by subclassing the `AbstractBenchmarkedAgent` base class, and plug in your agent's research and prediction functions into the `predict` method.

Use the `Benchmarker` class to compare your agent's predictions vs. the 'wisdom of the crowd' on a set of markets from your chosen prediction market platform.

For example:

```python
import prediction_market_agent_tooling.benchmark.benchmark as bm
from prediction_market_agent_tooling.markets.markets import MarketType, get_binary_markets

benchmarker = bm.Benchmarker(
    markets=get_binary_markets(limit=10, market_type=MarketType.MANIFOLD),
    agents=[...],
)
benchmarker.run_agents()
md = benchmarker.generate_markdown_report()
```

This produces a markdown report comparing agents:

![Benchmark results](assets/comparison-report.png)

## Deploying

Create a deployable agent by subclassing the `DeployableAgent` base class, and implementing the 

For example, deploy an agent that randomly picks an outcome:

```python
import random
from prediction_market_agent_tooling.deploy.agent import DeployableAgent
from prediction_market_agent_tooling.markets.agent_market import AgentMarket

class DeployableCoinFlipAgent(DeployableAgent):
    def answer_binary_market(self, market: AgentMarket) -> bool | None:
        return random.choice([True, False])

DeployableCoinFlipAgent().deploy_gcp(...)
```

## Monitoring

Monitor the performance of the agents deployed to GCP, as well as meta-metrics of the prediction market platforms they are deployed to.

This runs as a streamlit app on a localhost server, executed with:

```bash
PYTHONPATH=. streamlit run examples/monitor/monitor.py
```

Which launches in the browser:

![Monitoring](assets/monitoring.png)

## The Market Platforms

The following markets platforms are supported:

- [Manifold](https://manifold.markets/)
- [AIOmen](https://aiomen.eth.limo/)
- [Polymarket](https://polymarket.com/) - Benchmarking only. Deploy and monitor TODO

## Contributing

See the [Issues](https://github.com/gnosis/prediction-market-agent-tooling/issues) for ideas of things that need fixing or implementing. Or come up with your own :D.

We use `mypy` for static type checking, and `isort`, `black` and `autoflake` for linting. These all run as steps in CI.

