Metadata-Version: 2.1
Name: timed-batch-worker
Version: 0.0.4
Summary: Python library for efficiently batch-processing workloads asynchronously with batch-size- and time-based flushes.
Home-page: https://github.com/Minibrams/timed-batch-worker
Author: Anders Brams
Author-email: anders@brams.dk
Keywords: batch,worker,asynchronous,threading,timed
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3
Description-Content-Type: text/markdown
License-File: LICENSE

# Timed Batch Worker

Python library for efficiently batch-processing workloads asynchronously with batch-size- and time-based flushes using short-circuiting events.

## Installation
```bash
pip install timed-batch-worker
```

## Usage
```python
from timed_batch_worker import TimedBatchWorker


# Handle the batched items
def handle_batch(items):
  http.post('/api/logs/bulk', data=items)


# Create a worker that flushes the queue when:
# 1. The batch contains 20 objects, or
# 2. Five seconds have passed since the last flush
worker = TimedBatchWorker(
  handler=handle_batch,
  flush_interval=5,
  flush_batch_size=20,
)

# Worker started in separate thread to avoid throttling main event loop
# in case the batch handler includes synchronous workloads.
worker.start()

# The worker can be stopped by calling stop(), but
# it's important to note that only the flushing and batch-size-checking
# workers stop immediately. The worker checking for the flush interval
# will stop at the next flush interval.
worker.stop()


# Add objects to the current batch at any time
@app.get('/users/create')
def create_user():
  ...

  # Hand off an object to the worker
  worker.enqueue({
    'event': 'USER_CREATED',
    'data': user
  })

  ...
```



