Metadata-Version: 2.1
Name: kafka-lag-monitor
Version: 0.1.1
Summary: 
Author: Ninad-Bhangui
Requires-Python: >=3.11,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: pandas (>=2.1.4,<3.0.0)
Requires-Dist: paramiko (>=3.4.0,<4.0.0)
Requires-Dist: rich (>=13.7.0,<14.0.0)
Requires-Dist: tabulate (>=0.9.0,<0.10.0)
Requires-Dist: typer[all] (>=0.9.0,<0.10.0)
Description-Content-Type: text/markdown

# Kafka group lag aggregate monitor
This utility can currently 
1. Accept `kafka-consumer-groups` output like file from stdin and print aggregated output to stdout. 
2. ssh to a remote machine with kafka running on it, run `kafka-consumer-groups`, for multiple groups, collect the output, group by group and topic and finally print average and max lag.

Currently Tested on: Python 3.11.3 and macos

## Installation

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

## Run

### stdin-mode
Run the below to get a general idea of how the output would look like.
```bash
cat examples/example1.txt | python monitor.py stdin-mode
```
1. Add option -v to get verbose output
2. Try option --tablefmt psql to get tabular output just like psql. (Can try any format supported [here](https://github.com/astanin/python-tabulate#table-format))

### remote-mode
```bash
python monitor.py remote-mode -v --remote ubuntu@127.0.0.1 -i ~/.ssh/key.pem --bootstrap-server 127.0.0.1:9000 --groups group1 group2
```

## TODO
1. Explore using tui which can monitor lag live by recording last n entries and calculating lag increase/decrease.
2. Currently stdin-mode supports only one file? Look for ways to concat multiple outputs and aggregate?

