Metadata-Version: 2.1
Name: dimon-cli
Version: 0.1.0
Summary: dimensional monitoring cli
Home-page: https://github.com/InfuseAI/dimon
Author: InfuseAI
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: Click
Requires-Dist: pandas
Requires-Dist: python-dotenv
Requires-Dist: llama-index
Requires-Dist: llama-index-vector-stores-elasticsearch
Requires-Dist: llama-index-llms-ollama
Requires-Dist: llama-index-embeddings-ollama
Requires-Dist: rich

# dimon: Your Canonical Answer Management Tool
dimon is a command-line interface (CLI) tool for evaluating and managing golden dataset for evaluation your retrival mechnism.

## Installation
You can install dimon directly from PyPI:

```
pip install dimon
```

## Prepare env file

Create a file named `.env`, or copy it from `.env.example`.
This file should contain the default configuration values for dimon. Here is an example of what the `.env` file might look like:

```
DATABASE_URL=http://localhost:9200
DATABASE_INDEX=your_index
USERNAME=elastic
PASSWORD=your_password_here
DATASET_PATH=/path/to/your/dataset
```

Replace the values with the actual ones.

Note: Currently, the script only supports ElasticSearch as the vector database. Ensure that your ElasticSearch instance is correctly set up and accessible via the URL provided in the DATABASE_URL parameter. The DATABASE_INDEX should point to the specific index within ElasticSearch that you want to use for storing and querying vectors.


## The CLI tool provides three main commands:

### evaluate
This command evaluates a specified model on a given dataset. It computes embeddings for each document in the corpus, retrieves the top-k most similar documents for each query, and calculates the Mean Reciprocal Rank (MRR) for evaluation.

```
dimon evaluate --database_url 'http://localhost:9200' --dataset 'golden_dataset.csv'
```

### diff
This command reads a report file, filters out entries with MRR below a certain threshold

```
dimon diff --file_name 'mrr_report.csv' --threshold 0.2
```

### merge
This command reads a report file, filters out entries marked as 'Accept', and simulates a merging process.

```
dimon merge --report_name 'to_confirmed_change.csv'
```
