Metadata-Version: 2.1
Name: evaluateqa
Version: 0.1.1
Summary: 
Home-page: https://github.com/MihailSalnikov/EvaluateQA
License: MIT
Keywords: QA,KGQA,ODQA,Evaluate
Author: Mikhail Salnikov
Author-email: 2613180+MihailSalnikov@users.noreply.github.com
Requires-Python: >=3.8,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: joblib (>=1.2.0,<2.0.0)
Requires-Dist: pandas (>=2.0.1,<3.0.0)
Requires-Dist: regex (>=2023.3.23,<2024.0.0)
Requires-Dist: ujson (>=5.7.0,<6.0.0)
Project-URL: Repository, https://github.com/MihailSalnikov/EvaluateQA
Description-Content-Type: text/markdown

[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

# EvaluateQA

Package for evaluate QA datasets and Leaderboard with SOTA approaches

## Install

```bash
pip install evaluateqa
```

## Supported datasets


### [Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering](https://github.com/amazon-science/mintaka)

```python
from evaluateqa.mintaka import evaluate

predictions = {
    '9ace9041': 'Q90',
    '9ace9042': 3,
    ...
}

results = evaluate(
    predictions,
    split='test',
    mode='kg',
    lang='en',
)
```




