Metadata-Version: 2.1
Name: ferret-xai
Version: 0.2.0
Summary: A python package for NLP explainability
Home-page: https://github.com/g8a9/ferret
Author: Giuseppe Attanasio
Author-email: giuseppeattanasio6@gmail.com
License: MIT license
Keywords: ferret
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Requires-Python: >=3.7
Description-Content-Type: text/x-rst
License-File: LICENSE
License-File: AUTHORS.rst

ferret
========

|pypi badge| |docs badge|

|banner|

.. |pypi badge| image:: https://img.shields.io/pypi/v/ferret-xai.svg
    :target: https://pypi.python.org/pypi/ferret-xai
    :alt: Latest PyPI version

.. |Docs Badge| image:: https://readthedocs.org/projects/ferret/badge/?version=latest
    :alt: Documentation Status
    :scale: 100%
    :target: https://ferret.readthedocs.io/en/latest/?version=latest

.. |banner| image:: ./images/banner.png
    :alt: Ferret circular logo with the name to the right
    :scale: 100%

A python package for benchmarking interpretability techniques.

* Free software: MIT license
* Documentation: https://ferret.readthedocs.io.

.. code-block:: python

    from transformers import AutoModelForSequenceClassification, AutoTokenizer
    from ferret import Benchmark

    model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased")
    tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")

    bench = Benchmark(model, tokenizer)
    explanations = bench.explain("You look stunning!")
    evaluations = bench.evaluate_explanations(explanations)

    print(evaluations)


Features
--------

**ferret** builds on top of the transformers library. The library supports explanations using:

* Gradients
* Integrated Gradinets
* Gradient x Input word embeddings
* SHAP
* LIME

and evaluate explanations via:

Faithfulness measures.

* AOPC Comprehensiveness
* AOPC Sufficiency
* Kendall’s tau correlation with leave-one-feature out


Plausibility measures.

* AUPRC soft score plausibility
* Token f1 hard score plausibility
* Token IOU hard score plausibility

**TODOs**

* Possibility to run on select device ("cpu", "cuda")
* Sample-And-Occlusion explanations
* Discretized Integrated Gradients: https://arxiv.org/abs/2108.13654

Visualization
-------------

.. code-block:: python

    bench = Benchmark(...)

    explanations = ...
    bench.show_table(explanations)

    evaluations = bench.evaluate_explanations(explanations)
    bench.show_evaluation_table(evaluations)


Datasets evaluations
--------------------

.. code-block:: python

    bench = Benchmark(...)

    hatexdata = bench.load_dataset("hatexplain")
    dataset_explanations = bench.generate_dataset_explanations(hatexdata)
    dataset_evaluations = bench.evaluate_dataset_explanations(dataset_explanations)
    bench.show_dataset_evaluation_table(dataset_evaluations)


Credits
-------

This package was created with Cookiecutter and the *audreyr/cookiecutter-pypackage* project template.

- Cookiecutter: https://github.com/audreyr/cookiecutter
- `audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage

Logo and graphical assets made by `Luca Attanasio <https://www.behance.net/attanasiol624d>`_.


=======
History
=======

0.1.0 (2022-05-30)
------------------

* First release on PyPI.
