Metadata-Version: 2.1
Name: trustwise
Version: 2.1.15
Summary: Trustwise PyPi Package
Home-page: https://github.com/trustwiseai/trustwise
Author: Trustwise
Author-email: nihal@trustwise.ai
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp ==3.9.1
Requires-Dist: aiosignal ==1.3.1
Requires-Dist: annotated-types ==0.6.0
Requires-Dist: anyio ==4.2.0
Requires-Dist: attrs ==23.2.0
Requires-Dist: certifi ==2023.11.17
Requires-Dist: charset-normalizer ==3.3.2
Requires-Dist: click ==8.1.7
Requires-Dist: colorama ==0.4.6
Requires-Dist: dataclasses-json ==0.6.3
Requires-Dist: Deprecated ==1.2.14
Requires-Dist: distro ==1.9.0
Requires-Dist: frozenlist ==1.4.1
Requires-Dist: fsspec ==2023.12.2
Requires-Dist: docutils ==0.20.1
Requires-Dist: greenlet ==3.0.3
Requires-Dist: h11 ==0.14.0
Requires-Dist: httpcore ==1.0.2
Requires-Dist: httpx ==0.26.0
Requires-Dist: importlib-metadata ==7.0.1
Requires-Dist: jaraco.classes ==3.3.0
Requires-Dist: idna ==3.6
Requires-Dist: joblib ==1.3.2
Requires-Dist: llama-index ==0.9.36
Requires-Dist: marshmallow ==3.20.2
Requires-Dist: multidict ==6.0.4
Requires-Dist: mypy-extensions ==1.0.0
Requires-Dist: nest-asyncio ==1.6.0
Requires-Dist: networkx ==3.2.1
Requires-Dist: nltk ==3.8.1
Requires-Dist: numpy ==1.26.3
Requires-Dist: openai ==1.9.0
Requires-Dist: packaging ==23.2
Requires-Dist: pandas ==2.2.0
Requires-Dist: pydantic ==2.5.3
Requires-Dist: pydantic-core ==2.14.6
Requires-Dist: Pygments ==2.17.2
Requires-Dist: python-dateutil ==2.8.2
Requires-Dist: python-dotenv ==1.0.1
Requires-Dist: pytz ==2023.3.post1
Requires-Dist: ratelimit ==2.2.1
Requires-Dist: regex ==2023.12.25
Requires-Dist: requests ==2.31.0
Requires-Dist: six ==1.16.0
Requires-Dist: sniffio ==1.3.0
Requires-Dist: SQLAlchemy ==2.0.25
Requires-Dist: tenacity ==8.2.3
Requires-Dist: tiktoken ==0.5.2
Requires-Dist: tqdm ==4.66.1
Requires-Dist: typing-inspect ==0.9.0
Requires-Dist: typing-extensions ==4.9.0
Requires-Dist: tzdata ==2023.4
Requires-Dist: urllib3 ==2.1.0
Requires-Dist: wrapt ==1.16.0
Requires-Dist: yarl ==1.9.4
Requires-Dist: zipp ==3.17.0

# 🦉 Trustwise Safety Module

The Trustwise Safety Module is a Python package designed to evaluate the RAG pipelines of a Language Model (LLM). It provides an aggregated Safety Score, along with various other individual scores and logs events during the query and index construction of the pipelines. Users can conveniently check event logs and safety score results stored in a MongoDB backend.

## 🔧 Installation

Install the Trustwise Safety Module using pip:

```python
pip install trustwise
```

## 🔥 Features

1. Evaluate RAG pipelines of an LLM.
2. Aggregated Safety Score calculation.
3. Event logging during query and index construction.
4. Backend storage in MongoDB.
5. Support for bulk scanning of responses from LLMs.

## 🚀 Usage with Llama Index

```python

from llama_index.callbacks import CallbackManager
from trustwise.callback import TrustwiseCallbackHandler
from trustwise.request import request_eval

# Initialise Trustwise Callback Handler
tw_callback = TrustwiseCallbackHandler()

# Configure the Handler with LlamaIndex Callback Manager
callback_manager = CallbackManager([tw_callback])

# Enter Trustwise API Key
tw_api_key = 'TRUSTWISE_API_KEY'

# Set the API key using the set_api_key method
tw_callback.set_api_key(tw_api_key)

# Initialise Experiment ID for tracking scans and events
experiment_id = tw_callback.set_experiment_id()

# Rest of the llamaindex code like indexing, llm response generation comes here

###### Evaluate LLM responses #######

scores = request_eval(api_key=tw_api_key,experiment_id=experiment_id, query=query, response=response)
print(scores)
```
### 🔐 Trustwise API Key
Get your API Key by logging in through Github -> [link](http://35.199.62.235:8080/github-login)

## Contributing
We welcome contributions! If you find a bug, have a feature request, or want to contribute code, please create an issue or submit a pull request.

## 🪪 License
This project is licensed under the MIT License - see the LICENSE file for details.
