Metadata-Version: 2.1
Name: ASUllmAPI
Version: 1.0.0
Summary: A simple python package to facilitate connection to ASU LLM API
Author-email: Stella Wenxing Liu <stellawenxingliu@gmail.com>
Project-URL: Homepage, https://github.com/ASU/aiml-ssmdv-student-support-ml-data-visualization
Project-URL: Issues, https://github.com/ASU/aiml-ssmdv-student-support-ml-data-visualization/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE

# ASU LLM API
This package allows individuals at Arizona State University to access ASU GPT through API. You will need API access token, API endpoints in order to use this package.

## Install
Install via pip.
```
pip install ASUllmAPI
```

## Usage
- Use `query_model_info_api` module to pull list of available models from ASU GPT. See [Models](##Models) section below for details.

**Example code:**
```markdown
from ASUllmAPI import query_model_info_api, model_provider_mapper, model_list

model_info = query_model_info_api(access_token=access_token, url=model_list_url)
model_info
>>> [{"model": "gpt4", "owner": "openai", "provider": "openai"}, ...] 

model_provider_mapper(model_info)
>>> {"mistral-8x7b": "aws", "titang1express": "aws", ...}

model_list(model_info)
>>> ["gpt4", "gpt3_5", "claudeinstant", ...]
```

- Use `query_llm` module to query ASU GPT.

**Example code:**
```markdown
from ASUllmAPI import ModelConfig, query_llm

model = ModelConfig(name="gpt4", provider="openai", access_token=access_token, api_url=llm_api_url)
query_llm(model=model, query="Where is Phoenix Arizona?")
>>> "Phoenix is the capital city of the U.S. state of Arizona. It's located in the south-central part of the state, approximately halfway between Tucson to the southeast and Flagstaff to the north. With its coordinates being 33.4484° N, 112.0740° W, it lies within the Sonoran Desert surrounded by mountains on all sides."
```

## Models

### Model Params
```markdown
"model_params": {
    "temperature": float,
    "max_tokens": int,
    "top_p": float,
    "top_k": int,
},
"enable_search": bool,
"search_params": {
    "collection": "asu",
    "top_k": int,
}
```
#### temperature
Randomness and Diversity parameter. Use a lower value to decrease randomness in the response.
#### top_p
Randomness and Diversity parameter. Use a lower value to ignore less probable options.
#### top_k
Randomness and Diversity parameter. The number of token choices the model uses to generate the next token.
#### max_tokens
The maximum number of tokens in the generated response.

## Provider: AES
Checkout [Amazon Bedrock User Guide](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html) for base model ids and versions.

### Titan Text Models

Model IDs:
- titang1lite
- titang1express

[Model documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-text.html):

| Category                 | Parameter   | Key         | Minimum | Maximum | Default |
|--------------------------|-------------|-------------|---------|---------|---------|
| Randomness and diversity | Temperature | temperature | 0       | 1       | 0.5     |
| Randomness and diversity | Top P       | top_p       | 0       | 1       | 1       |
| Length                   | max_tokens  | max_tokens  | 0       | 8000    | 512     |


### Anthropic Claude models

Model IDs:
- claude2_1 
- claude2 
- claude1_3 
- claudeinstant

[Model documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-claude.html):

| Category                 | Parameter   | Key         | Minimum | Maximum | Default |
|--------------------------|-------------|-------------|---------|---------|---------|
| Randomness and diversity | Temperature | temperature | 0       | 1       | 0.5     |
| Randomness and diversity | Top P       | top_p       | 0       | 1       | 0.5     |
| Randomness and diversity | Top K       | top_k       | 0       | 500     | 250     |
| Length                   | max_tokens  | max_tokens  | 0       | 4096    | 200     |

### AI21 Labs Jurassic-2 models

Model IDs:
- j2ultra
- j2mid

[Model documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-jurassic2.html)

| Category                 | Parameter   | Key         | Minimum | Maximum | Default |
|--------------------------|-------------|-------------|---------|---------|---------|
| Randomness and diversity | Temperature | temperature | 0       | 1       | 0.5     |
| Randomness and diversity | Top P       | top_p       | 0       | 1       | 0.5     |
| Length                   | max_tokens  | max_tokens  | 0       | 8191    | 200     |

### Cohere Command Models

Model IDs:
- command
- commandlight

[Model documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-cohere-command.html)

| Category                 | Parameter   | Key         | Minimum | Maximum | Default |
|--------------------------|-------------|-------------|---------|---------|---------|
| Randomness and diversity | Temperature | temperature | 0       | 5       | 0.9     |
| Randomness and diversity | Top P       | top_p       | 0       | 1       | 0.75    |
| Randomness and diversity | Top K       | top_k       | 0       | 500     | 0       |
| Length                   | max_tokens  | max_tokens  | 1       | 4096    | 20      |

### Meta Llama2 Models

Model IDs:
- llama2-13b

[Model Documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-meta.html)

| Category                 | Parameter   | Key         | Minimum | Maximum | Default |
|--------------------------|-------------|-------------|---------|---------|---------|
| Randomness and diversity | Temperature | temperature | 0       | 1       | 0.5     |
| Randomness and diversity | Top P       | top_p       | 0       | 1       | 0.9     |
| Length                   | max_tokens  | max_tokens  | 1       | 2048    | 512     |


## Provider: Azure

[Model Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#model-summary-table-and-region-availability)

| Model name | Model ID         | Max Request (tokens) |
|------------|------------------|----------------------|
| gpt3_5     | gpt-35-turbo     | 4096                 |
| gpt3_5-16k | gpt-35-turbo-16k | 16384                |
| gpt4       | gpt-4            | 8192                 |
| gpt4-32k   | gpt-4-32k        | 32768                |

## Provider: GCP

[Model Documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/models)

| Model name          | Model ID       | Max Input Tokens     | Max Output Tokens |
|---------------------|----------------|----------------------|-------------------|
| PaLM 2 for Chat     | chat-bison     | 8192                 | 1024              |
| PaLM 2 for Chat 32k | chat-bison-32k | 32768 (input+output) | 8192              |
