Metadata-Version: 2.1
Name: remoteinference
Version: 0.0.5
Summary: Remote inference for language models
Home-page: UNKNOWN
Author: Jaris Küken
Author-email: jaris.kueken@gmail.com
License: UNKNOWN
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown

# llm_inference

Simple package to perform remote inference on language models of different providers.

## getting started
Install the package
```python
pip install remoteinference
```
If you have a LLM running on a remote server using llama.cpp for example you can initalize the model by running:
```python
from remoteinference.models.models import LlamaCPPLLM
from remoteinference.util.config import ServerConfig

# initalize the model
cfg = ServerConfig(server_address="localhost", server_port=8080)
model = LlamaCPPLLM(cfg)

# run simple completion
completion = model.completion("How is the weather today?",
                               temperature=0.5,
                               max_tokens=50)

```



