Metadata-Version: 2.1
Name: google-jetstream
Version: 0.1.1
Summary: A throughput and memory optimized engine for LLM inference on TPU and GPU.
Home-page: https://github.com/google/JetStream
Author: Google LLC
License: UNKNOWN
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3.10
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: AUTHORS

# JetStream - A throughput and memory optimized engine for LLM inference on TPU and GPU

## About

JetStream is a fast library for LLM inference and serving on TPU and GPU.

## Getting Started

### Run local server & Testing

Use the following commands to run a server locally:
```
# Start a server
python -m jetstream.core.implementations.mock.server

# Test local mock server
python -m jetstream.core.tools.requester

# Load test local mock server
python -m jetstream.core.tools.load_tester

```

### Test core modules
```
# Test JetStream core orchestrator
python -m jetstream.core.orchestrator_test

# Test JetStream core server library
python -m jetstream.core.server_test

# Test mock JET engine implementation
python -m jetstream.engine.mock_engine_test

# Test mock JET engine implementation
python -m jetstream.engine.utils_test

```


