Metadata-Version: 2.1
Name: flageval-serving
Version: 0.2.1
Summary: 
Author: FlagEval
Author-email: flageval@baai.ac.cn
Requires-Python: >=3.8,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: blinker (>=1.6.2,<2.0.0)
Requires-Dist: cached-property (>=1.5.2,<2.0.0)
Requires-Dist: click (>=8.1.3,<9.0.0)
Requires-Dist: dataclasses-json (>=0.5.7,<0.6.0)
Requires-Dist: flask (>=2.3.2,<3.0.0)
Requires-Dist: gevent (>=22.10.2,<23.0.0)
Requires-Dist: gunicorn (>=20.1.0,<21.0.0)
Requires-Dist: requests (>=2.30.0,<3.0.0)
Requires-Dist: tqdm (>=4.65.0,<5.0.0)
Description-Content-Type: text/markdown

# FlagEval Serving

> Serving Framework of AI Models for Evaluating on FlagEval Platform.


## Installation

``` shell
pip install --upgrade flageval-serving
```

## Usage

1. **Model**: of course we have a model that is ready be evaluated, let's assume it lives in the path: `/path/to/model`;
2. Then we can write our service code, let's put the service code in `service.py` or './tests/service.py' and take a NLP model as the example:


    ``` python
    from flageval.serving.service import NLPModelService, NLPEvalRequest, NLPEvalResponse, NLPCompletion


    class DemoService(NLPModelService):
        def global_init(self, model_path: str):
            print("Initial model with path", model_path)

        def infer(self, req: NLPEvalRequest) -> NLPEvalResponse:
            return NLPEvalResponse(
                completions=[
                    NLPCompletion(
                        text='Hello, world!',
                        tokens='Hello, world!',
                    ),
                ]
            )

    ```

3. Finally, we use the `flageval-serving` command to serve:

    ```shell
    flageval-serving --service service:DemoService dev /path/to/model  # start a development server
    flageval-serving --service service:DemoService run /path/to/model  # start a production server
    ```
## Dockerfile
FlagEval evaluation platform construction image
