Metadata-Version: 2.1
Name: jina
Version: 3.27.20
Summary: Multimodal AI services & pipelines with cloud-native stack: gRPC, Kubernetes, Docker, OpenTelemetry, Prometheus, Jaeger, etc.
Home-page: https://github.com/jina-ai/jina/
Author: Jina AI
Author-email: hello@jina.ai
License: Apache 2.0
Download-URL: https://github.com/jina-ai/jina/tags
Project-URL: Documentation, https://jina.ai/serve
Project-URL: Source, https://github.com/jina-ai/jina/
Project-URL: Tracker, https://github.com/jina-ai/jina/issues
Description: # Jina-Serve
        <a href="https://pypi.org/project/jina/"><img alt="PyPI" src="https://img.shields.io/pypi/v/jina?label=Release&style=flat-square"></a>
        <a href="https://discord.jina.ai"><img src="https://img.shields.io/discord/1106542220112302130?logo=discord&logoColor=white&style=flat-square"></a>
        <a href="https://pypistats.org/packages/jina"><img alt="PyPI - Downloads from official pypistats" src="https://img.shields.io/pypi/dm/jina?style=flat-square"></a>
        <a href="https://github.com/jina-ai/jina/actions/workflows/cd.yml"><img alt="Github CD status" src="https://github.com/jina-ai/jina/actions/workflows/cd.yml/badge.svg"></a>
        
        Jina-serve is a framework for building and deploying AI services that communicate via gRPC, HTTP and WebSockets. Scale your services from local development to production while focusing on your core logic.
        
        ## Key Features
        
        - Native support for all major ML frameworks and data types
        - High-performance service design with scaling, streaming, and dynamic batching
        - LLM serving with streaming output
        - Built-in Docker integration and Executor Hub
        - One-click deployment to Jina AI Cloud
        - Enterprise-ready with Kubernetes and Docker Compose support
        
        <details>
        <summary><strong>Comparison with FastAPI</strong></summary>
        
        Key advantages over FastAPI:
        
        - DocArray-based data handling with native gRPC support
        - Built-in containerization and service orchestration
        - Seamless scaling of microservices
        - One-command cloud deployment
        </details>
        
        ## Install 
        
        ```bash
        pip install jina
        ```
        
        See guides for [Apple Silicon](https://jina.ai/serve/get-started/install/apple-silicon-m1-m2/) and [Windows](https://jina.ai/serve/get-started/install/windows/).
        
        ## Core Concepts
        
        Three main layers:
        - **Data**: BaseDoc and DocList for input/output
        - **Serving**: Executors process Documents, Gateway connects services
        - **Orchestration**: Deployments serve Executors, Flows create pipelines
        
        ## Build AI Services
        
        Let's create a gRPC-based AI service using StableLM:
        
        ```python
        from jina import Executor, requests
        from docarray import DocList, BaseDoc
        from transformers import pipeline
        
        
        class Prompt(BaseDoc):
            text: str
        
        
        class Generation(BaseDoc):
            prompt: str
            text: str
        
        
        class StableLM(Executor):
            def __init__(self, **kwargs):
                super().__init__(**kwargs)
                self.generator = pipeline(
                    'text-generation', model='stabilityai/stablelm-base-alpha-3b'
                )
        
            @requests
            def generate(self, docs: DocList[Prompt], **kwargs) -> DocList[Generation]:
                generations = DocList[Generation]()
                prompts = docs.text
                llm_outputs = self.generator(prompts)
                for prompt, output in zip(prompts, llm_outputs):
                    generations.append(Generation(prompt=prompt, text=output))
                return generations
        ```
        
        Deploy with Python or YAML:
        
        ```python
        from jina import Deployment
        from executor import StableLM
        
        dep = Deployment(uses=StableLM, timeout_ready=-1, port=12345)
        
        with dep:
            dep.block()
        ```
        
        ```yaml
        jtype: Deployment
        with:
         uses: StableLM
         py_modules:
           - executor.py
         timeout_ready: -1
         port: 12345
        ```
        
        Use the client:
        
        ```python
        from jina import Client
        from docarray import DocList
        from executor import Prompt, Generation
        
        prompt = Prompt(text='suggest an interesting image generation prompt')
        client = Client(port=12345)
        response = client.post('/', inputs=[prompt], return_type=DocList[Generation])
        ```
        
        ## Build Pipelines
        
        Chain services into a Flow:
        
        ```python
        from jina import Flow
        
        flow = Flow(port=12345).add(uses=StableLM).add(uses=TextToImage)
        
        with flow:
            flow.block()
        ```
        
        ## Scaling and Deployment
        
        ### Local Scaling
        
        Boost throughput with built-in features:
        - Replicas for parallel processing
        - Shards for data partitioning
        - Dynamic batching for efficient model inference
        
        Example scaling a Stable Diffusion deployment:
        
        ```yaml
        jtype: Deployment
        with:
         uses: TextToImage
         timeout_ready: -1
         py_modules:
           - text_to_image.py
         env:
          CUDA_VISIBLE_DEVICES: RR
         replicas: 2
         uses_dynamic_batching:
           /default:
             preferred_batch_size: 10
             timeout: 200
        ```
        
        ### Cloud Deployment
        
        #### Containerize Services
        
        1. Structure your Executor:
        ```
        TextToImage/
        ├── executor.py
        ├── config.yml
        ├── requirements.txt
        ```
        
        2. Configure:
        ```yaml
        # config.yml
        jtype: TextToImage
        py_modules:
         - executor.py
        metas:
         name: TextToImage
         description: Text to Image generation Executor
        ```
        
        3. Push to Hub:
        ```bash
        jina hub push TextToImage
        ```
        
        #### Deploy to Kubernetes
        ```bash
        jina export kubernetes flow.yml ./my-k8s
        kubectl apply -R -f my-k8s
        ```
        
        #### Use Docker Compose
        ```bash
        jina export docker-compose flow.yml docker-compose.yml
        docker-compose up
        ```
        
        #### JCloud Deployment
        
        Deploy with a single command:
        ```bash
        jina cloud deploy jcloud-flow.yml
        ```
        
        ## LLM Streaming
        
        Enable token-by-token streaming for responsive LLM applications:
        
        1. Define schemas:
        ```python
        from docarray import BaseDoc
        
        
        class PromptDocument(BaseDoc):
            prompt: str
            max_tokens: int
        
        
        class ModelOutputDocument(BaseDoc):
            token_id: int
            generated_text: str
        ```
        
        2. Initialize service:
        ```python
        from transformers import GPT2Tokenizer, GPT2LMHeadModel
        
        
        class TokenStreamingExecutor(Executor):
            def __init__(self, **kwargs):
                super().__init__(**kwargs)
                self.model = GPT2LMHeadModel.from_pretrained('gpt2')
        ```
        
        3. Implement streaming:
        ```python
        @requests(on='/stream')
        async def task(self, doc: PromptDocument, **kwargs) -> ModelOutputDocument:
            input = tokenizer(doc.prompt, return_tensors='pt')
            input_len = input['input_ids'].shape[1]
            for _ in range(doc.max_tokens):
                output = self.model.generate(**input, max_new_tokens=1)
                if output[0][-1] == tokenizer.eos_token_id:
                    break
                yield ModelOutputDocument(
                    token_id=output[0][-1],
                    generated_text=tokenizer.decode(
                        output[0][input_len:], skip_special_tokens=True
                    ),
                )
                input = {
                    'input_ids': output,
                    'attention_mask': torch.ones(1, len(output[0])),
                }
        ```
        
        4. Serve and use:
        ```python
        # Server
        with Deployment(uses=TokenStreamingExecutor, port=12345, protocol='grpc') as dep:
            dep.block()
        
        
        # Client
        async def main():
            client = Client(port=12345, protocol='grpc', asyncio=True)
            async for doc in client.stream_doc(
                on='/stream',
                inputs=PromptDocument(prompt='what is the capital of France ?', max_tokens=10),
                return_type=ModelOutputDocument,
            ):
                print(doc.generated_text)
        ```
        
        ## Support
        
        Jina-serve is backed by [Jina AI](https://jina.ai) and licensed under [Apache-2.0](./LICENSE).
        
Keywords: jina cloud-native cross-modal multimodal neural-search query search index elastic neural-network encoding embedding serving docker container image video audio deep-learning mlops
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Unix Shell
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Database :: Database Engines/Servers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Description-Content-Type: text/markdown
Provides-Extra: core
Provides-Extra: numpy
Provides-Extra: protobuf
Provides-Extra: grpcio
Provides-Extra: grpcio-reflection
Provides-Extra: grpcio-health-checking
Provides-Extra: pyyaml
Provides-Extra: packaging
Provides-Extra: docarray
Provides-Extra: jina-hubble-sdk
Provides-Extra: jcloud
Provides-Extra: opentelemetry-api
Provides-Extra: opentelemetry-instrumentation-grpc
Provides-Extra: perf
Provides-Extra: devel
Provides-Extra: uvloop
Provides-Extra: standard
Provides-Extra: prometheus_client
Provides-Extra: opentelemetry-sdk
Provides-Extra: opentelemetry-exporter-otlp
Provides-Extra: opentelemetry-exporter-prometheus
Provides-Extra: opentelemetry-instrumentation-aiohttp-client
Provides-Extra: opentelemetry-instrumentation-fastapi
Provides-Extra: standrad
Provides-Extra: opentelemetry-exporter-otlp-proto-grpc
Provides-Extra: fastapi
Provides-Extra: uvicorn[standard]
Provides-Extra: docker
Provides-Extra: pathspec
Provides-Extra: filelock
Provides-Extra: requests
Provides-Extra: websockets
Provides-Extra: pydantic
Provides-Extra: python-multipart
Provides-Extra: aiofiles
Provides-Extra: aiohttp
Provides-Extra: test
Provides-Extra: scipy
Provides-Extra: Pillow
Provides-Extra: pytest
Provides-Extra: pytest-timeout
Provides-Extra: pytest-mock
Provides-Extra: pytest-cov
Provides-Extra: coverage
Provides-Extra: pytest-repeat
Provides-Extra: pytest-asyncio
Provides-Extra: pytest-reraise
Provides-Extra: flaky
Provides-Extra: mock
Provides-Extra: requests-mock
Provides-Extra: pytest-custom_exit_code
Provides-Extra: black
Provides-Extra: kubernetes
Provides-Extra: pytest-kind
Provides-Extra: pytest-lazy-fixture
Provides-Extra: torch
Provides-Extra: cicd
Provides-Extra: psutil
Provides-Extra: strawberry-graphql
Provides-Extra: sgqlc
Provides-Extra: bs4
Provides-Extra: jsonschema
Provides-Extra: portforward
Provides-Extra: tensorflow
Provides-Extra: opentelemetry-test-utils
Provides-Extra: prometheus-api-client
Provides-Extra: watchfiles
Provides-Extra: urllib3
Provides-Extra: all
