Metadata-Version: 2.1
Name: partial-json-parser
Version: 0.1.2
Summary: Parse partial JSON generated by LLM
Keywords: JSON parser LLM nlp
Home-page: https://promplate.dev
Author-Email: Muspi Merol <me@promplate.dev>
License: MIT
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Project-URL: Repository, https://github.com/promplate/partial-json-parser
Project-URL: Homepage, https://promplate.dev
Requires-Python: >=3.8
Provides-Extra: playground
Requires-Dist: rich; extra == "playground"
Description-Content-Type: text/markdown

# Partial JSON Parser

Sometimes we need **LLM (Large Language Models)** to produce **structural information** instead of natural language. The easiest way is to use JSON.

But before receiving the last token of response, the JSON is broken, which means you can't use `JSON.parse` to decode it. But we still want to stream the data to the user.

Here comes `partial-json-parser`, a lightweight and customizable library for parsing partial JSON strings. Here is a [demo](https://promplate.dev/partial-json-parser).

(Note that there is [a JavaScript implementation](https://github.com/promplate/partial-json-parser-js) too)

## Installation

```sh
pip install partial-json-parser # or poetry / pdm / conda
```

`partial-json-parser` is implemented purely in Python, with good type hints.

## Usage

### Importing the library

You can import the `loads` function and the `Allow` object from the library like this:

```py
from partial_json_parser import loads, Allow
```

The `Allow` object is just an Enum for options. It determines what types can be partial. types not included in `allow` only appears after its completion can be ensured.

### Parsing complete / partial JSON strings

The `loads` function works just like the built-in `json.loads` when parsing a complete JSON string:

```py
result = loads('{"key":"value"}')
print(result)  # Outputs: {'key': 'value'}
```

You can parse a partial JSON string by passing an additional parameter to the `loads` function. This parameter is a **bitwise OR** of the constants from the `Allow` flag:

(Note that you can directly import the constants you need from `partial-json-parser.options`)

```py
from partial_json_parser import loads, Allow
from partial_json_parser.options import STR, OBJ

result = loads('{"key": "v', STR | OBJ)
print(result)  # Outputs: {'key': 'v'}
```

In this example, `Allow.STR` tells the parser that it's okay if a string is incomplete, and `Allow.OBJ` tells the parser so as a dict. The parser then try to return as much data as it can.

If you don't allow partial strings, then it will not add `"key"` to the object because `"v` is not close:

```py
result = loads('{"key": "v', OBJ)
print(result)  # Outputs: {}

result = loads('{"key": "value"', OBJ)
print(result)  # Outputs: {'key': 'value'}
```

Similarity, you can parse partial lists or even partial special values if you allow it:

(Note that `allow` defaults to `Allow.ALL`)

```py
result = loads('[ {"key1": "value1", "key2": [ "value2')
print(result)  # Outputs: [{'key1': 'value1', 'key2': ['value2']}]

result = loads("-Inf")
print(result)  # Outputs: -inf
```

### Handling malformed JSON

If the JSON string is malformed, the `parse` function will throw an error:

```py
loads("wrong")  # MalformedJSON: Malformed node or string on line 1
```

## API Reference

### loads(json_string, [allow_partial])

- `json_string` `<string>`: The JSON string to parse.
- `allow_partial` `<Allow | int>`: Specify what kind of partialness is allowed during JSON parsing (default: `Allow.ALL`).

Returns the parsed Python value.

### Allow

Enum class that specifies what kind of partialness is allowed during JSON parsing. It has the following members:

- `STR`: Allow partial string.
- `NUM`: Allow partial number.
- `ARR`: Allow partial array.
- `OBJ`: Allow partial object.
- `NULL`: Allow partial null.
- `BOOL`: Allow partial boolean.
- `NAN`: Allow partial NaN.
- `INFINITY`: Allow partial Infinity.
- `_INFINITY`: Allow partial -Infinity.
- `INF`: Allow both partial Infinity and -Infinity.
- `SPECIAL`: Allow all special values.
- `ATOM`: Allow all atomic values.
- `COLLECTION`: Allow all collection values.
- `ALL`: Allow all values.

## Testing

To run the tests for this library, you should clone the repository and install the dependencies:

```sh
git clone https://github.com/promplate/partial-json-parser.git
cd partial-json-parser
pdm install
```

Then, you can run the tests using [Hypothesis](https://hypothesis.works/) and [Pytest](https://pytest.org/):

```sh
pdm test
```

Please note that while we strive to cover as many edge cases as possible, it's always possible that some cases might not be covered.

## License

This project is licensed under the MIT License.
