Metadata-Version: 2.1
Name: post-processor-pi
Version: 0.1.2
Summary: Post processor for Document AI
Home-page: https://github.com/paoloitaliani/post-processor-pi
Author: Paolo Italiani
Author-email: paoita@hotmail.it
License: MIT
Description: **post-processor-pi** is a Python module that exploits the output of Document AI (tool available 
        on GCP) to create a JSON file that stores the text contained in a document in an organized way, so 
        that it reflects its original structure. To work with different types of 
        documents having different structures, post-processor-pi has to be configured using the GUI
        that I developed.
        
        Installation
        ------------
        
        The easiest way to install post-processor-pi is using pip:
        
            pip install post-processor-pi
        
        Documentation
        ------------
        
        As I mentioned above in order to implement the post-processor we need to configure it first and this
        can be done using the Configuration GUI that I developed. 
        
        ### Configuration GUI 
        
        In order to start the Configuration GUI we need an example corresponding to the output of Document
        AI for one document. Below you can find the code example that enables you to start the Configuration
        GUI.
        
        ```python
        from dai_post_processor import post_processor as pp
        
        # Open one example from Document AI output
        path = "your/path/to/DocumentAI/example"
        doc_ai = pp.open_doc_ai(path)
        
        # Start Configuration GUI
        mywin = pp.configGUI(doc_ai)
        mywin.start()
        ```
        
        After running the above instructions the GUI should open.
        
        #### Configuration GUI Tuning
        
        The Configuration GUI consists of two windows:
        
        1) **Main**: here you can find the essential instructions that are needed to
         configure post-processor-pi.
            - **Select text of interest**: in this section you can select the portion
            of the pages in the documents you're interested in. In the **Page number** 
            field you can provide the page number of the Document example you want to open.
            I you press the **Draw Box** button a new window will open, and here you can
            select the portion of the page drawing a rectangle. All the text outside this
            rectangle will be ignored for all pages and documents. Once you performed the
            selection you can close the interactive window. Below you have a demo.
            <p align="center"><img src="https://github.com/paoloitaliani/post-processor-pi/raw/master/images/gui.gif" width=600></p>
        
            - Structuring Filter: in this section you can specify the first and second
            level *points* that give structure to the document. In the 
            **Main Structure** field you need to 
            specify the regex rule that lets you find all the first level *points*. If
            you press the **Add Point** button an additional field, where you can specify
            the regex rule for the second level *points*, will appear.
            
            - Output: in this section you can specify what to include in the JSON output
            of the Post Processor. You can choose to include the list of all lines and
            paragraphs in the document and the structured text. You can also choose to 
            ignore those lines that are recognised as *Headers* with the Filter Headers
            checkbox.
         
        2) **Advanced**: here you can perform some more advanced configurations:
        
           - **Line threshold**: helpful for reconstructing lines in the document when 
           Document AI fails to do so. If the difference between the y top coordinates 
           of a line and the previous one is less than **Line Threshold** in absolute
           value, these two lines are merged together to form a new *reconstructed line*.
           
           - **Line repeat**: specify how many lines from the Document AI output are 
           needed to build a single *reconstructed line*
           
           - **Paragraph Multiplier**: helpful for reconstructing paragraphs in the 
           document when  Document AI fails to do so.
           Every line in the document that has a vertical
           gap with respect to the previous line lower than the median vertical gap of
           all lines in the document multiplied by the **Paragraph Multiplier**, will be
           appended to the previous line to form a paragraph.
           
           - **Header Multiplier**: has a similar role of the Paragraph Multiplier, but 
           it's useful for finding the centered headers in the document.
           
        Once you're happy with your changes you can press the **Save** button and a 
        configuration file will be created and saved as data/config.json
        
        
        Example
        ------------
        
        In this example I'm going to show how to implement post-processor-pi on the 
        output of Document AI. The example document is a World Bank Loan Agreement and has
        this kind of structure. 
        
        <p align="center"><img src="https://github.com/paoloitaliani/post-processor-pi/raw/master/images/image1.png" width=600></p>
        
        The first level points are the dotted numbers such as **2.07** and the second 
        level points are the letters between round brackets such as **(a)**. Below you
        can find the code implementation that lets create the structured JSON using the
        config.json file that we created using the GUI.
        
        ```python
        from dai_post_processor import post_processor as pp
        import json
        
        # Open one example from Document AI output
        path = "your/path/to/DocumentAI/example"
        doc_ai = pp.open_doc_ai(path)
        
        # Open config.json file
        with open("data/config.json") as json_file:
            config_dict = json.load(json_file)
        
        # Apply the Post Processor to the Document AI output
        doc_class = pp.DocumentClass(doc_ai, config_dict)
        structured_doc = doc_class.create_json(file_name="example")
        ```
        After running the above code the structured JSON will be saved as
        data/pp_output/example.json. Below you can find a snippet of example.json, in 
        particular the *structured_output* section. As you can see the JSON file reflects
        the original structure of the document.
        
        <p align="center"><img src="https://github.com/paoloitaliani/post-processor-pi/raw/master/images/image2.png" width=600></p>
        
Platform: UNKNOWN
Description-Content-Type: text/markdown
