Metadata-Version: 2.1
Name: fpipe
Version: 0.0.13
Summary: Library for working with file-likes as piped streams
Home-page: https://github.com/vkvam/fpipe
Author: Vemund Kvam
Author-email: vemund.kvam@gmail.com
License: MIT
Description: [![Build Status](https://api.travis-ci.org/vkvam/fpipe.svg?branch=master)](https://travis-ci.org/vkvam/fpipe)
        [![codecov](https://codecov.io/gh/vkvam/fpipe/branch/master/graph/badge.svg)](https://codecov.io/gh/vkvam/fpipe)
        [![PyPI](https://img.shields.io/pypi/v/fpipe)](https://pypi.org/project/fpipe/)
        ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/fpipe)
        [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
        # fPipe
        
        python framework for data manipulation and metadata extraction built around the python file-like api.
        
        *Disclaimer: framework is currently in Alpha, production use discouraged*
        
        
        ### Installing
        
        for S3 support you need boto3
        
        ```bash
        brew install python3
        # apt, yum, apk...
        pip3 install fpipe
        # Optional
        pip3 install boto3
        ```
        
        ### Getting started
        ##### Simple example
        *Calculates size and md5 of stream, while storing stream to disk and prints content. When file is read finished, md5 is ready and printed* 
        
        ```python
        from fpipe.file import ByteFile
        from fpipe.gen import LocalGen, MetaGen
        from fpipe.meta import Path, SizeCalculated, MD5Calculated
        from fpipe.workflow import WorkFlow
        
        workflow = WorkFlow(
            LocalGen(pass_through=True),
            MetaGen(SizeCalculated, MD5Calculated)
        )
        
        for stream in workflow.compose(ByteFile(b'x' * 10, Path('x.dat')), ByteFile(b'y' * 20, Path('y.dat'))):
            print(f'\n{"-"*46}\n')
            print("Path name:", stream.meta(Path).value)
            print("Stream content: ", stream.file.read().decode('utf-8'))
            with open(stream.meta(Path).value) as f:
                print("File content:", f.read())
            print("Stream md5:", stream.meta(MD5Calculated).value)
            print("Stream size:", stream.meta(SizeCalculated).value)
        ```
        
        ##### Subprocess script example
        *Stores original stream, calculates md5, encrypts using cli, stores, calculates md5, decrypts using cli and stores. Using flush_iter() we know all files have been completely read(), so MD5Calculated will be readable.*
        
        ```python
        from fpipe.file import ByteFile
        from fpipe.gen import LocalGen, MetaGen, ProcessGen
        from fpipe.meta import Path, MD5Calculated
        from fpipe.workflow import WorkFlow
        
        workflow = WorkFlow(
            MetaGen(MD5Calculated),
            LocalGen(pass_through=True),
        
            ProcessGen("gpg --batch --symmetric --passphrase 'secret'"),
            MetaGen(MD5Calculated),
            LocalGen(pass_through=True, pathname_resolver=lambda x: f'{x.meta(Path).value}.gpg'),
        
            ProcessGen("gpg --batch --decrypt --passphrase 'secret'"),
            MetaGen(MD5Calculated),
            LocalGen(pass_through=True, pathname_resolver=lambda x: f'{x.meta(Path).value}.decrypted')
        )
        
        for f in workflow.compose(ByteFile(b'x' * 10, Path('x.orig')), ByteFile(b'y' * 20, Path('y.orig'))).flush_iter():
            print(f'\n{"-"*46}\n')
            print("Original path:", f.meta(Path, 2).value)
            print("Original md5:", f.meta(MD5Calculated, 2).value, end='\n\n')
        
            print("Encrypted path:", f.meta(Path, 1).value)
            print("Encrypted md5:", f.meta(MD5Calculated, 1).value, end='\n\n')
        
            print("Decrypted path:", f.meta(Path).value)
            print("Decrypted md5:", f.meta(MD5Calculated).value)
        
        ```
        
        See unittests for more examples
        
        ## Run tests and verify pypi compatibility 
        
        To run tests install tox and twine with pip, go to project root and run tox
        ```bash
        # python3 -m venv .venv
        # Activate virtualenv
        source .venv/bin/activate
        # Run tests
        tox -e py37
        # Build distribution
        python setup.py sdist bdist_wheel
        # Validate distribution
        twine check dist/*
        ```
        
        
        ## Built With
        
        * [Travis CI](https://travis-ci.org/)
        * [Codecov](https://codecov.io/)
        * [Tox](https://tox.readthedocs.io/)
        
        ## Contributing
        
        Bug-reports and pull requests on github  
        
        ## Versioning
        Any version change could break the public API (until 1.0.0 release)
         
        
        * [Pypi](https://pypi.org/project/fpipe/#history)
        * [Github](https://github.com/vkvam/fpipe/releases)
        
        ## License
            
        This project is licensed under the MIT License - see the [LICENSE.txt](https://github.com/vkvam/fpipe/blob/master/LICENSE.txt) file for details
        
Keywords: pipe file stream
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Intended Audience :: Developers
Description-Content-Type: text/markdown
