Metadata-Version: 1.0
Name: httpimport
Version: 0.5.11
Summary: Module for remote in-memory Python package/module loading through HTTP
Home-page: https://github.com/operatorequals/httpimport
Author: John Torakis - operatorequals
Author-email: john.torakis@gmail.com
License: Apache2
Description: ``httpimport``
        ==============
        
        Module for *remote*, *in-memory* Python *package/module* ``import``\ ing
        **through HTTP/S**
        
        |PyPI version|
        
        A feature that *Python2/3* **misses** and has become popular in other
        languages is the **remote loading of packages/modules**.
        
        ``httpimport`` lets a *Python2/3* packages/modules to be imported
        directly in Python interpreter's process memory, through **remote
        ``URIs``**, and *more*...
        
        Example - In a Nutshell
        ~~~~~~~~~~~~~~~~~~~~~~~
        
        .. code:: python
        
            >>> import httpimport
            >>> httpimport.__all__
            ['HttpImporter', 'add_remote_repo', 'remove_remote_repo', 'remote_repo', 'github_repo', 'bitbucket_repo']
        
        .. code:: python
        
            >>> with httpimport.remote_repo(['package1','package2','package3'], 'http://my-codes.example.com/python_packages'):
            ...     import package1
            ...
        
        .. code:: python
        
            >>> with httpimport.github_repo('operatorequals', 'covertutils', branch = master):
            ...     import covertutils
            ... # Also works with 'bitbucket_repo'
        
        .. code:: python
        
            >>> # A depends to B and B depends to C (A, B, C : Python modules/packages in different domains):
            >>> # A exists in "repo_a.my-codes.example.com" |
            >>> # B exists in "repo_b.my-codes.example.com" | <-- Different domains
            >>> # C exists in "repo_c.my-codes.example.com" |
            >>> with httpimport.remote_repo(['C'], 'http://repo_c.my-codes.example.com/python_packages'):
            ...  with httpimport.remote_repo(['B'], 'http://repo_b.my-codes.example.com/python_packages'):
            ...     with httpimport.remote_repo(['A'], 'http://repo_a.my-codes.example.com/python_packages'):
            ...     import A
            ... # Asks for A, Searches for B, Asks for B, Searches for C, Asks for C --> Resolves --> Imports A
            >>>
        
        .. code:: python
        
            >>> module_object = httpimport.load('package1', 'http://my-codes.example.com/python_packages')
            >>> module_object
            <module 'package1' from 'http://my-codes.example.com/python_packages/package1/__init__.py'>
        
        Example - The Whole Picture
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~
        
        Using the ``SimpleHTTPServer``, a whole directory can be served through
        HTTP as follows:
        
        .. code:: bash
        
            user@hostname:/tmp/test_directory$ ls -R
            .:
            test_package
        
            ./test_package:
            __init__.py  __init__.pyc  module1.py  module2.py
            user@hostname:/tmp/test_directory$
            user@hostname:/tmp/test_directory$ python -m SimpleHTTPServer &
            [1] 9565
            Serving HTTP on 0.0.0.0 port 8000 ...
        
            user@hostname:/tmp/test_directory$
            user@hostname:/tmp/test_directory$
            user@hostname:/tmp/test_directory$ curl http://localhost:8000/test_package/module1.py
            127.0.0.1 - - [22/Aug/2017 17:42:49] "GET /test_package/module1.py HTTP/1.1" 200 -
        
        
            def dummy_func() : return 'Function Loaded'
        
        
            class dummy_class :
        
                def dummy_method(self) : return 'Class and method loaded'
        
        
            dummy_str = 'Constant Loaded'
        
            user@hostname:/tmp/test_directory$
            user@hostname:/tmp/test_directory$ curl http://localhost:8000/test_package/__init__.py
            127.0.0.1 - - [22/Aug/2017 17:45:20] "GET /test_package/__init__.py HTTP/1.1" 200 -
            __all__ = ["module1", "module2"]
        
        Using this simple built-in feature of ``Py2/3``, a custom importer can
        been created, that given a base URL and a list of package names, it
        fetches and automatically loads all modules and packages to the local
        namespace.
        
        Usage
        ~~~~~
        
        Making the HTTP repo
        ^^^^^^^^^^^^^^^^^^^^
        
        .. code:: bash
        
            user@hostname:/tmp/test_directory$ ls -R
            .:
            test_package
        
            ./test_package:
            __init__.py  __init__.pyc  module1.py  module2.py
        
            user@hostname:/tmp/test_directory$
            user@hostname:/tmp/test_directory$ python -m SimpleHTTPServer
            Serving HTTP on 0.0.0.0 port 8000 ...
        
        Importing Remotely
        ~~~~~~~~~~~~~~~~~~
        
        ``add_remote_repo()`` and ``remove_remote_repo()``
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        
        These 2 functions will *add* and *remove* to the default
        ``sys.meta_path`` custom ``HttpImporter`` objects, given the URL they
        will look for packages/modules and a list of packages/modules its one
        can serve.
        
        .. code:: python
        
            >>> import test_package
            Traceback (most recent call last):
              File "<stdin>", line 1, in <module>
            ImportError: No module named test_package
            >>>
            >>> from httpimport import add_remote_repo, remove_remote_repo
            >>> # In the given URL the 'test_package/' is available
            >>> add_remote_repo(['test_package'], 'http://localhost:8000/') #  
            >>> import test_package
            >>>
            >>> remove_remote_repo('http://localhost:8000/')
            >>> import test_package.module1
            Traceback (most recent call last):
              File "<stdin>", line 1, in <module>
            ImportError: No module named module1
        
        The ``remote_repo()`` context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        
        *Adding* and *removing* Remote Repos can be a pain, *specially* if there
        are packages that are available in **more than one** repos. So the
        ``with`` keyword does the trick again:
        
        .. code:: python
        
            >>> from httpimport import remote_repo
            >>>
            >>>
            >>> with remote_repo(['test_package'], 'http://localhost:8000/') :
            ...     from test_package import module1
            ...
            >>>
            >>> from test_package import module2
            Traceback (most recent call last):
              File "<stdin>", line 1, in <module>
            ImportError: cannot import name module2
        
            >>> module1.dummy_str
            'Constant Loaded'
            >>> module1.dummy_func
            <function dummy_func at 0x7f7a8a170410>
        
        The Tiny Test for your amusement
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        
        The ``test.py`` file contains a minimal test. Try changing working
        directories and package names and see what happens...
        
        .. code:: bash
        
            $ python test.py
            serving at port 8000
            127.0.0.1 - - [22/Aug/2017 17:36:44] code 404, message File not found
            127.0.0.1 - - [22/Aug/2017 17:36:44] "GET /test_package/module1/__init__.py HTTP/1.1" 404 -
            127.0.0.1 - - [22/Aug/2017 17:36:44] "GET /test_package/module1.py HTTP/1.1" 200 -
            Constant Loaded
            Function Loaded
            Class and method loaded
        
        The *Github* Use Case!
        ----------------------
        
        Such HTTP Servers (serving Python packages in a *directory structured
        way*) can be found in the wild, not only created with
        ``SimpleHTTPServer``. **Github repos can serve as Python HTTPS Repos as
        well!!!**
        
        Here is an example with my beloved ``covertutils`` project:
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        
        .. code:: python
        
            >>>
            >>> import covertutils
            Traceback (most recent call last):
              File "<stdin>", line 1, in <module>
            ImportError: No module named covertutils
            >>> # covertutils is not available through normal import!
            >>>
            >>> covertutils_url = 'https://raw.githubusercontent.com/operatorequals/covertutils/master/'
            >>>
            >>> from httpimport import remote_repo
            >>>
            >>> with remote_repo(['covertutils'], covertutils_url) :
            ...     import covertutils
            ...
            >>> print covertutils.__author__
            John Torakis - operatorequals
        
        The **dedicated** ``github_repo()`` context:
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        
        .. code:: python
        
            >>> from httpimport import github_repo
            >>> with github_repo( 'operatorequals', 'covertutils', ) :
            ...     import covertutils
            ...
            >>> covertutils.__author__
            'John Torakis - operatorequals'
            >>>
        
        What about branches?
        ^^^^^^^^^^^^^^^^^^^^
        
        .. code:: python
        
            >>> from httpimport import github_repo
            >>> with github_repo( 'operatorequals', 'covertutils', branch='py3_compatibility' ) :
            ...     import covertutils
            ...
            >>> covertutils.__author__
            'John Torakis - operatorequals'
            >>>
        
        And ad-hoc commits too?
        ^^^^^^^^^^^^^^^^^^^^^^^
        
        What if you need to stick to a fixed -*known to work*- commit?
        
        .. code:: python
        
            >>> from httpimport import github_repo
            >>> with github_repo( 'operatorequals', 'covertutils', commit='cf3f78c77c437edf2c291bd5b4ed27e0a93e6a77' ) :
            ...     import covertutils
            ...
            >>> covertutils.__author__
            'John Torakis - operatorequals'
            >>>
        
        The newer sibling ``bitbucket_repo()`` (as of ``0.5.9``)
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        
        .. code:: python
        
            >>> with bitbucket_repo('atlassian', 'python-bitbucket', module='pybitbucket'):
            ...     import pybitbucket
            ...
            >>>
        
        Recursive Dependencies
        ----------------------
        
        If package ``A`` requires module ``B`` and ``A`` exists in
        ``http://example.com/a_repo/``, while ``B`` exists in
        ``http://example.com/b_repo/``, then ``A`` can be imported using the
        following technique:
        
        .. code:: python
        
            >>> from httpimport import remote_repo
            >>> with remote_repo(['B'],"http://example.com/b_repo/") :
            ...     with remote_repo(['A'],"http://example.com/a_repo/") :
            ...             import A
            ... 
            [!] 'B' not found in HTTP repository. Moving to next Finder.
            >>> 
            >>> A
            <module 'A' from 'http://example.com/a_repo/A/__init__.py'>
            >>> B
            <module 'B' from 'http://example.com/a_repo/B.py'>
            >>> 
        
        Any combination of *packages* and *modules* can be imported this way!
        
        *The ``[!]`` Warning was emitted by the ``HttpImporter`` object created
        for ``A``, as it couldn't locate ``B``, and passed control to the next
        ``Finder`` object, that happened to be the ``HttpImporter`` object
        created for ``B``!*
        
        The ``load()`` function (as of ``0.5.10``)
        ------------------------------------------
        
        The ``load()`` function was added to make module loading possible
        without ``Namespace`` pollution.
        
        .. code:: python
        
            >>> import httpimport
            >>> pack1 = httpimport.load('random-package','http://localhost:8000/')
            >>> pack1
            <module 'random-package' from 'http://localhost:8000//random-package/__init__.py'>
            >>>
            >>> # Trying to load 'os' module from the URL will fail, as it won't delegate to to other Finders/Loaders.
            >>> httpimport.load('os','http://localhost:8000/')
            [!] 'non-existent-package' not found in HTTP repository. Moving to next Finder.
            Traceback (most recent call last):
              File "<stdin>", line 1, in <module>
              File "httpimport.py", line 287, in load
                raise ImportError("Module '%s' cannot be imported from '%s'" % (module_name, url) )
            ImportError: Module 'os' cannot be imported from 'http://localhost:8000/'
        
        And no data touches the disk, nor any virtual environment. The import happens just to the running Python process!
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        
        Life suddenly got simpler for Python module testing!!!
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        
        Imagine the breeze of testing *Pull Requests* and packages that you
        aren't sure they will work for you!
        
        Debugging...
        ------------
        
        .. code:: python
        
            >>> from httpimport import *
            >>>
            >>> import logging
            >>> logging.getLogger('httpimport').setLevel(logging.DEBUG)
            >>>
            >>> with github_repo('operatorequals','covertutils') :
            ...     import covertutils
            ...
            FINDER=================
            [!] Searching covertutils
            [!] Path is None
            [@]Checking if in domain >
            [@]Checking if built-in >
            [@]Checking if it is name repetition >
            [*]Module/Package 'covertutils' can be loaded!
            LOADER=================
            [+] Loading covertutils
            [+] Trying to import as package from: 'https://raw.githubusercontent.com/operatorequals/covertutils/master//covertutils/__init__.py'
            [+] Importing 'covertutils'
            [+] Ready to execute 'covertutils' code
            [+] 'covertutils' imported succesfully!
            >>>
        
        Beware: **Huge Security Implications!**
        ---------------------------------------
        
        *Using the ``httpimport`` with **HTTP URLs** is highly discouraged
        outside the ``localhost`` interface!*
        
        As HTTP traffic isn't encrypted and/or integrity checked (*unlike
        HTTPS*), it is trivial for a remote attacker to intercept the HTTP
        responses (via an *ARP MiTM* probably), and add arbitrary *Python* code
        to the downloaded *packages/modules*. This will directly result in
        *Remote Code Execution* to your current user's context! In other words,
        you get **totally F\*ed**... #### You have been warned! Use **HTTPS
        URLs** with ``httpimport``!
        
        Did I hear you say "Staging protocol for `covertutils <https://github.com/operatorequals/covertutils>`__ backdoors"?
        ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
        
        Technique documentation on `using ``httpimport`` to stage
        ``covertutils`` backdoor
        code <http://covertutils.readthedocs.io/en/latest/staging_exec.html>`__,
        making *EXE packed* and *unreadable* code load *non-included module
        dependencies*.
        
        .. |PyPI version| image:: https://badge.fury.io/py/httpimport.svg
           :target: https://pypi.python.org/pypi/httpimport
        
Keywords: import,memory,http
Platform: UNKNOWN
