Metadata-Version: 1.0
Name: hyperstream
Version: 0.3.7
Summary: Hyperstream is a large-scale, flexible and robust software package for processing streaming data
Home-page: https://irc-sphere.github.io/HyperStream/
Author: Tom Diethe; Meelis Kull; Niall Twomey; Kacper Sokol; Hao Song; Miquel Perello-Nieto; Emma Tonkin; Peter Flach
Author-email: hyperstreamhq@googlegroups.com
License: MIT
Description: .. figure:: https://cdn.rawgit.com/IRC-SPHERE/HyperStream/dfbac332/hyperstream.svg
           :alt: HyperStream logo
        
           HyperStream logo
        
        HyperStream
        ===========
        
        |DOI| |Join the chat at https://gitter.im/IRC-SPHERE-HyperStream/Lobby|
        |Build Status| |Dependency Status| |Test Coverage| |Issue Count|
        |Documentation Status|
        
        Hyperstream is a large-scale, flexible and robust software package for
        processing streaming data.
        
        -  HyperStream `homepage <https://irc-sphere.github.io/HyperStream/>`__
        -  Tutorial
           `notebooks <http://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/master/examples/>`__
        -  Gitter `chat room <https://gitter.im/IRC-SPHERE-HyperStream/Lobby>`__
        -  Developer
           `documentation <http://hyperstream.readthedocs.io/en/latest/>`__
        
        Hyperstream overcomes the limitations of other computational engines and
        provides high-level interfaces to execute complex nesting, fusion, and
        prediction both in online and offline forms in streaming environments.
        Although developed specifically for SPHERE, Hyperstream is a general
        purpose tool that is well-suited for the design, development, and
        deployment of algorithms and predictive models in a wide space of
        sequential predictive problems.
        
        This software has been designed from the outset to be
        domain-independent, in order to provide maximum value to the wider
        community. Key aspects of the software include the capability to create
        complex interlinked workflows, and a computational engine that is
        designed to be "compute-on-request", meaning that no unnecessary
        resources are used.
        
        Installation
        ============
        
        Docker images
        -------------
        
        If you do not want to install all the packages separately you can use
        our Docker bundle available
        `here <https://github.com/IRC-SPHERE/Hyperstream-Dockerfiles>`__.
        
        Local machine
        -------------
        
        Install via pip
        
        ::
        
            pip install hyperstream
            python -c 'from hyperstream import HyperStream'
        
        To get the latest version
        
        ::
        
            pip install -U git+git://github.com/IRC-SPHERE/HyperStream.git#egg=hyperstream
            pip install -r requirements.txt
        
        Or clone the repository
        
        .. code:: bash
        
            git clone git@github.com:IRC-SPHERE/HyperStream.git
            cd HyperStream
            virtualenv venv
            . venv/bin/activate
            pip install -r requirements.txt
            python -c 'from hyperstream import HyperStream'
        
        Additionally, one of the requirements to run Hyperstream is a MongoDB
        server. By default, Hyperstream tries to connect to the port 27017 on
        the localhost.
        
        To see the installation steps of MongoDB go to the `official
        documentation <https://docs.mongodb.com/manual/installation/>`__. E.g.
        in a Debian OS it is possible to install with the following command
        
        .. code:: bash
        
            sudo apt-get install mongodb
        
        Once the MongoDB server is installed, it can be started with the
        following command
        
        .. code:: bash
        
            service mongod start
        
        Running tests
        =============
        
        Run the following command
        
        ::
        
            nosetests
        
        Note that for the MQTT logging test to succeed, you will need to have an
        MQTT broker running (e.g. Mosquitto). For example:
        
        ::
        
            docker run -ti -p 1883:1883 -p 9001:9001 toke/mosquitto
        
        or on OSX you will need pidof and mosquitto:
        
        ::
        
            brew install pidof
            brew install mosquitto
            brew services start mosquitto
        
        Tutorials
        =========
        
        The following tutorials show how to use HyperStream in a step-by-step
        guide.
        
        -  `Tutorial 1:
           Introduction <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/master/examples/tutorial_01.ipynb>`__
        -  `Tutorial 2: Creating
           tools <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/master/examples/tutorial_02.ipynb>`__
        -  `Tutorial 3: Stream
           composition <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/master/examples/tutorial_03.ipynb>`__
        -  `Tutorial 4: Real-time
           streams <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/master/examples/tutorial_04.ipynb>`__
        -  `Tutorial 5:
           Workflows <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/master/examples/tutorial_05.ipynb>`__
        
        Running the tutorials in a docker container
        -------------------------------------------
        
        It is possible to run all the tutorials in your own machine ussing
        Docker containers defined in
        `IRC-SPHERE/Hyperstream-Dockerfiles <https://github.com/IRC-SPHERE/HyperStream-Dockerfiles>`__.
        You can do that by running the following commands:
        
        .. code:: bash
        
            git clone https://github.com/IRC-SPHERE/Hyperstream-Dockerfiles.git
            cd Hyperstream-Dockerfiles
            docker-compose -f docker-compose-tutorials.yml -p hyperstream-tutorials up
        
        And then open the url http://0.0.0.0:8888/tree in a web-browser
        
        Running the tutorials in the cloned folder
        ------------------------------------------
        
        To run the tutorials in the cloned repository you will need to install
        additional dependencies. First you should activate the virtual
        environment and installed the general requirements to run HyperStream
        following the instructions above. After that, install the dependencies
        for the tutorial with
        
        .. code:: bash
        
            pip install -r requirements_tutorial.txt
        
        and go to the experiments folder
        
        .. code:: bash
        
            cd experiments
        
        And run a Jupyter notebook
        
        .. code:: bash
        
            jupyter notebook
        
        Now you can follow the instructions from the first tutorial.
        
        Simple use-case
        ---------------
        
        .. code:: python
        
            from hyperstream import HyperStream, StreamId, TimeInterval
            from hyperstream.utils import utcnow, UTC
            from datetime import timedelta
        
            hs = HyperStream(loglevel=20)
            M = hs.channel_manager.memory
            T = hs.channel_manager.tools
            clock = StreamId(name="clock")
            clock_tool = T[clock].window().last().value()
            ticker = M.get_or_create_stream(stream_id=StreamId(name="ticker"))
            now = utcnow()
            before = (now - timedelta(seconds=30)).replace(tzinfo=UTC)
            ti = TimeInterval(before, now)
            clock_tool.execute(sources=[], sink=ticker, interval=ti, alignment_stream=None)
            list(ticker.window().tail(5))
        
        The last list contains
        
        .. code:: python
        
            [StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 45, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 45, tzinfo=<UTC>)),
             StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 46, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 46, tzinfo=<UTC>)),
             StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 47, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 47, tzinfo=<UTC>)),
             StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 48, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 48, tzinfo=<UTC>)),
             StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 49, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 49, tzinfo=<UTC>))]
        
        HyperStream Viewer
        ==================
        
        The `HyperStream
        Viewer <https://github.com/IRC-SPHERE/HyperStreamViewer>`__ is a
        python/Flask web-app for interacting with HyperStream. In order to keep
        HyperStream to a minimum, this web-app is released as a separate
        repository that takes the core as a dependency.
        
        License
        =======
        
        This code is released under the `MIT
        license <https://github.com/IRC-SPHERE/HyperStream/blob/master/LICENSE>`__.
        
        Acknowledgements
        ================
        
        This work has been funded by the UK Engineering and Physical Sciences
        Research Council (EPSRC) under Grant
        `EP/K031910/1 <http://gow.epsrc.ac.uk/NGBOViewGrant.aspx?GrantRef=EP/K031910/1>`__
        - "SPHERE Interdisciplinary Research Collaboration".
        
        .. |DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.242227.svg
           :target: https://doi.org/10.5281/zenodo.242227
        .. |Join the chat at https://gitter.im/IRC-SPHERE-HyperStream/Lobby| image:: https://badges.gitter.im/IRC-SPHERE-HyperStream/Lobby.svg
           :target: https://gitter.im/IRC-SPHERE-HyperStream/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
        .. |Build Status| image:: https://travis-ci.org/IRC-SPHERE/HyperStream.svg?branch=master
           :target: https://travis-ci.org/IRC-SPHERE/HyperStream
        .. |Dependency Status| image:: https://www.versioneye.com/user/projects/58e423cb26a5bb005220301e/badge.svg?style=flat-square
           :target: https://www.versioneye.com/user/projects/58e423cb26a5bb005220301e
        .. |Test Coverage| image:: https://codeclimate.com/github/IRC-SPHERE/HyperStream/badges/coverage.svg
           :target: https://codeclimate.com/github/IRC-SPHERE/HyperStream/coverage
        .. |Issue Count| image:: https://codeclimate.com/github/IRC-SPHERE/HyperStream/badges/issue_count.svg
           :target: https://codeclimate.com/github/IRC-SPHERE/HyperStream
        .. |Documentation Status| image:: https://readthedocs.org/projects/hyperstream/badge/?version=latest
           :target: http://hyperstream.readthedocs.io/en/latest/?badge=latest
        
Platform: UNKNOWN
