Metadata-Version: 1.0
Name: hyperstream
Version: 0.3.3
Summary: Hyperstream is a large-scale, flexible and robust software package for processing streaming data
Home-page: https://irc-sphere.github.io/HyperStream/
Author: Tom Diethe; Meelis Kull; Niall Twomey; Kacper Sokol; Hao Song; Miquel Perello-Nieto; Emma Tonkin; Peter Flach
Author-email: hyperstreamhq@googlegroups.com
License: MIT
Description: .. figure:: https://cdn.rawgit.com/IRC-SPHERE/HyperStream/dfbac332/hyperstream.svg
           :alt: HyperStream logo
        
           HyperStream logo
        
        HyperStream
        ===========
        
        |DOI| |Join the chat at https://gitter.im/IRC-SPHERE-HyperStream/Lobby|
        |Build Status| |Dependency Status| |Test Coverage| |Issue Count|
        |Documentation Status|
        
        Hyperstream is a large-scale, flexible and robust software package for
        processing streaming data.
        
        -  HyperStream `homepage <https://irc-sphere.github.io/HyperStream/>`__
        -  Tutorial
           `notebooks <http://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/tutorials/examples/>`__
        -  Gitter `chat room <https://gitter.im/IRC-SPHERE-HyperStream/Lobby>`__
        -  Developer
           `documentation <http://hyperstream.readthedocs.io/en/latest/>`__
        
        Hyperstream overcomes the limitations of other computational engines and
        provides high-level interfaces to execute complex nesting, fusion, and
        prediction both in online and offline forms in streaming environments.
        Although developed specifically for SPHERE, Hyperstream is a general
        purpose tool that is well-suited for the design, development, and
        deployment of algorithms and predictive models in a wide space of
        sequential predictive problems.
        
        This software has been designed from the outset to be
        domain-independent, in order to provide maximum value to the wider
        community. Key aspects of the software include the capability to create
        complex interlinked workflows, and a computational engine that is
        designed to be "compute-on-request", meaning that no unnecessary
        resources are used.
        
        Installation
        ============
        
        Docker images
        -------------
        
        If you do not want to install all the packages separately you can use
        our Docker bundle available
        `here <https://github.com/IRC-SPHERE/Hyperstream-Dockerfiles>`__.
        
        Local machine
        -------------
        
        Install via pip
        
        ::
        
            pip install hyperstream
            python -c 'from hyperstream import HyperStream'
        
        To get the latest version
        
        ::
        
            pip install -U git+git://github.com/IRC-SPHERE/HyperStream.git#egg=hyperstream
            pip install -r requirements.txt
        
        Or clone the repository
        
        .. code:: bash
        
            git clone git@github.com:IRC-SPHERE/HyperStream.git
            cd HyperStream
            virtualenv venv
            . venv/bin/activate
            pip install -r requirements.txt
            python -c 'from hyperstream import HyperStream'
        
        Additionally, one of the requirements to run Hyperstream is a MongoDB
        server. By default, Hyperstream tries to connect to the port 27017 on
        the localhost.
        
        To see the installation steps of MongoDB go to the `official
        documentation <https://docs.mongodb.com/manual/installation/>`__. E.g.
        in a Debian OS it is possible to install with the following command
        
        .. code:: bash
        
            sudo apt-get install mongodb
        
        Once the MongoDB server is installed, it can be started with the
        following command
        
        .. code:: bash
        
            service mongod start
        
        Running tests
        =============
        
        Run the following command
        
        ::
        
            nosetests
        
        Note that for the MQTT logging test to succeed, you will need to have an
        MQTT broker running (e.g. Mosquitto). For example:
        
        ::
        
            docker run -ti -p 1883:1883 -p 9001:9001 toke/mosquitto
        
        or on OSX you will need pidof and mosquitto:
        
        ::
        
            brew install pidof
            brew install mosquitto
            brew services start mosquitto
        
        Tutorials
        =========
        
        The following tutorials show how to use HyperStream in a step-by-step
        guide.
        
        -  `Tutorial 1:
           Introduction <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/tutorials/examples/tutorial_01.ipynb>`__
        -  `Tutorial 2: Creating
           tools <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/tutorials/examples/tutorial_02.ipynb>`__
        -  `Tutorial 3: Stream
           composition <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/tutorials/examples/tutorial_03.ipynb>`__
        -  `Tutorial 4: Real-time
           streams <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/tutorials/examples/tutorial_04.ipynb>`__
        -  `Tutorial 5:
           Workflows <https://nbviewer.jupyter.org/github/IRC-SPHERE/HyperStream/blob/tutorials/examples/tutorial_05.ipynb>`__
        
        .. code:: python
        
                from hyperstream import HyperStream, StreamId, TimeInterval
                from hyperstream.utils import utcnow, UTC
                from datetime import timedelta
        
                hs = HyperStream(loglevel=20)
                M = hs.channel_manager.memory
                T = hs.channel_manager.tools
                clock = StreamId(name="clock")
                clock_tool = T[clock].window().last().value()
                ticker = M.get_or_create_stream(stream_id=StreamId(name="ticker"))
                now = utcnow()
                before = (now - timedelta(seconds=30)).replace(tzinfo=UTC)
                ti = TimeInterval(before, now)
                clock_tool.execute(sources=[], sink=ticker, interval=ti, alignment_stream=None)
                print(list(ticker.window().tail(5)))
        
                [StreamInstance(timestamp=datetime.datetime(2017, 6, 30, 16, 23, 39, tzinfo=<UTC>), value=datetime.datetime(2017, 6, 30, 16, 23, 39, tzinfo=<UTC>)), StreamInstance(timestamp=datetime.datetime(2017, 6, 30, 16, 23, 40, tzinfo=<UTC>), value=datetime.datetime(2017, 6, 30, 16, 23, 40, tzinfo=<UTC>)), StreamInstance(timestamp=datetime.datetime(2017, 6, 30, 16, 23, 41, tzinfo=<UTC>), value=datetime.datetime(2017, 6, 30, 16, 23, 41, tzinfo=<UTC>)), StreamInstance(timestamp=datetime.datetime(2017, 6, 30, 16, 23, 42, tzinfo=<UTC>), value=datetime.datetime(2017, 6, 30, 16, 23, 42, tzinfo=<UTC>)), StreamInstance(timestamp=datetime.datetime(2017, 6, 30, 16, 23, 43, tzinfo=<UTC>), value=datetime.datetime(2017, 6, 30, 16, 23, 43, tzinfo=<UTC>))]
        
        HyperStream Viewer
        ==================
        
        The `HyperStream
        Viewer <https://github.com/IRC-SPHERE/HyperStreamViewer>`__ is a
        python/Flask web-app for interacting with HyperStream. In order to keep
        HyperStream to a minimum, this web-app is released as a separate
        repository that takes the core as a dependency.
        
        License
        =======
        
        This code is released under the `MIT
        license <https://github.com/IRC-SPHERE/HyperStream/blob/master/LICENSE>`__.
        
        Acknowledgements
        ================
        
        This work has been funded by the UK Engineering and Physical Sciences
        Research Council (EPSRC) under Grant
        `EP/K031910/1 <http://gow.epsrc.ac.uk/NGBOViewGrant.aspx?GrantRef=EP/K031910/1>`__
        - "SPHERE Interdisciplinary Research Collaboration".
        
        .. |DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.242227.svg
           :target: https://doi.org/10.5281/zenodo.242227
        .. |Join the chat at https://gitter.im/IRC-SPHERE-HyperStream/Lobby| image:: https://badges.gitter.im/IRC-SPHERE-HyperStream/Lobby.svg
           :target: https://gitter.im/IRC-SPHERE-HyperStream/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
        .. |Build Status| image:: https://travis-ci.org/IRC-SPHERE/HyperStream.svg?branch=master
           :target: https://travis-ci.org/IRC-SPHERE/HyperStream
        .. |Dependency Status| image:: https://www.versioneye.com/user/projects/58e423cb26a5bb005220301e/badge.svg?style=flat-square
           :target: https://www.versioneye.com/user/projects/58e423cb26a5bb005220301e
        .. |Test Coverage| image:: https://codeclimate.com/github/IRC-SPHERE/HyperStream/badges/coverage.svg
           :target: https://codeclimate.com/github/IRC-SPHERE/HyperStream/coverage
        .. |Issue Count| image:: https://codeclimate.com/github/IRC-SPHERE/HyperStream/badges/issue_count.svg
           :target: https://codeclimate.com/github/IRC-SPHERE/HyperStream
        .. |Documentation Status| image:: https://readthedocs.org/projects/hyperstream/badge/?version=latest
           :target: http://hyperstream.readthedocs.io/en/latest/?badge=latest
        
Platform: UNKNOWN
