Metadata-Version: 1.1
Name: hisser
Version: 0.10
Summary: Fast TSDB backend for graphite
Home-page: https://github.com/baverman/hisser
Author: Anton Bobrov
Author-email: baverman@gmail.com
License: MIT
Description: Hisser
        ======
        
        |travis| |coverage| |pyver|
        
        .. |travis| image:: https://travis-ci.org/baverman/hisser.svg?branch=master
           :target: https://travis-ci.org/baverman/hisser
        
        .. |coverage| image:: https://img.shields.io/badge/coverage-100%25-brightgreen.svg
        
        .. |pyver| image:: https://img.shields.io/badge/python-3.5%2C_3.6-blue.svg
        
        
        Time series database, backend for graphite, fast alternative to carbon + whisper.
        
        Features:
        
        * Low disk usage (IOPS) for metric store, it depends from actual data
          volumes instead of a number of metrics (in case of whisper). Hisser
          was designed to process million of metrics.
        
        * Fast queries. Optimized query parsing and response rendering (~3x
          boost comparing with vanilla graphite-web).
        
        * Tag support.
        
        * Drop-in replacement for whisper + carbon.
        
        * 100% test coverage.
        
        
        .. contents:: **Table of Contents**
        
        
        Configuration
        -------------
        
        Default options and documentation for them can be read in
        `default config`_.
        
        .. _default config: hisser/defaults.py
        
        You can create custom configuration file and use ``--config`` cli option or
        use ``HISSER_*`` environment variables to override default values.
        For example ``HISSER_DATA_DIR`` will set ``DATA_DIR`` configuration
        parameter.
        
        
        Run
        ---
        
        Simplest way is to use official `docker image`::
        
           docker run --rm -u $(id -u):$(id -g) -p 2003:2003 -p 8080:8080 -v /path/to/data:/data baverman/graphite-hisser
        
        Likely you will use ``hisser`` as storage for ``grafana`` so you don't need 8080 port
        accessible from external network. In this case you should use separate docker network
        and map 2003 port only or use ``--network host`` and specify ``GRAPHITE_BIND=127.0.0.1:8080``
        envvar.
        
        IMPORTANT! To use tag support with grafana you need grafana 5.x and set graphite
        version ``1.1.x`` in storage settings.
        
        Note: for grafana you can use tiny `grafana image`_.
        
        .. _docker image: https://hub.docker.com/r/baverman/graphite-hisser/
        .. _grafana image: https://hub.docker.com/r/baverman/grafana/
        
        
        Internals
        ---------
        
        Hisser is a very simple metric storage. All heavy work is done by `lmdb`_.
        Metrics are organized into blocks (lmdb databases). Each block
        contains all metrics and their data for particular amount of time. Blocks
        with same resolution are grouped under corresponding directory:
        
        Example data layout:
        
        ::
        
           data_dir/
           ├── 300  # resolution (1 data point every 5-minute)
           │   ├── 1533990300.519.hdb   # timestamp-of-block-start.number-of-points.hdb
           │   ├── 1534621800.191.hdb
           │   ├── 1534679100.48.hdb
           │   └── blocks.state         # lock file
           ├── 60   # resolution (1 data point every minute)
           │   ├── 1534621920.700.hdb
           │   ├── 1534663920.320.hdb
           │   ├── 1534683120.160.hdb
           │   ├── 1534692720.40.hdb
           │   ├── 1534695120.11.hdb
           │   ├── 1534695900.6.hdb
           │   └── blocks.state
           └── metric.index       # metric name and tag index
        
        
        This layout allows to dump data from memory buffer very efficiently (whisper
        needs one io-operation per metric and can kneel a host with several hundreds of
        metrics).
        
        If points in memory exceed ``BUFFER_FLUSH_SIZE`` or ``BUFFER_MAX_POINTS`` it will be
        flushed into separate block::
        
           +----------+----------+----------+
           |  block1  |  block2  |  block3  |  resolution 60
           +----------+----------+----------+
        
        From time to time small blocks are merged into greater one::
        
           +---------------------+----------+
           |       block12       |  block3  |  resolution 60
           +---------------------+----------+
        
        And from time to time big blocks are downsampled into blocks with lower
        resolution::
        
           +---------------------+----------+
           |       block12       |  block3  |  resolution 60
           +---------------------+----------+
                      |
                      v
                +----------+
                | block12' |  resolution 300
                +----------+
        
        Yes, it is very simple.
        
        .. _lmdb: http://www.lmdb.tech/doc/
        
        
        FAQ
        ---
        
        1. But there is a better alternative to whisper. InfluxDB!
        
           Yes, InfluxDB is a way better than whisper. But is has some drawbacks
           comparing to hisser.
        
           * Requires more data space.
           * Consumes more IOPS, memory and CPU.
           * Needs manual retention configuration.
           * Slower to query.
           * Implicit metric grouping can lead to confusing graphs in grafana.
             You have to limit groups to explicit tag values or do ``group by
             $tag``.
        
Platform: UNKNOWN
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
