Metadata-Version: 2.0
Name: esrally
Version: 0.4.0
Summary: Macrobenchmarking framework for Elasticsearch
Home-page: https://github.com/elastic/rally
Author: Daniel Mitterdorfer
Author-email: daniel.mitterdorfer@gmail.com
License: Apache License, Version 2.0
Platform: UNKNOWN
Classifier: Topic :: System :: Benchmark
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Intended Audience :: Developers
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Requires-Dist: Jinja2 (==2.8)
Requires-Dist: certifi
Requires-Dist: elasticsearch (==2.3.0)
Requires-Dist: jsonschema (==2.5.1)
Requires-Dist: psutil (==4.1.0)
Requires-Dist: py-cpuinfo (==0.2.3)
Requires-Dist: tabulate (==0.7.5)
Requires-Dist: thespian (==3.0.4)

Rally
=====

Rally is the macrobenchmarking framework for Elasticsearch

What is Rally?
--------------

So you want to benchmark Elasticsearch? Then Rally is for you. Rally started as an effort to help developers in the Elasticsearch development team to run benchmarks on their machines. As our users are very creative and use Elasticsearch for all kinds of things, we have to cover a broad range of different performance characteristics and to find out how Elasticsearch performs under various conditions we run different benchmarks.

Rally is build around a few assumptions:

* Everything is run on the same machine (but `we are about to change that <https://github.com/elastic/rally/issues/71>`_)
* You want to add a specific data set to an Elasticsearch index and then run benchmarking queries on it

We are continuously working to remove these restrictions. In contrast to other home-grown benchmarking scripts, we have put considerable effort in Rally to ensure the benchmarking data are reproducible.

First Time Setup
----------------

Prerequisites
~~~~~~~~~~~~~

Please ensure that the following packages are installed before installing Rally:

* Python 3.4+ available as `python3` on the path (verify with: ``python3 --version`` which should print ``Python 3.4.0`` (or higher))
* ``pip3`` available on the path (verify with ``pip3 --version``)
* JDK 8+
* git

Rally does not support Windows and is only actively tested on Mac OS X and Linux.

Installing Rally
~~~~~~~~~~~~~~~~

**Note**: If you actively develop on Elasticsearch, we recommend that you `install Rally in development mode <http://esrally.readthedocs.io/en/latest/developing.html#installation-instructions-for-development>`_ instead as Elasticsearch is fast moving and Rally always adapts accordingly to the latest master version.

Simply install Rally with pip: ``pip3 install esrally``

**Note**: Depending on your system setup you may need to prepend this command with ``sudo``.

If you get errors during installation, it is probably due to the installation of ``psutil`` which we use to gather system metrics like CPU utilization. Please check the `installation instructions of psutil <https://github.com/giampaolo/psutil/blob/master/INSTALL.rst>`_ in this case. Keep in mind that Rally is based on Python 3 and you need to install the Python 3 header files instead of the Python 2 header files on Linux.

Non-sudo Install
~~~~~~~~~~~~~~~~

If you don't want to use ``sudo`` when installing Rally, installation is still possible but a little more involved:

1. Specify the ``--user`` option when installing Rally (step 2 above), so the command to be issued is: ``python3 setup.py develop --user``.
2. Check the output of the install script or lookup the `Python documentation on the variable site.USER_BASE <https://docs.python.org/3.5/library/site.html#site.USER_BASE>`_ to find out where the script is located. On Linux, this is typically ``~/.local/bin``.

You can now either add ``~/.local/bin`` to your path or invoke Rally via ``~/.local/bin/esrally`` instead of just ``esrally``.

Configuring Rally
-----------------

Before we can run our first benchmark, we have to configure Rally. Just invoke ``esrally configure`` and Rally will automatically detect that its configuration file is missing and prompt you for some values and write them to `~/.rally/rally.ini`. After you've configured Rally, it will exit.

For more information see :doc:`configuration help page </configuration>`.

Running the first benchmark
---------------------------

Now we are ready to run the first benchmark with Rally: Just invoke ``esrally``. This will start Rally with sensible defaults. It will download the necessary benchmark data, checkout the latest version of Elasticsearch, build it and finally run the benchmark.

**Note**: If you want to benchmark a binary distribution instead of a source distribution then run ``esrally --pipeline=from-distribution --distribution-version=VERSION_NUMBER`` (``VERSION_NUMBER`` is for example ``5.0.0-alpha1`` and is identical to the version number used in the download URL)

When the benchmark is done, a summary report is written to the command line:::

   ------------------------------------------------------
       _______             __   _____
      / ____(_)___  ____ _/ /  / ___/_________  ________
     / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
    / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
   /_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
   ------------------------------------------------------
                                                     Metric      Value
   --------------------------------------------------------  ---------
                           Min Indexing Throughput [docs/s]      19501
                        Median Indexing Throughput [docs/s]      20232
                           Max Indexing Throughput [docs/s]      21172
                                        Indexing time [min]    55.7989
                                           Merge time [min]    12.9766
                                         Refresh time [min]    5.20067
                                           Flush time [min]  0.0648667
                                  Merge throttle time [min]   0.796417
               Query latency default (50.0 percentile) [ms]    5.89058
               Query latency default (90.0 percentile) [ms]    6.71282
              Query latency default (100.0 percentile) [ms]    7.65307
    Query latency country_agg_cached (50.0 percentile) [ms]    1.70223
    Query latency country_agg_cached (90.0 percentile) [ms]    2.34819
   Query latency country_agg_cached (100.0 percentile) [ms]    3.42547
                Query latency scroll (50.0 percentile) [ms]    16.1226
                Query latency scroll (90.0 percentile) [ms]    17.2383
               Query latency scroll (100.0 percentile) [ms]    18.8419
                  Query latency term (50.0 percentile) [ms]    2.10049
                  Query latency term (90.0 percentile) [ms]    2.77537
                 Query latency term (100.0 percentile) [ms]    4.52081
                Query latency phrase (50.0 percentile) [ms]    1.82687
                Query latency phrase (90.0 percentile) [ms]    2.63714
               Query latency phrase (100.0 percentile) [ms]    5.39892
           Query latency country_agg (50.0 percentile) [ms]    112.049
           Query latency country_agg (90.0 percentile) [ms]    128.426
          Query latency country_agg (100.0 percentile) [ms]    155.989
                               Median CPU usage (index) [%]    668.025
                               Median CPU usage (stats) [%]     143.75
                              Median CPU usage (search) [%]      223.1
                                     Total Young Gen GC [s]     39.447
                                       Total Old Gen GC [s]      7.108
                                            Index size [GB]    3.25475
                                       Totally written [GB]    17.8434
                                Heap used for segments [MB]    21.7504
                              Heap used for doc values [MB]    0.16436
                                   Heap used for terms [MB]    20.0293
                                   Heap used for norms [MB]   0.105469
                                  Heap used for points [MB]   0.773487
                                  Heap used for points [MB]   0.677795
                                              Segment count        136
                        Indices Stats(90.0 percentile) [ms]    3.16053
                        Indices Stats(99.0 percentile) [ms]    5.29526
                       Indices Stats(100.0 percentile) [ms]    5.64971
                          Nodes Stats(90.0 percentile) [ms]    3.19611
                          Nodes Stats(99.0 percentile) [ms]    4.44111
                         Nodes Stats(100.0 percentile) [ms]    5.22527

Before relying too much on the numbers, please double-check that you did not introduce any bottlenecks and that your hardware is sufficient (e.g. spinning disks are not a good idea, better use SSDs). For additional insights and metrics you can activate different telemetry devices in Rally.

Also be very careful and get a deep understanding of the measurement approaches when comparing performance numbers on different OS. Sometimes certain measurements are supported only on one OS but not on another (e.g. disk I/O statistics) and different OS handle I/O differently.

Other command line flags
------------------------

Rally supports more command line flags, just run ``esrally --help`` to see what's possible. For more details, please refer to `Rally's user guide <https://esrally.readthedocs.io/>`_.

Getting help
------------

Please ask questions about Rally in the `Rally Discuss forum <https://discuss.elastic.co/c/rally>`_.

How to Contribute
-----------------

See all details in the `contributor guidelines <https://github.com/elastic/rally/blob/master/CONTRIBUTING.md>`_.

License
-------

This software is licensed under the Apache License, version 2 ("ALv2"), quoted below.

Copyright 2015-2016 Elasticsearch <https://www.elastic.co>

Licensed under the Apache License, Version 2.0 (the "License"); you may not
use this file except in compliance with the License. You may obtain a copy of
the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations under
the License.

