Metadata-Version: 1.1
Name: pytest-spark
Version: 0.4.1
Summary: pytest plugin to run the tests with support of pyspark.
Home-page: https://github.com/malexer/pytest-spark
Author: Alex (Oleksii) Markov
Author-email: alex@markovs.me
License: MIT
Description: pytest-spark
        ############
        
        pytest_ plugin to run the tests with support of pyspark (`Apache Spark`_).
        
        This plugin will allow to specify SPARK_HOME directory in ``pytest.ini``
        and thus to make "pyspark" importable in your tests which are executed
        by pytest.
        
        Also it defines session scope fixture ``spark_context`` which can be
        used in your tests.
        
        
        Install
        =======
        
        .. code-block:: shell
        
            $ pip install pytest-spark
        
        
        Usage
        =====
        
        Set Spark location
        ------------------
        
        To run tests with required spark_home location just add "spark_home"
        value to ``pytest.ini`` in your project directory::
        
            [pytest]
            spark_home = /opt/spark
        
        Or set the "SPARK_HOME" environment variable.
        
        pytest-spark will try to import ``pyspark`` from specified location.
        
        Order of reading the "spark_home":
        
        1. pytest.ini
        2. "SPARK_HOME" environment variable
        3. Try to find it in common locations (i.e. OS X Homebrew)
        
        
        Using the ``spark_context`` fixture
        -----------------------------------
        
        Use fixture ``spark_context`` in your tests as a regular pyspark fixture.
        SparkContext instance will be created once and reused for the whole test
        session.
        
        Example::
        
            def test_my_case(spark_context):
                test_rdd = spark_context.parallelize([1, 2, 3, 4])
                # ...
        
        
        Using the ``spark_session`` fixture (Spark 2.0 and above)
        ---------------------------------------------------------
        
        Use fixture ``spark_session`` in your tests as a regular pyspark fixture.
        A SparkSession instance with Hive support enabled will be created once and reused for the whole test
        session.
        
        Example::
        
            def test_spark_session_dataframe(spark_session):
                test_df = spark_session.createDataFrame([[1,3],[2,4]], "a: int, b: int")
                # ...
        
        .. _pytest: http://pytest.org/
        .. _Apache Spark: https://spark.apache.org/
        
Keywords: pytest spark pyspark unittest test
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.2
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Topic :: Software Development :: Testing
