Metadata-Version: 1.0
Name: testharness
Version: 1.0.2
Summary: This is intended to be used as a generic functional test framework.
Home-page: UNKNOWN
Author: Oisin Mulvihill
Author-email: oisin mulvihill at gmail com
License: LGPL
Description: ===================
        Test Harness Readme
        ===================
        
        :author: Oisin Mulvihill
        
        .. contents::
        
        
        Introduction
        ============
        
        This is intended to be used as a generic functional test framework.
        
        This uses a simple script system, based on the python console, to load and run commands from test
        modules. The test modules are python classes that implement the details of how to test the project or
        program. The module provides a simple set of functions that can be called from a script. The functions
        can for example, take care of the complexity of starting some service or actually setting/getting some
        value in a database. In the script each function call can then be asserted to test what's returned. This
        the is used to pass or fail a test.
        
        I've used this project to test interactions between network services, database driven applications, GUI
        testing via pyGTK or in many other situations. I use it mainly to implement user acceptance stories to
        verify the behavior of a some user functionality. I don't use it to test web pages as there are better
        things out there for this (I use selenium remote control).
        
        I originally developed this project back in 2001 and I've used it to test a few commercial products, that
        my company and others have developed. I've got several versions of this project in various places. I've
        its time to bring them all together into one release. This is also an exercise for to learn how to write
        packages using setuptools.
        
        
        Running the test harness
        ========================
        
        Currently the test harness is run out of the pytester directory. The following
        instructions apply only to this situation.
        
        * Copy the file ./lib/testharness/config/tester.ini.in to ./tester.ini
        
        * Copy the file ./lib/testharness/testharness_log.ini.in to ./testharness_log.ini
        
        * Edit the tester.ini if needed.
        
        * Edit the log.ini if needed.
        
        * From the command line issues the following to run the test harness. All the tests mentioned in the tester.ini will now be run one after another.::
        
        PYTHONPATH=lib python ./lib/testharness/scripts/runacceptance
        
        Or
        
        goharness
        
        
        
        Command line arguments
        ======================
        
        The test harness takes a number of command line arguments.
        
        -c, --config <config_file>
        --------------------------
        
        This specifies the configuration file the test harness is going to use. This
        file is required and if it is not specified, then the file "tester.ini" is
        looked for in the current path.
        
        
        -l, --logconfig <log_config>
        ----------------------------
        
        This specifies the logging configuration use. This file set up where log
        messages go, etc. By default the test harness looks for a file called
        "log.ini" in the current path.
        
        
        -i, --interactive
        -----------------
        
        This tells the test harness not to start running tests mentioned in the
        tester ini file, but instead to go into interaction mode. This allows the
        user to type the commands in manually.
        
        --testpath <path_to_tests>
        --------------------------
        
        This tells the test harness where to find all the tests and test modules it
        is going to use. If this is not specified the the testharness will look for
        the "acceptancetests" directory in the current path.
        
        
        tester.ini configuration
        -------------------------
        
        This file is used to configure the test harness and any modules that are
        loaded. The TestHarness section contains, as you might expect, settings
        used by the test harness. The following is a typical tester.ini::
        
        [TestHarness]
        assert_equals_failure_causes_exit = yes
        tests = my_default_tests_to_run.script
        
        
        TestHarness section
        -------------------
        
        assert_equals_failure_causes_exit
        +++++++++++++++++++++++++++++++++
        
        This tells the test harness to stop processing a test script if an assertEquals
        fails in a test. If this was "no" the test harness would try to continue
        processing the script. A "yes" value here is the default, "no" is only used in
        debugging and is not recomended in normal situations.
        
        tests
        +++++
        
        This is a list of scripts the test harness will run when it starts. These are
        file names seperated by spaces and all on the same line. Currently you cannot
        span multiple lines. Note also that the test harness expects to find these
        files in the --testpath <path_to_tests> directory.
        
        
        
        Test scripts
        ============
        
        All tests usually start with \*.script. Thats just a convention I started using.
        The files can be called whatever you want.
        
        
        Test script and test harness console commands
        ---------------------------------------------
        
        The test harness has a number of built in functions that can be called. These
        functions can be called from a script file or from the console. There is no real
        distinction.
        
        
        load <module name>
        ------------------
        
        This loads a test module into the test harness. Test harness modules are python
        script files that contain classes derived from TestBase. When you specify the
        module you don't specify '.py' extension.
        
        *Note*
        
        Only class derived from TestBase will be loaded everything else is ignored.
        
        
        help [<function name>]
        ----------------------
        
        This function gives help on a command name or lists all functions in the
        TestHarness.
        
        
        exit
        ----
        
        This tells the test harness to exit after call shutdown.
        
        
        rem
        ----
        
        This is only really useful inside a script file. Any line starting with this is
        ignored.
        
        
        print <string to be printed>
        ----------------------------
        
        This is only really useful inside a script file. This function prints a line of
        text to stdout.
        
        
        shutdown
        --------
        
        This function is usually called just before the test harness exits. This
        function goes through each loaded module and calls its start() method.
        
        
        start <module name>
        -------------------
        
        This function call a loaded modules start() method.
        
        
        stop <module name>
        -------------------
        
        This function call a loaded modules start() method.
        
        list
        ----
        
        This function lists all loaded modules in the test harness.
        
        
        inspect <module name>
        ---------------------
        
        This function calls a modules inspect() method to print out a list of available
        function calls.
        
        Note: It is up to module writer to actually write the information that is
        returned for printing.
        
        
        evaluate <module name> <method call> <arg 1>,<arg 2>,<arg 3>,...<arg X>
        -----------------------------------------------------------------------
        
        This function calls a method in a module. Normally this function isn't called by
        the user directly. Instead anything which isn't a built in function is assumed
        to be a module method call.
        
        For example:
        
        A module called 'mytest' has one class called 'MainTest'. The 'MainTest'
        function has a method called 'doMyTest1' and it takes two arguments, a number
        and a string.
        
        This could be invoked using the evaluate function as follows::
        
        evaluate mytest.MainTest doMyTest1 123,this is a string
        
        However the following also works::
        
        mytest.MainTest doMyTest1 123,this is a string
        
        *Notes*
        
        Arguments are seperated by commas and these cannot be used by the user in
        the script.
        
        Arguemts should have no white space before or after them, unless you want
        this as it will be passed to the function.
        
        For example::
        
        doMyTest1 123,this is a string,4321
        doMyTest2 123, this is a string  ,4321 ,
        
        doMyTest1 will passed the args "123" "this is a string" and "4321"
        doMyTest1 will passed the args "123" " this is a string  " and "4321 "
        
        
        assertEquals <error message if fails>,<compare value>
        ------------------------------------------------------
        
        This function compares the provided <compare value> string, with that returned
        by the last user module script call.
        
        For example::
        
        mytest.MainTest doMyTest1 123,this is a string
        
        This call returns a number. This function is converted to a string and stored by
        the test harness. Now the assertEquals can be used to test the return as follows::
        
        assertEquals doMyTest1 didn't work,12
        
        If the returned value from doMyTest1 was 12 then assertEquals will pass. If
        doMyTest1 didn't return 12 then the message 'doMyTest1 didn't work' will be
        printed.
        
        *Note*
        
        If the test harness assert config option is 'yes' the a failing
        assertEquals call will cause the test harness to stop running the script and
        return with a failure code.
        
        
        
        
        
Platform: UNKNOWN
