Package csb :: Package test |
[frames] | no frames]

# Source Code for Package csb.test

      1  """
      2  This is a top level package, hosting the entire CSB test framework. It is divided
      3  into several major parts:
      4
      5      - test cases, located under csb.test.cases
      6      - test data, in C{/csb/test/data} (not a package)
      7      - test console, in C{/csb/test/app.py}
      8
      9  This module, csb.test, contains all the glue-code functions, classes and
     10  decorators you would need in order to write tests for CSB.
     11
     12      1. Configuration and Tree
     13
     14         L{Config<csb.test.Config>} is a common config object shared between CSB
     15         tests. Each config instance contains properties like:
     16
     17              - data: the data folder, automatically discovered and loaded in
     18                csb.test.Config.DATA at module import time
     19              - temp: a default temp folder, which test cases can use
     20
     21         Each L{Config<csb.test.Config>} provides a convenient way to retrieve
     22         files from C{/csb/test/data}. Be sure to check out L{Config.getTestFile}
     23         and L{Config.getPickle}. In case you need a temp file, use
     24         L{Config.getTempStream} or have a look at L{csb.io.TempFile} and
     25         L{csb.io.TempFolder}.
     26
     27         All test data files should be placed in the C{data} folder. All test
     28         modules must be placed in the root package: csb.test.cases. There is
     29         a strict naming convention for test modules: the name of a test module
     30         should be the same as the name of the CSB API package it tests. For
     31         example, if you are writing tests for C{csb/bio/io/__init__.py}, the
     32         test module must be C{csb/test/cases/bio/io/__init__.py}. C{csb.test.cases}
     33         is the root package of all test modules in CSB.
     34
     35      2. Writing Tests
     36
     37         Writing a test is easy. All you need is to import csb.test and then
     38         create your own test cases, derived from L{csb.test.Case}:
     39
     40             >>> import csb.test
     41             >>> @csb.test.unit
     42                 class TestSomeClass(csb.test.Case):
     43                     def setUp(self):
     44                         super(TestSomeClass, self).setUp()
     45                         # do something with self.config here...
     46
     47         In this way your test case instance is automatically equipped with a
     48         reference to the test config, so your test method can be:
     49
     50             >>> @csb.test.unit
     51                 class TestSomeClass(csb.test.Case):
     52                     def testSomeMethod(self):
     53                         myDataFile = self.config.getTestFile('some.file')
     54                         self.assert...
     55
     56         The "unit" decorator marks a test case as a collection of unit tests.
     57         All possibilities are: L{csb.test.unit}, L{csb.test.functional}, L{csb.test.custom},
     58         and L{csb.test.regression}.
     59
     60         Writing custom (a.k.a. "data", "slow", "dynamic") tests is a little bit
     61         more work. Custom tests must be functions, not classes. Basically a
     62         custom test is a function, which builds a unittest.TestSuite instance
     63         and then returns it when called without arguments.
     64
     65         Regression tests are usually created in response to reported bugs. Therefore,
     66         the best practice is to mark each test method with its relevant bug ID:
     67
     68             >>> @csb.test.regression
     69                 class SomeClassRegressions(csb.test.Case)
     70                     def testSomeFeature(self)
     71                     \"""
     72                     @see: [CSB 000XXXX]
     73                     \"""
     74                     # regression test body...
     75
     76      3. Style Guide:
     77
     78         - name test case packages as already described
     79         - group tests in csb.test.Case-s and name them properly
     80         - prefix test methods with "test", like "testParser" - very important
     81         - use camelCase for methods and variables. This applies to all the
     82           code under csb.test (including test) and does not apply to the rest
     83           of the library!
     84         - for functional tests it's okay to define just one test method: runTest
     85         - for unit tests you should create more specific test names, for example:
     86           "testParseFile" - a unit test for some method called "parse_file"
     87         - use csb.test decorators to mark tests as unit, functional, regression, etc.
     88         - make every test module executable::
     89
     90             if __name__ == '__main__':
     91                 csb.test.Console()   # Discovers and runs all test cases in the module
     92
     93      4. Test Execution
     94
     95         Test discovery is handled by C{test builders} and a test runner
     96         C{app}. Test builders are subclasses of L{AbstractTestBuilder}.
     97         For every test type (unit, functional, regression, custom) there is a
     98         corresponding test builder. L{AnyTestBuilder} is a special builder which
     99         scans for unit, regression and functional tests at the same time.
    100
    101         Test builder classes inherit the following test discovery methods:
    102
    103             - C{loadTests} - load tests from a test namespace. Wildcard
    104               namespaces are handled by C{loadAllTests}
    105             - C{loadAllTests} - load tests from the given namespace, and
    106               from all sub-packages (recursive)
    107             - C{loadFromFile} - load tests from an absolute file name
    108             - C{loadMultipleTests} - calls C{loadTests} for a list of
    109               namespaces and combines all loaded tests in a single suite
    110
    111         Each of those return test suite objects, which can be directly executed
    112         with python's unittest runner.
    113
    114         Much simpler way to execute a test suite is to use our test app
    115         (C{csb/test/app.py}), which is simply an instance of L{csb.test.Console}::
    116
    117             $ python csb/test/app.py --help
    118
    119         The app has two main arguments:
    120
    121             - test type - tells the app which TestBuilder to use for test dicsovery
    122               ("any" triggers L{AnyTestBuilder}, "unit" - L{UnitTestBuilder}, etc.)
    123             - test namespaces - a list of "dotted" test modules, for example::
    124
    125                  csb.test.cases.bio.io.*   # io and sub-packages
    126                  csb.test.cases.bio.utils  # only utils
    127                  .                         # current module
    128
    129         In addition to running the app from the command line, you can run it
    130         also programmatically by instantiating L{csb.test.Console}. You can
    131         construct a test console object by passing a list of test namespace(s)
    132         and a test builder class to the Console's constructor.
    133
    134
    135      5. Commit Policies
    136
    137         Follow these guidelines when making changes to the repository:
    138
    139             - B{no bugs in "trunk"}: after fixing a bug or implementing a new
    140               feature, make sure at least the default test set passes by running
    141               the test console without any arguments. This is equivalent to:
    142               app.py -t any "csb.test.cases.*". (If no test case from this set covers
    143               the affected code, create a test case first, as described in the other
    144               policies)
    145
    146             - B{no recurrent issues}: when a bug is found, first write a regression
    147               test with a proper "@see: BugID" tag in the docstring. Run the test
    148               to make sure it fails. After fixing the bug, run the test again before
    149               you commit, as required by the previous policy
    150
    151             - B{test all new features}: there should be a test case for every new feature
    152               we implement. One possible approach is to write a test case first and
    153               make sure it fails; when the new feature is ready, run the test again
    154               to make sure it passes
    155
    156  @warning: for compatibility reasons do NOT import and use the unittest module
    157            directly. Always import unittest from csb.test, which is guaranteed
    158            to be python 2.7+ compatible. The standard unittest under python 2.6
    159            is missing some features, that's why csb.test will take care of
    160            replacing it with unittest2 instead.
    161  """
    162  import os
    163  import sys
    164  import imp
    165  import types
    166  import time
    167  import tempfile
    168  import traceback
    169  import argparse
    170
    171  import csb.io
    172  import csb.core
    173
    174  try:
    175      from unittest import skip, skipIf
    176      import unittest
    177  except ImportError:
    178      import unittest2 as unittest
    179
    180  from abc import ABCMeta, abstractproperty
    181
    182
    183 -class Attributes(object):
    184
    185      UNIT       = '__CSBUnitTest__'
    186      CUSTOM     = '__CSBCustomTest__'
    187      FUNCTIONAL = '__CSBFunctionalTest__'
    188      REGRESSION = '__CSBRegressionTest__'
    189
    190 -class Config(object):
    191      """
    192      General CSB Test Config. Config instances contain the following properties:
    193
    194          - data - path to the CSB Test Data directory. Default is L{Config.DATA}
    195          - temp - path to the system's temp directory. Default is L{Config.TEMP}
    196          - config - the L{Config} class
    197      """
    198
    199      DATA = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'data')
    200      """
    201      @cvar: path to the default test data directory: <install dir>/csb/test/data
    202      """
    203      GENERATED_DATA = DATA
    204      """
    205      @cvar: path to the default data directory for generated test files
    206      """
    207      TEMP = os.path.abspath(tempfile.gettempdir())
    208      """
    209      @cvar: path to the default system's temp directory
    210      """
    211
    212      @staticmethod
    213 -    def setDefaultDataRoot(path):
    214          """
    215          Override the default L{Config.DATA} with a new data root directory.
    216
    217          @param path: full directory path
    218          @type path: str
    219          """
    220          if not os.path.isdir(path):
    221              raise IOError('Path not found: {0}'.format(path))
    222
    223          Config.DATA = os.path.abspath(path)
    224
    225      @staticmethod
    226 -    def setDefaultGeneratedDataRoot(path):
    227          """
    228          Override the default L{Config.GENERATED_DATA} with a new data root directory.
    229
    230          @param path: full directory path
    231          @type path: str
    232          """
    233          if not os.path.isdir(path):
    234              raise IOError('Path not found: {0}'.format(path))
    235
    236          Config.GENERATED_DATA = os.path.abspath(path)
    237
    238      @property
    239 -    def data(self):
    240          """
    241          Test data directory
    242          @rtype: str
    243          """
    244          return Config.DATA
    245
    246      @property
    247 -    def generated_data(self):
    248          """
    249          Test data directory for generated files
    250          @rtype: str
    251          """
    252          return Config.GENERATED_DATA
    253
    254      @property
    255 -    def temp(self):
    256          """
    257          Test temp directory
    258          @rtype: str
    259          """
    260          return Config.TEMP
    261
    262 -    def getTestFile(self, fileName, subDir=''):
    263          """
    264          Search for C{fileName} in the L{Config.DATA} directory. If not found,
    265          try also L{Config.GENERATED_DATA} (if different).
    266
    267          @param fileName: the name of a test file to retrieve
    268          @type fileName: str
    269          @param subDir: scan a sub-directory of L{Config.DATA}
    270          @type subDir: str
    271
    272          @return: full path to C{fileName}
    273          @rtype: str
    274
    275          @raise IOError: if no such file is found
    276          """
    277          for data in [self.data, self.generated_data]:
    278              file = os.path.join(data, subDir, fileName)
    279
    280              if os.path.isfile(file):
    281                  return file
    282
    283          raise IOError('Test file not found: {0}'.format(fileName))
    284
    285 -    def getPickle(self, fileName, subDir=''):
    286          """
    287          Same as C{self.getTestFile}, but try to unpickle the the file
    288          and return the unpickled object. Pickles are usually stored in
    289          L{Config.GENERATED_DATA}.
    290
    291          @param fileName: the name of a test file to retrieve
    292          @type fileName: str
    293          @param subDir: scan a sub-directory of L{Config.DATA}
    294          @type subDir: str
    295
    296          @rtype: object
    297          """
    298          file = self.getTestFile(fileName, subDir)
    299          return csb.io.Pickle.load(open(file, 'rb'))
    300
    301 -    def getContent(self, fileName, subDir=''):
    302          """
    303          Same as C{self.getTestFile}, but also read and return the contents of
    304          the file.
    305
    306          @param fileName: the name of a test file to retrieve
    307          @type fileName: str
    308          @param subDir: scan a sub-directory of L{Config.DATA}
    309          @type subDir: str
    310
    311          @rtype: str
    312          """
    313          with open(self.getTestFile(fileName, subDir)) as f:
    314              return f.read()
    315
    316 -    def getTempStream(self, mode='t'):
    317          """
    318          Return a temporary file stream::
    319
    320              with self.getTempStream() as tmp:
    321                  tmp.write(something)
    322                  tmp.flush()
    323                  file_name = tmp.name
    324
    325          @param mode: file open mode (text, binary), default=t
    326          @type mode: str
    327          @rtype: file stream
    328          """
    329          return csb.io.TempFile(mode=mode)
    330
    331 -    def ensureDataConsistency(self):
    332          """
    333          Try to deserialize some pickled data files. Call L{Config.updateDataFiles}
    334          if the pickles appeared incompatible with the current interpreter.
    335          """
    336          try:
    337              self.getPickle('1nz9.model1.pickle')
    338          except:
    339              self.updateDataFiles()
    340
    341 -    def updateDataFiles(self):
    342          """
    343          Refresh the pickled structures in csb/test/data. This might be needed when
    344          the internal representation of some classes has changed.
    345          """
    346          from csb.io import Pickle
    347          from csb.bio.io.wwpdb import RegularStructureParser
    348          from csb.bio.structure import Ensemble, ChemElements
    349
    350          parser = RegularStructureParser(self.getTestFile('1nz9.pdb'))
    351          model1 = parser.parse_structure(model=1)
    352          model2 = parser.parse_structure(model=2)
    353
    354          ensemble = Ensemble()
    355          ensemble.models.append(model1)
    356          ensemble.models.append(model2)
    357          Pickle.dump(ensemble, open(os.path.join(self.generated_data, '1nz9.full.pickle'), 'wb'))
    358
    359          mse = model1.chains['A'].find(164)
    360          mse.label = 'MSE'
    361          mse.atoms['SD']._element = ChemElements.Se
    362          mse.atoms['SD']._full_name = 'SE  '
    363          Pickle.dump(model1, open(os.path.join(self.generated_data, '1nz9.model1.pickle'), 'wb'))
    364
    365 -class Case(unittest.TestCase):
    366      """
    367      Base class, defining a CSB Test Case. Provides a default implementation
    368      of C{unittest.TestCase.setUp} which grabs a reference to a L{Config}.
    369      """
    370
    371      @property
    372 -    def config(self):
    373          """
    374          Test config instance
    375          @rtype: L{Config}
    376          """
    377          return self.__config
    378
    379 -    def setUp(self):
    380          """
    381          Provide a reference to the CSB Test Config in the C{self.config} property.
    382          """
    383          self.__config = Config()
    384          assert hasattr(self.config, 'data'), 'The CSB Test Config must contain the data directory'
    385          assert self.config.data, 'The CSB Test Config must contain the data directory'
    386
    387 -    def reRaise(self, addArgs=()):
    388          """
    389          Re-raise the last exception with its full traceback, but modify the
    390          argument list with C{addArgs} and the original stack trace.
    391
    392          @param addArgs: additional arguments to append to the exception
    393          @type addArgs: tuple
    394          """
    395          klass, ex, _tb = sys.exc_info()
    396          ex.args = list(ex.args) + list(addArgs) + [''.join(traceback.format_exc())]
    397
    398          raise klass(ex.args)
    399
    400 -    def assertAlmostEqual(self, first, second, places=None, msg=None, delta=None):
    401
    402          if first == second:
    403              return
    404          if delta is not None and places is not None:
    405              raise TypeError("specify delta or places not both")
    406
    407          if delta is not None:
    408
    409              if abs(first - second) <= delta:
    410                  return
    411
    412              m = '{0} != {1} within {2} delta'.format(first, second, delta)
    413              msg = self._formatMessage(msg, m)
    414
    415              raise self.failureException(msg)
    416
    417          else:
    418              if places is None:
    419                  places = 7
    420
    421              return super(Case, self).assertAlmostEqual(first, second, places=places, msg=msg)
    422
    423 -    def assertFasterThan(self, duration, callable, *args, **kargs):
    424          """
    425          Fail if it took more than C{duration} seconds to invoke C{callable}.
    426
    427          @param duration: maximum amount of seconds allowed
    428          @type duration: float
    429          """
    430
    431          start = time.time()
    432          callable(*args, **kargs)
    433          execution = time.time() - start
    434
    435          if execution > duration:
    436              self.fail('{0}s is slower than {1}s)'.format(execution, duration))
    437
    438      @classmethod
    439 -    def execute(cls):
    440          """
    441          Run this test case.
    442          """
    443          suite = unittest.TestLoader().loadTestsFromTestCase(cls)
    444          runner = unittest.TextTestRunner()
    445
    446          return runner.run(suite)
    447
    448 -class InvalidNamespaceError(NameError, ImportError):
    449      pass
    450
    451 -class AbstractTestBuilder(object):
    452      """
    453      This is a base class, defining a test loader which exposes the C{loadTests}
    454      method.
    455
    456      Subclasses must override the C{labels} abstract property, which controls
    457      what kind of test cases are loaded by the test builder.
    458      """
    459
    460      __metaclass__ = ABCMeta
    461
    462      @abstractproperty
    463 -    def labels(self):
    464          pass
    465
    466 -    def loadFromFile(self, file):
    467          """
    468          Load L{csb.test.Case}s from a module file.
    469
    470          @param file: test module file name
    471          @type file: str
    472
    473          @return: a C{unittest.TestSuite} ready for the test runner
    474          @rtype: C{unittest.TestSuite}
    475          """
    476          mod = self._loadSource(file)
    477          suite = unittest.TestLoader().loadTestsFromModule(mod)
    478          return unittest.TestSuite(self._filter(suite))
    479
    480 -    def loadTests(self, namespace):
    481          """
    482          Load L{csb.test.Case}s from the given CSB C{namespace}. If the namespace
    483          ends with a wildcard, tests from sub-packages will be loaded as well.
    484          If the namespace is '__main__' or '.', tests are loaded from __main__.
    485
    486          @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will
    487                            load tests from '/csb/test/cases/bio/__init__.py'
    488          @type namespace: str
    489
    490          @return: a C{unittest.TestSuite} ready for the test runner
    491          @rtype: C{unittest.TestSuite}
    492          """
    493          if namespace.strip() == '.*':
    494              namespace = '__main__.*'
    495          elif namespace.strip() == '.':
    496              namespace = '__main__'
    497
    498          if namespace.endswith('.*'):
    499              return self.loadAllTests(namespace[:-2])
    500          else:
    501              loader = unittest.TestLoader()
    502              tests = loader.loadTestsFromName(namespace)
    503              return unittest.TestSuite(self._filter(tests))
    504
    505 -    def loadMultipleTests(self, namespaces):
    506          """
    507          Load L{csb.test.Case}s from a list of given CSB C{namespaces}.
    508
    509          @param namespaces: a list of test module namespaces, e.g.
    510                             ('csb.test.cases.bio', 'csb.test.cases.bio.io') will
    511                             load tests from '/csb/test/cases/bio.py' and
    512                             '/csb/test/cases/bio/io.py'
    513          @type namespaces: tuple of str
    514
    515          @return: a C{unittest.TestSuite} ready for the test runner
    516          @rtype: C{unittest.TestSuite}
    517          """
    518          if not csb.core.iterable(namespaces):
    519              raise TypeError(namespaces)
    520
    521          return unittest.TestSuite(self.loadTests(n) for n in namespaces)
    522
    523 -    def loadAllTests(self, namespace, extension='.py'):
    524          """
    525          Load L{csb.test.Case}s recursively from the given CSB C{namespace} and
    526          all of its sub-packages. Same as::
    527
    528              builder.loadTests('namespace.*')
    529
    530          @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will
    531                            load tests from /csb/test/cases/bio/*'
    532          @type namespace: str
    533
    534          @return: a C{unittest.TestSuite} ready for the test runner
    535          @rtype: C{unittest.TestSuite}
    536          """
    537          suites = []
    538
    539          try:
    540              base = __import__(namespace, level=0, fromlist=['']).__file__
    541          except ImportError:
    542              raise InvalidNamespaceError('Namespapce {0} is not importable'.format(namespace))
    543
    544          if os.path.splitext(os.path.basename(base))[0] != '__init__':
    545              suites.append(self.loadTests(namespace))
    546
    547          else:
    548
    549              for entry in os.walk(os.path.dirname(base)):
    550
    551                  for item in entry[2]:
    552                      file = os.path.join(entry[0], item)
    553                      if extension and item.endswith(extension):
    554                          suites.append(self.loadFromFile(file))
    555
    556          return unittest.TestSuite(suites)
    557
    558 -    def _loadSource(self, path):
    559          """
    560          Import and return the Python module identified by C{path}.
    561
    562          @note: Module objects behave as singletons. If you import two different
    563                 modules and give them the same name in imp.load_source(mn), this
    564                 counts for a redefinition of the module originally named mn, which
    565                 is basically the same as reload(mn). Therefore, you need to ensure
    566                 that for every call to imp.load_source(mn, src.py) the mn parameter
    567                 is a string that uniquely identifies the source file src.py.
    568          """
    569          name = os.path.splitext(os.path.abspath(path))[0]
    570          name = name.replace('.', '-').rstrip('__init__').strip(os.path.sep)
    571
    572          return imp.load_source(name, path)
    573
    574 -    def _recurse(self, obj):
    575          """
    576          Extract test cases recursively from a test C{obj} container.
    577          """
    578          cases = []
    579          if isinstance(obj, unittest.TestSuite) or csb.core.iterable(obj):
    580              for item in obj:
    581                  cases.extend(self._recurse(item))
    582          else:
    583              cases.append(obj)
    584          return cases
    585
    586 -    def _filter(self, tests):
    587          """
    588          Filter a list of objects using C{self.labels}.
    589          """
    590          filtered = []
    591
    592          for test in self._recurse(tests):
    593              for label in self.labels:
    594                  if hasattr(test, label) and getattr(test, label) is True:
    595                      filtered.append(test)
    596
    597          return filtered
    598
    599 -class AnyTestBuilder(AbstractTestBuilder):
    600      """
    601      Build a test suite of cases, marked as either unit, functional or regression
    602      tests. For detailed documentation see L{AbstractTestBuilder}.
    603      """
    604      @property
    605 -    def labels(self):
    606          return [Attributes.UNIT, Attributes.FUNCTIONAL, Attributes.REGRESSION]
    607
    608 -class UnitTestBuilder(AbstractTestBuilder):
    609      """
    610      Build a test suite of cases, marked as unit tests.
    611      For detailed documentation see L{AbstractTestBuilder}.
    612      """
    613      @property
    614 -    def labels(self):
    615          return [Attributes.UNIT]
    616
    617 -class FunctionalTestBuilder(AbstractTestBuilder):
    618      """
    619      Build a test suite of cases, marked as functional tests.
    620      For detailed documentation see L{AbstractTestBuilder}.
    621      """
    622      @property
    623 -    def labels(self):
    624          return [Attributes.FUNCTIONAL]
    625
    626 -class RegressionTestBuilder(AbstractTestBuilder):
    627      """
    628      Build a test suite of cases, marked as regression tests.
    629      For detailed documentation see L{AbstractTestBuilder}.
    630      """
    631      @property
    632 -    def labels(self):
    633          return [Attributes.REGRESSION]
    634
    635 -class CustomTestBuilder(AbstractTestBuilder):
    636      """
    637      Build a test suite of cases, marked as custom tests. CustomTestBuilder will
    638      search for functions, marked with the 'custom' test decorator, which return
    639      a dynamically built C{unittest.TestSuite} object when called without
    640      parameters. This is convenient when doing data-related tests, e.g.
    641      instantiating a single type of a test case many times iteratively, for
    642      each entry in a database.
    643
    644      For detailed documentation see L{AbstractTestBuilder}.
    645      """
    646      @property
    647 -    def labels(self):
    648          return [Attributes.CUSTOM]
    649
    650 -    def loadFromFile(self, file):
    651
    652          mod = self._loadSource(file)
    653          suites = self._inspect(mod)
    654
    655          return unittest.TestSuite(suites)
    656
    657 -    def loadTests(self, namespace):
    658
    659          if namespace.strip() == '.*':
    660              namespace = '__main__.*'
    661          elif namespace.strip() == '.':
    662              namespace = '__main__'
    663
    664          if namespace.endswith('.*'):
    665              return self.loadAllTests(namespace[:-2])
    666          else:
    667              try:
    668                  mod = __import__(namespace, fromlist=[''])
    669              except ImportError:
    670                  raise InvalidNamespaceError('Namespace {0} is not importable'.format(namespace))
    671              suites = self._inspect(mod)
    672              return unittest.TestSuite(suites)
    673
    674 -    def _inspect(self, module):
    675
    676          objects = map(lambda n: getattr(module, n), dir(module))
    677          return self._filter(objects)
    678
    679 -    def _filter(self, factories):
    680          """
    681          Filter a list of objects using C{self.labels}.
    682          """
    683          filtered = []
    684
    685          for obj in factories:
    686              for label in self.labels:
    687                  if hasattr(obj, label) and getattr(obj, label) is True:
    688                      suite = obj()
    689                      if not isinstance(suite, unittest.TestSuite):
    690                          raise ValueError('Custom test function {0} must return a '
    691                                           'unittest.TestSuite, not {1}'.format(obj.__name__, type(suite)))
    692                      filtered.append(suite)
    693
    694          return filtered
    695
    696 -def unit(klass):
    697      """
    698      A class decorator, used to label unit test cases.
    699
    700      @param klass: a C{unittest.TestCase} class type
    701      @type klass: type
    702      """
    703      if not isinstance(klass, type):
    704          raise TypeError("Can't apply class decorator on {0}".format(type(klass)))
    705
    706      setattr(klass, Attributes.UNIT, True)
    707      return klass
    708
    709 -def functional(klass):
    710      """
    711      A class decorator, used to label functional test cases.
    712
    713      @param klass: a C{unittest.TestCase} class type
    714      @type klass: type
    715      """
    716      if not isinstance(klass, type):
    717          raise TypeError("Can't apply class decorator on {0}".format(type(klass)))
    718
    719      setattr(klass, Attributes.FUNCTIONAL, True)
    720      return klass
    721
    722 -def regression(klass):
    723      """
    724      A class decorator, used to label regression test cases.
    725
    726      @param klass: a C{unittest.TestCase} class type
    727      @type klass: type
    728      """
    729      if not isinstance(klass, type):
    730          raise TypeError("Can't apply class decorator on {0}".format(type(klass)))
    731
    732      setattr(klass, Attributes.REGRESSION, True)
    733      return klass
    734
    735 -def custom(function):
    736      """
    737      A function decorator, used to mark functions which build custom (dynamic)
    738      test suites when called.
    739
    740      @param function: a callable object, which returns a dynamically compiled
    741                       C{unittest.TestSuite}
    742      @type function: callable
    743      """
    744      if isinstance(function, type):
    745          raise TypeError("Can't apply function decorator on a class")
    746      elif not hasattr(function, '__call__'):
    747          raise TypeError("Can't apply function decorator on non-callable {0}".format(type(function)))
    748
    749      setattr(function, Attributes.CUSTOM, True)
    750      return function
    751
    752 -def skip(reason, condition=None):
    753      """
    754      Mark a test case or method for skipping.
    755
    756      @param reason: message
    757      @type reason: str
    758      @param condition: skip only if the specified condition is True
    759      @type condition: bool/expression
    760      """
    761      if isinstance(reason, types.FunctionType):
    762          raise TypeError('skip: no reason specified')
    763
    764      if condition is None:
    765          return unittest.skip(reason)
    766      else:
    767          return unittest.skipIf(condition, reason)
    768
    769 -class Console(object):
    770      """
    771      Build and run all tests of the specified namespace and kind.
    772
    773      @param namespace: a dotted name, which specifies the test module
    774                        (see L{csb.test.AbstractTestBuilder.loadTests})
    775      @type namespace: str
    776      @param builder: test builder to use
    777      @type builder: any L{csb.test.AbstractTestBuilder} subclass
    778      @param verbosity: verbosity level for C{unittest.TestRunner}
    779      @type verbosity: int
    780      @param update: if True, refresh all pickles in csb/test/data
    781      @type update: bool
    782      @param generated_data: where to cache generated test files (directory)
    783      @type generated_data: str
    784      """
    785
    786      BUILDERS = {'unit': UnitTestBuilder, 'functional': FunctionalTestBuilder,
    787                  'custom': CustomTestBuilder, 'any': AnyTestBuilder,
    788                  'regression': RegressionTestBuilder}
    789
    790
    791 -    def __init__(self, namespace=('__main__',), builder=AnyTestBuilder, verbosity=1,
    792                   update=False, generated_data=Config.GENERATED_DATA, argv=None):
    793
    794          if not argv:
    795              argv = sys.argv
    796
    797          self._namespace = None
    798          self._builder = None
    799          self._verbosity = 1
    800          self._update = False
    801          self._gendata = str(generated_data)
    802          self._program = os.path.basename(argv[0])
    803
    804          self.namespace = namespace
    805          self.builder = builder
    806          self.verbosity = verbosity
    807          self.update = update
    808          self.generated_data = generated_data
    809
    810          self.parseArguments(argv[1:])
    811          self.run()
    812
    813      @property
    814 -    def namespace(self):
    815          return self._namespace
    816      @namespace.setter
    817 -    def namespace(self, value):
    818          if csb.core.iterable(value):
    819              self._namespace = list(value)
    820          else:
    821              self._namespace = [value]
    822
    823      @property
    824 -    def builder(self):
    825          return self._builder
    826      @builder.setter
    827 -    def builder(self, value):
    828          self._builder = value
    829
    830      @property
    831 -    def verbosity(self):
    832          return self._verbosity
    833      @verbosity.setter
    834 -    def verbosity(self, value):
    835          self._verbosity = value
    836
    837      @property
    838 -    def builders(self):
    839          return ', '.join(Console.BUILDERS)
    840
    841      @property
    842 -    def program(self):
    843          return self._program
    844
    845      @property
    846 -    def update(self):
    847          return self._update
    848      @update.setter
    849 -    def update(self, value):
    850          self._update = bool(value)
    851
    852      @property
    853 -    def generated_data(self):
    854          return self._gendata
    855      @generated_data.setter
    856 -    def generated_data(self, value):
    857          self._gendata = os.path.abspath(value)
    858
    859 -    def run(self):
    860
    861          Config.setDefaultGeneratedDataRoot(self.generated_data)
    862
    863          if self.update:
    864              Config().updateDataFiles()
    865          else:
    866              Config().ensureDataConsistency()
    867
    868          builder = self.builder()
    869          suite = builder.loadMultipleTests(self.namespace)
    870
    871          runner = unittest.TextTestRunner(verbosity=self.verbosity)
    872          runner.run(suite)
    873
    874 -    def parseArguments(self, argv):
    875
    876          parser = argparse.ArgumentParser(prog=self.program, description="CSB Test Runner Console.")
    877
    878          parser.add_argument("-t", "--type", type=str, default="any", choices=list(Console.BUILDERS),
    879                              help="Type of tests to load from each namespace (default=any)")
    880          parser.add_argument("-v", "--verbosity", type=int, default=1,
    881                              help="Verbosity level passed to unittest.TextTestRunner (default=1).")
    882          parser.add_argument("-u", "--update-files", default=False, action="store_true",
    883                              help="Force update of the test pickles in " + Config.GENERATED_DATA)
    884          parser.add_argument("-g", "--generated-resources", type=str, default=Config.GENERATED_DATA,
    885                              help="Generate, store and load additional test resources in this directory"
    886                              " (default=" + Config.GENERATED_DATA + ")")
    887
    888          parser.add_argument("namespaces", nargs='*',
    889                              help="""An optional list of CSB test dotted namespaces, from which to
    890                                      load tests. '__main__' and '.' are interpreted as the
    891                                      current module. If a namespace ends with an asterisk
    892                                      '.*', all sub-packages will be scanned as well.
    893
    894                                      Examples:
    895                                          "csb.test.cases.bio.*"
    896                                          "csb.test.cases.bio.io" "csb.test.cases.bio.utils"
    897                                          ".")""")
    898
    899          args = parser.parse_args(argv)
    900
    901          self.builder = Console.BUILDERS[args.type]
    902          self.verbosity = args.verbosity
    903          self.update = args.update_files
    904          self.generated_data = args.generated_resources
    905
    906          if args.namespaces:
    907              self.namespace = args.namespaces
    908
    909
    910  if __name__ == '__main__':
    911
    912      Console()
    913