Metadata-Version: 2.0
Name: taskgraph
Version: 0.2.2
Summary: Parallel task graph framework.
Home-page: https://bitbucket.org/richsharp/taskgraph
Author: Rich Sharp
Author-email: richpsharp@gmail.com
License: BSD
Keywords: parallel multiprocessing distributed computing
Platform: UNKNOWN
Classifier: Intended Audience :: Developers
Classifier: Development Status :: 5 - Production/Stable
Classifier: Natural Language :: English
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python :: 2 :: Only
Classifier: License :: OSI Approved :: BSD License

TaskGraph:
=================================================

About TaskGraph
===============

TaskGraph is great.

TaskGraph Dependencies
======================

Task Graph is written in pure Python, but if the `psutils` package is
installed the distributed multiprocessing processeses will be `nice`\d.

Example Use
===========

Install taskgraph with

`pip install taskgraph`

Then

.. code-block:: python

  def _create_list_on_disk(value, length, target_path):
      """Create a numpy array on disk filled with value of `size`."""
      target_list = [value] * length
      pickle.dump(target_list, open(target_path, 'wb'))


  def _sum_lists_from_disk(list_a_path, list_b_path, target_path):
      """Read two lists, add them and save result."""
      list_a = pickle.load(open(list_a_path, 'rb'))
      list_b = pickle.load(open(list_b_path, 'rb'))
      target_list = []
      for a, b in zip(list_a, list_b):
          target_list.append(a+b)
      pickle.dump(target_list, open(target_path, 'wb'))

  # create a taskgraph that uses 4 multiprocessing subprocesses when possible
  task_graph = taskgraph.TaskGraph(self.workspace_dir, 4)
  target_a_path = os.path.join(self.workspace_dir, 'a.dat')
  target_b_path = os.path.join(self.workspace_dir, 'b.dat')
  result_path = os.path.join(self.workspace_dir, 'result.dat')
  result_2_path = os.path.join(self.workspace_dir, 'result2.dat')
  value_a = 5
  value_b = 10
  list_len = 10
  task_a = task_graph.add_task(
      target=_create_list_on_disk,
      args=(value_a, list_len, target_a_path),
      target_path_list=[target_a_path])
  task_b = task_graph.add_task(
      target=_create_list_on_disk,
      args=(value_b, list_len, target_b_path),
      target_path_list=[target_b_path])
  sum_task = task_graph.add_task(
      target=_sum_lists_from_disk,
      args=(target_a_path, target_b_path, result_path),
      target_path_list=[result_path],
      dependent_task_list=[task_a, task_b])
  sum_task.join()

  # expect that result is a list `list_len` long with `value_a+value_b` in it
  result = pickle.load(open(result_path, 'rb'))


