Metadata-Version: 1.1
Name: tensorflow-qnd
Version: 0.0.9
Summary: Quick and Dirty TensorFlow command framework
Home-page: https://github.com/raviqqe/tensorflow-qnd/
Author: Yota Toyama
Author-email: raviqqe@gmail.com
License: Public Domain
Description: # tensorflow-qnd
        
        [![PyPI version](https://badge.fury.io/py/tensorflow-qnd.svg)](https://badge.fury.io/py/tensorflow-qnd)
        [![Python versions](https://img.shields.io/pypi/pyversions/tensorflow-qnd.svg)](setup.py)
        [![Build Status](https://travis-ci.org/raviqqe/tensorflow-qnd.svg?branch=master)](https://travis-ci.org/raviqqe/tensorflow-qnd)
        [![License](https://img.shields.io/badge/license-unlicense-lightgray.svg)](https://unlicense.org)
        
        Quick and Dirty TensorFlow command framework
        
        tensorflow-qnd is a TensorFlow framework to create commands to train and
        evaluate models and make inference with them.
        The framework is built on top of
        [TF Learn](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/learn/python/learn).
        
        
        ## Features
        
        - Creation of commands
          - To train and evaluate models
          - To infer labels or regression values with trained models
        - Configuration of command line arguments to set hyperparameters of models etc.
        - Distributed TensorFlow (only training and evaluation)
          - Just set an optional argument `distributed ` of `def_train_and_evaluate()`
            as `True`. (i.e. `def_train_and_evaluate(distributed=True)`)
        
        
        ## Installation
        
        Python 3.5+ and TensorFlow 0.12+ are required.
        
        ```
        pip3 install --user --upgrade tensorflow-qnd
        ```
        
        
        ## Usage
        
        1. Add command line arguments with `add_flag` and `add_required_flag` functions.
        2. Define a `train_and_evaluate` or `infer` function with
           `def_train_and_evaluate` or `def_infer` function
        3. Pass `model_fn` (model constructor) and `input_fn` (input producer) functions
           to that defined function.
        4. Run the script with appropriate command line arguments.
        
        For more information, see [documentation](https://raviqqe.github.io/tensorflow-qnd/qnd).
        
        
        ## Examples
        
        `train.py` (command script):
        
        ```python
        import logging
        import os
        
        import qnd
        
        import mnist
        
        
        train_and_evaluate = qnd.def_train_and_evaluate(
            distributed=("distributed" in os.environ))
        
        
        model = mnist.def_model()
        
        
        def main():
            logging.getLogger().setLevel(logging.INFO)
            train_and_evaluate(model, mnist.read_file)
        
        
        if __name__ == "__main__":
            main()
        ```
        
        `mnist.py` (module):
        
        ```python
        import qnd
        import tensorflow as tf
        
        
        def read_file(filename_queue):
            _, serialized = tf.TFRecordReader().read(filename_queue)
        
            scalar_feature = lambda dtype: tf.FixedLenFeature([], dtype)
        
            features = tf.parse_single_example(serialized, {
                "image_raw": scalar_feature(tf.string),
                "label": scalar_feature(tf.int64),
            })
        
            image = tf.decode_raw(features["image_raw"], tf.uint8)
            image.set_shape([28**2])
        
            return tf.to_float(image) / 255 - 0.5, features["label"]
        
        
        def minimize(loss):
            return tf.train.AdamOptimizer().minimize(
                loss,
                tf.contrib.framework.get_global_step())
        
        
        def def_model():
            qnd.add_flag("hidden_layer_size", type=int, default=64,
                         help="Hidden layer size")
        
            def model(image, number=None, mode=None):
                h = tf.contrib.layers.fully_connected(image,
                                                      qnd.FLAGS.hidden_layer_size)
                h = tf.contrib.layers.fully_connected(h, 10, activation_fn=None)
        
                predictions = tf.argmax(h, axis=1)
        
                if mode == tf.contrib.learn.ModeKeys.INFER:
                    return predictions
        
                loss = tf.reduce_mean(
                    tf.nn.sparse_softmax_cross_entropy_with_logits(h, number))
        
                return predictions, loss, minimize(loss), {
                    "accuracy": tf.contrib.metrics.streaming_accuracy(predictions,
                                                                      number)[1],
                }
        
            return model
        ```
        
        With the code above, you can create a command with the following interface.
        
        ```
        usage: train.py [-h] [--output_dir OUTPUT_DIR] [--train_steps TRAIN_STEPS]
                        [--eval_steps EVAL_STEPS]
                        [--min_eval_frequency MIN_EVAL_FREQUENCY]
                        [--num_cores NUM_CORES] [--log_device_placement]
                        [--save_summary_steps SAVE_SUMMARY_STEPS]
                        [--save_checkpoints_steps SAVE_CHECKPOINTS_STEPS]
                        [--batch_size BATCH_SIZE]
                        [--batch_queue_capacity BATCH_QUEUE_CAPACITY] --train_file
                        TRAIN_FILE [--filename_queue_capacity FILENAME_QUEUE_CAPACITY]
                        --eval_file EVAL_FILE [--hidden_layer_size HIDDEN_LAYER_SIZE]
        
        optional arguments:
          -h, --help            show this help message and exit
          --output_dir OUTPUT_DIR
                                Directory where checkpoint and event files are stored
                                (default: output)
          --train_steps TRAIN_STEPS
                                Maximum number of train steps (default: None)
          --eval_steps EVAL_STEPS
                                Maximum number of eval steps (default: None)
          --min_eval_frequency MIN_EVAL_FREQUENCY
                                Minimum evaluation frequency in number of model
                                savings (default: 1)
          --num_cores NUM_CORES
                                Number of CPU cores used. 0 means use of a default
                                value. (default: 0)
          --log_device_placement
                                If specified, log device placement information
                                (default: False)
          --save_summary_steps SAVE_SUMMARY_STEPS
                                Number of steps every time of which summary is saved
                                (default: 100)
          --save_checkpoints_steps SAVE_CHECKPOINTS_STEPS
                                Number of steps every time of which a model is saved
                                (default: None)
          --batch_size BATCH_SIZE
                                Mini-batch size (default: 64)
          --batch_queue_capacity BATCH_QUEUE_CAPACITY
                                Batch queue capacity (default: 1024)
          --train_file TRAIN_FILE
                                File path of train data file(s). A glob is available.
                                (e.g. train/*.tfrecords) (default: None)
          --filename_queue_capacity FILENAME_QUEUE_CAPACITY
                                Capacity of filename queues of train, eval and infer
                                data (default: 32)
          --eval_file EVAL_FILE
                                File path of eval data file(s). A glob is available.
                                (e.g. eval/*.tfrecords) (default: None)
          --hidden_layer_size HIDDEN_LAYER_SIZE
                                Hidden layer size (default: 64)
        ```
        
        Explore [examples](examples) directory for more information and see how to run
        them.
        
        
        ## Caveats
        
        ### Necessary update of a global step variable
        
        As done in [examples](examples), you must get a global step variable with
        `tf.contrib.framework.get_global_step()` and update (increment) it in each
        training step.
        
        
        ### Use streaming metrics for `eval_metric_ops`
        
        When non-streaming ones such as `tf.contrib.metrics.accuracy` are used in a
        return value `eval_metric_ops` of your `model_fn` or as arguments of
        `ModelFnOps`, their values will be ones of the last batch in every evaluation
        step.
        
        
        ## License
        
        [The Unlicense](https://unlicense.org)
        
        
        ## References
        
        - [Distributed TensorFlow | TensorFlow](https://www.tensorflow.org/how_tos/distributed/)
        
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: Public Domain
Classifier: Programming Language :: Python :: 3.5
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Networking
