Metadata-Version: 1.0
Name: torchtest
Version: 0.2.1dev
Summary: UNKNOWN
Home-page: UNKNOWN
Author: UNKNOWN
Author-email: UNKNOWN
License: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
Description: = TorchTest
        
        A tiny test suite for pytorch based Machine Learning models, inspired by https://github.com/Thenerdstation/mltest/blob/master/mltest/mltest.py[mltest]. 
        Chase Roberts lists out 4 basic tests in his https://medium.com/@keeper6928/mltest-automatically-test-neural-network-models-in-one-function-call-eb6f1fa5019d[medium post] about mltest. 
        torchtest is sort of a pytorch port of mltest which was written for tensorflow models.
        
        
        == Tests
        
        
        [source, python]
        ----
        import torch
        import torch.nn as nn
        import torch.nn.functional as F
        from torch.autograd import Variable
        
        from torchtest import torchtest as tt
        ----
        
        
        === Variables Change
        
        [source, python]
        ----
        inputs = Variable(torch.randn(20, 20))
        targets = Variable(torch.randint(0, 2, (20,))).long()
        batch = [inputs, targets]
        model = nn.Linear(20, 2)
        
        # what are the variables?
        print('Our list of parameters', [ np[0] for np in model.named_parameters() ])
        
        # do they change after a training step?
        #  let's run a train step and see
        tt.assert_vars_change(
            model=model, 
            loss_fn=F.cross_entropy, 
            optim=torch.optim.Adam(model.parameters()),
            batch=batch)
        ----
        
        [source, python]
        ----
        # let's try to break this, so the test fails
        params_to_train = [ np[1] for np in model.named_parameters() if np[0] is not 'bias' ]
        # run test now
        """ FAILURE """
        tt.assert_vars_change(
            model=model, 
            loss_fn=F.cross_entropy, 
            optim=torch.optim.Adam(params_to_train),
            batch=batch)
        
        # YES! bias did not change
        ----
        
        
        === Variables Don't Change
        
        [source, python]
        ----
        # What if bias is not supposed to change, by design?
        #  test to see if bias remains the same after training
        tt.assert_vars_same(
            model=model, 
            loss_fn=F.cross_entropy, 
            optim=torch.optim.Adam(params_to_train),
            batch=batch,
            params=[('bias', model.bias)] 
            )
        # it does? good. let's move on
        ----
        
        === Output Range
        
        [source, python]
        ----
        # NOTE : bias is fixed (not trainable)
        optim = torch.optim.Adam(params_to_train)
        loss_fn=F.cross_entropy
        
        tt.test_suite(model, loss_fn, optim, batch, 
            output_range=(-2, 2),
            test_output_range=True
            )
        
        # seems to work
        ----
        
        [source, python]
        ----
        #  let's tweak the model to fail the test
        model.bias = nn.Parameter(2 + torch.randn(2, ))
        
        tt.test_suite(
            model,
            loss_fn, optim, batch, 
            output_range=(-1, 1),
            test_output_range=True
            )
        
        # as expected, it fails; yay!
        ----
        
        === NaN Tensors
        
        [source, python]
        ----
        model.bias = nn.Parameter(float('NaN') * torch.randn(2, ))
        
        tt.test_suite(
            model,
            loss_fn, optim, batch, 
            test_nan_vals=True
            )
        ----
        
        === Inf Tensors
        
        [source, python]
        ----
        model.bias = nn.Parameter(float('Inf') * torch.randn(2, ))
        
        tt.test_suite(
            model,
            loss_fn, optim, batch, 
            test_inf_vals=True
            )
        ----
        
Platform: UNKNOWN
