Metadata-Version: 1.1
Name: minform
Version: 0.2.1
Summary: WTForms/struct integration to validate and serialize to packed buffers of binary data.
Home-page: https://github.com/daviddonna/minform
Author: David Donna
Author-email: davidadonna@gmail.com
License: ISCL
Description: Minform
        =======
        
        .. image:: https://img.shields.io/travis/daviddonna/minform.svg
           :target: https://travis-ci.org/daviddonna/minform
        .. image:: http://img.shields.io/pypi/v/minform.svg
           :target: https://pypi.python.org/pypi/minform
        
        The declarative ease of
        `WTForms <https://github.com/wtforms/wtforms>`_, the small data
        footprint of
        `struct <https://docs.python.org/3/library/struct.html>`_.
        
        For more detailed examples, read the full docs at
        https://minform.readthedocs.org.
        
        Why does Minform exist?
        -----------------------
        
        Let's talk about data size.
        
        Say you have a fleet of devices with cell modems that report to a web server.
        Here's an example of a data packet struct you might use:
        
        .. code-block:: c
        
            struct sensor_data {
                char version[8];
                char serial[12];
                int32_t latitude;    // fixed-point * 100000
                int32_t longitude;   // fixed-point * 100000
                int16_t temperature; // fixed-point * 100
                uint16_t pings;
                uint8_t battery_pct;
            };
        
        Let's say they're reporting their data in JSON format, because that's lighter
        than XML but still coherent to most server frameworks. Here's a data packet:
        
        .. code-block:: javascript
        
            {
                "version": "1.0",
                "serial": "DEADBEEF",
                "latitude": 4071270,
                "longitude": -7400590,
                "temperature": 3200,
                "pings": 123,
                "battery_pct": 62
            }
        
        If you take out all the whitespace, that's **144 bytes**. (ASCII encoding).
        Maybe that's all you need, but maybe you need to store billions or trillions
        of these little guys. Worse, maybe you need to pay through the tear ducts for
        cellular data. It would be nice for that data to be smaller, and to be
        predictably sized.
        
        Besides, let's face it: serializing to JSON from C is a pain in the patoot.
        Depending on the library, you could be looking at ten or twenty lines of code
        (or a truly epic ``sprintf``), just to serialize a record.
        
        On the Python side, we can probably just use a form library like WTForms to
        validate incoming data, but we've already paid a price to make that data
        server-friendly.
        
        What can Minform do for me?
        ---------------------------
        
        Let's build a Minform form to handle incoming sensor data.
        
        .. code-block:: python
        
            from minform import *
        
            class SensorData(BinaryForm):
        
                order = LITTLE_ENDIAN  # let's say our devices are little-endian
                version = BytesField(max_length=8, length=AUTOMATIC)
                serial = BytesField(max_length=12, length=AUTOMATIC)
                latitude = Int32Field()
                longitude = Int32Field()
                temperature = Int16Field()
                pings = UInt16Field()
                battery_pct = UInt8Field()
        
                # We could cut the serialized record off three bytes early, but let's
                # say we want to break on four-byte word boundaries for reasons.
                padding = BlankBytes(3)
        
        Here's the C code that will serialize your structure:
        
        .. code-block:: c
        
            #include <string.h>
        
            char send_buffer[36];
            struct sensor_data data;
        
            // ...
        
            memcpy(send_buffer, &data, sizeof(data));
        
        And here's the Python that will receive it:
        
        .. code-block:: python
        
            form = SensorData.unpack(serialized_data)
        
        That serialized record is **36 bytes**. 36 on the wire, 36 in a file. You may
        need to tweak the form definition, depending on your C compiler and the target
        architecture, but Minforms gives you the tools to cope with padding bytes, and
        even mixed byte ordering.
        
        Let's fill in some gaps
        -----------------------
        
        Minforms are an awful lot like WTForms: you subclass ``minform.BinaryForm``,
        and add ``BinaryField``\ s as class properties. Here's another quick example:
        
        .. code:: python
        
            import minform
        
            class MyForm(minform.BinaryForm):
                '''
                This is a subclass of wtforms.Form: you can validate data with it,
                construct it from an HTML form, extract the data as a Python dict, etc.
                '''
                first_name = minform.BytesField('First Name', max_length=10)
                last_name = minform.BytesField('Last Name', max_length=10)
                age = minform.UInt8Field('Age')
        
            #               first_name (10)          last_name (10)           age (1)
            packed_data = b'David\x00\x00\x00\x00\x00Donna\x00\x00\x00\x00\x00\x18'
            form = MyForm.unpack(packed_data)
            assert form.data == {
                'first_name': b'David',
                'last_name': b'Donna',
                'age': 24,
            }
        
            next_form = MyForm(first_name=b'Foo', last_name=b'Barsson', age=100)
            packed = next_form.pack()
            assert packed == b'Foo\x00\x00\x00\x00\x00\x00\x00Barsson\x00\x00\x00\x64'
        
        Because the library is built on ``struct``, binary serializations of a form's
        data are in fixed-length buffers. This makes them easier to store, and easy to
        map onto relatively naive serializations of C structs. It also allows for
        clear documentation of the binary format, because the data maps predictably
        onto different positions in a packed buffer.
        
        Compound BinaryFields allow you to create nested structures that still
        serialize into flat buffers.
        
        .. code:: python
        
            class MyBigBadForm(minform.BinaryForm):
                """
                This is taking a turn for campy criminality.
                """
                riches = minform.Int16Field()
                goons = minform.BinaryFieldList(Person, max_entries=4, length=minform.EXPLICIT)
        
            squad = MyBigBadForm(riches=55223, goons=[
                {'first_name': 'Joey', 'last_name': 'Schmoey', 'age': 32},
                {'first_name': 'Manny', 'last_name': 'The Man', 'age': 40},
                {'first_name': 'Gerta', 'last_name': 'Goethe', 'age': 52},
            ])
            assert squad.pack() == (b'\xd7\xb7' +                                  # riches
                                    b'\x03' +                                      # goons prefix
                                    b'Joey\0\0\0\0\0\0Schmoey\0\0\0\x20' +         # goons[0]
                                    b'Manny\0\0\0\0\0The Man\0\0\0\x28' +          # goons[1]
                                    b'Gerta\0\0\0\0\0Goethe\0\0\0\0\x34' +         # goons[2]
                                    b'\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0') # goons[3]
        
        Even with an entire set of blank bytes for ``goons[3]``, that's 87 bytes, vs
        185 for the JSON representation.
        
        
        
        
        History
        -------
        
        0.1.0 (10/28/2015)
        
            Initial release
        
        0.2.1 (11/22/2015)
        
            - Add pack_into and unpack_from methods to BinaryForm and BinaryItem.
            - Rename all 'buf' arguments to 'buffer'.
Keywords: minform wtforms struct binary
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: ISC License (ISCL)
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
