
.. _build pipeline:

==================
The Build Pipeline
==================

Engineer goes through a number of steps when building a site. Understanding this order and the steps that are taken
might be helpful when building your site. At the very least, it might be interesting.


Basic Flow
==========

Engineer operates almost entirely on something called the 'output cache.' This is a location in the
:attr:`~engineer.conf.EngineerConfiguration.CACHE_DIR`. Engineer essentially 'stages' all content there during
processing. Once that's all done, then it synchronizes the actual
:attr:`~engineer.conf.EngineerConfiguration.OUTPUT_DIR` with the cache.

With that in mind, these are the basic steps that Engineer goes through. You might also find it interesting to look at
the code for the :func:`engineer.engine.build` function - it is the primary entry point for the build process.

#. **Copy base Engineer static content**

   First, any static content from the core Engineer libraries is copied to the output cache. This includes things
   like the built-in jQuery, Modernizr, and Foundation libraries.

#. **Copy theme static content**

   Any static content needed by the theme is copied to the output cache.

#. **Generate template pages**

   :ref:`template pages` are generated and copied to the output cache.

#. **Load posts**

   Posts are loaded from the post cache and :attr:`~engineer.conf.EngineerConfiguration.POST_DIR` and stored in
   memory in a :class:`~engineer.models.PostCollection`.

#. **Generate posts**

   Based on settings like :attr:`~engineer.conf.EngineerConfiguration.PUBLISH_DRAFTS`
   and :attr:`~engineer.conf.EngineerConfiguration.PUBLISH_PENDING`, the list of posts is trimmed to those that
   should be output, then they are rendered and placed in the output cache.

#. **Generate rollup pages**

   The front page and other 'rollup' pages are generated and placed in the output cache.

#. **Generate archive page**

   The archive page is generated and placed in the output cache.

#. **Generate tag pages**

   Tag pages are generated and placed in the output cache.

#. **Generate feed and sitemap**

   The RSS feed and sitemap are generated and placed in the output cache.

#. **Copy raw content to output cache**

   Any site-specific :ref:`raw content` is copied to the output cache.

#. **Compress/minify CSS/JS**

   If the :attr:`~engineer.conf.EngineerConfiguration.COMPRESSOR_ENABLED` setting is on,
   then any CSS or JS files used in the site will be compressed.

   .. seealso:: :ref:`compression`

#. **Remove source LESS files from output cache**

   If the :attr:`~engineer.conf.EngineerConfiguration.PREPROCESS_LESS` setting is on,
   then the source LESS files will be removed from the output cache. This is safe to do since the LESS preprocessing
   happens during rendering, so at this point the corresponding CSS file has already been generated. The LESS source
   file is no longer needed.

#. **Synchronize output directory with output cache**

   Finally, the contents of the :attr:`~engineer.conf.EngineerConfiguration.OUTPUT_DIR` is synchronized with the
   output cache. This approach ensures that the actual site output is disturbed as little as possible. All the major
   copying/generating/rendering has already happened separately on the output cache,
   so the actual site output directory has only changes/additions/deletions propagated to it.


.. _raw content:

Raw Content
===========

Raw content is simply content that you want to include in your site as-is with no processing. Static content such as
JavaScript and CSS could also be considered raw content but those sorts of files don't need to live at a specific
place in your site's URL. The raw content feature is specifically for content where placement matters,
such as for robots.txt files and favicons. These files must be in the root of your site,
so just treating them like regular static content won't work.

Raw content should be placed in your site's :attr:`~engineer.conf.EngineerConfiguration.CONTENT_DIR`,
and since it's the last thing copied in the build pipeline, it will overwrite any content that was generated by the
other phases of the build pipeline. Keep this in mind.

The structure of :attr:`~engineer.conf.EngineerConfiguration.CONTENT_DIR` should match your site's. In other words,
if you want something to wind up in the root of your site, you would put it at the root of your
:attr:`~engineer.conf.EngineerConfiguration.CONTENT_DIR`. Similarly, if you want something to wind up in ``/foo/`` in
your site, you'd put it in a ``foo`` folder inside your :attr:`~engineer.conf.EngineerConfiguration.CONTENT_DIR`.


.. _compression:

CSS/JS Compression
==================

TODO


.. _LESS preprocessing:

LESS Preprocessing
==================

TODO
