Advanced builder capabilities
-----------------------------

.. _development mode:

Development mode
~~~~~~~~~~~~~~~~

Development mode provides a way for a user to request to process packages
against development versions of sources rather than using fixed versions.
A package will typically target a stable release, either pointing to a
specific archive to download or a specific tag to clone from. However, for
some builds, a user may wish to build a specific package against their main
development branch (e.g. the ``main`` branch of a Git repository) or a
long-term stable release branch. Packages can be defined to target these
specific revisions if running in development mode.

To enable development mode, invoking ``releng-tool`` with the ``--development``
argument will enable the mode. Future calls to releng-tool for the project
will use a development revision for packages where appropriate. For example:

.. code-block:: shell-session

   $ releng-tool --development [<mode>]
   (success) configured root for development mode
   $ releng-tool
   ~building against development sources~
   ...

Development mode is persisted through the use of a file flag in the root
directory.

Consider the following example: a package defines multiple revisions to
fetch sources from:

.. code-block:: python

   LIBFOO_SITE = 'https://example.com/libfoo.git'
   LIBFOO_REVISION = {
       DEFAULT_REVISION: '1.2',
       'develop': 'main',
       'lts': '1.1.x',
   }

A build would normally use the ``1.2`` tag for this package. However, if
an environment is configured to use the ``develop`` development mode:

.. code-block:: shell-session

   $ releng-tool --development develop

This package would use the ``main`` branch instead.

Projects can also target specific sites based off the development mode. This
can be useful if a package uses a built archive for a stable release, but
having development sources fetched from a repository. For example:

.. code-block:: python

   LIBFOO_SITE = {
       DEFAULT_SITE: 'https://pkgs.example.com/releases/libfoo-${LIBFOO_VERSION}.tar.gz',
       'test': 'https://git.example.com/libfoo.git',
   }

   LIBFOO_REVISION = {
       'test': 'main',
   }

In a normal execution, a tar.gz archive would be downloaded for the package.
However, if an environment is configured to use the ``test`` development
mode, sources will be fetched from the Git repository on the ``main`` branch.

Simple development modes are also supported. Packages can use the
``LIBFOO_DEVMODE_REVISION`` option to hint at a development revision to pull.

.. code-block:: python

   LIBFOO_DEVMODE_REVISION = 'main'
   LIBFOO_REVISION = 'v3.0'

A build would normally use the ``v3.0`` tag for this package. However, if
an environment is configured a non-explicit development mode:

.. code-block:: shell-session

   $ releng-tool --development

This package would use the ``main`` branch instead.

A user can either disable development mode by performing a
|ACTION_MRPROPER|_ or can manually remove the file flag.

.. _local-sources mode:

Local-sources mode
~~~~~~~~~~~~~~~~~~

.. note::

   Clean events (such as ``releng-tool clean``) will not touch packages using
   sources found alongside the output directory

Local-sources mode provides a way for a developer to build internal-flagged
packages using sources found alongside the root directory (or a specific
provided directory), instead of having releng-tool attempt to fetch them
from remote instances. This is primarily for developers who desire to
manually manage source content outside the releng-tool environment.
Local-sources mode only works for internally flagged packaged. Consider
the following example: a releng-tool project has a package called
``liba``. When releng-tool is invoked in normal configurations, the
package will do fetching, extraction and patching to prepare the directory
``<root>/output/build/liba-<version>``. However, if a builder has
configured the working root for local-sources mode, sources for ``liba``
will be used from the folder ``<root>/../liba`` instead.

When in local-sources mode, an internal package will skip the fetching,
extraction and patching stages in order to prevent undesired manipulation
of developer-prepared sources. Another consideration to note is the use
of clean operators while in local-sources mode. Continuing with the above
example, if a user invokes ``releng-tool liba-clean``, the operation will
not remove the ``<root>/../liba`` folder. Responsibility to managing a
clean ``liba`` package will be left with the user.

To enable local-sources mode, invoking ``releng-tool`` with the
``--local-sources`` argument will enable the mode. Future calls to releng-tool
for the project will use local sources for packages defined as internal
packages. For example:

.. code-block:: shell-session

   $ releng-tool --local-sources
   (*) <parent>
   (success) configured root for local-sources mode
   $ releng-tool
   ~building against local sources~
   ...

Local-sources mode is persisted through the use of a file flag in the root
directory.

If a user provides a directory for the ``--local-sources`` argument, packages
will be looked for in the provided folder instead of the parent of the
configured root directory. For example:

.. code-block:: shell-session

   $ releng-tool --local-sources ~/workdir
   (*) ~/workdir
   (success) configured root for local-sources mode
   $ releng-tool
   ~building against local sources~
   ...

In the above example, if a project had an internal package ``liba``,
sources for ``liba`` will be used from the folder ``~/workdir/liba``.

Users can also provide package-specific overrides. If a user provides a
path which is prefixed with a package's name and at sign (``@``), the
sources for the provided package will be used from the respective folder:

.. code-block:: shell-session

   $ releng-tool -L ~/workdir
   $ releng-tool -L libb@/mnt/sources/libb
   $ releng-tool -L libc@
   (*) ~/workdir
   (libb) /mnt/sources/libb
   (libc) <unset>
   (success) configured root for local-sources mode
   $ releng-tool
   ~building against local sources~
   ...

In the above example, if a project had internal packages ``liba``,
``libb`` and ``libc``, the following paths will be used:

- Sources for ``liba`` will be used from the folder ``~/workdir/liba``,
- Sources for ``libb`` will be used from the folder ``/mnt/sources/libb``; and,
- Sources for ``libc`` will not be fetched locally.

A user can either disable local sources mode by performing a
|ACTION_MRPROPER|_ or can manually remove the file flag.

.. _conf_overrides:

Configuration overrides
~~~~~~~~~~~~~~~~~~~~~~~

If a builder needs to (for example) override a tool location or package site, a
user can define either environment options or setup a configuration override
script ``releng-overrides``. It is never recommended to persist a configuration
overrides file into a project's source repository.

Extraction tool overrides
^^^^^^^^^^^^^^^^^^^^^^^^^

The ``override_extract_tools`` option inside a configuration override script
allows a dictionary to be provided to map an extension type to an external tool
to indicate which tool should be used for extraction. For example, when a
``.zip`` archive is being processed for extraction, releng-tool will internally
extract the archive; however, a user may wish to override this tool with their
own extraction utility. Consider the following example:

.. code-block:: python

   override_extract_tools = {
       'zip': '/opt/my-custom-unzip {file} {dir}',
   }

The ``{file}`` key will be replaced with the file to be extracted, and the
``{dir}`` key will be replaced where the contents should extract to.

Revision overrides
^^^^^^^^^^^^^^^^^^

The ``override_revisions`` option inside a configuration override script allows
a dictionary to be provided to map a package name to a new revision value.
Consider the following example: a project defines ``module-a`` and ``module-b``
packages with package ``module-b`` depending on package ``module-a``. A
developer may be attempting to tweak package ``module-b`` on the fly to test a
new capabilities against the current stable version of ``module-a``; however,
the developer does not want to explicitly change the revision inside package
``module-b``'s definition. To avoid this, an override can be used instead:

.. code-block:: python

   override_revisions = {
       'module-b': '<test-branch>',
   }

The above example shows that package ``module-b`` will fetch using a test branch
instead of what is defined in the actual package definition.

Site overrides
^^^^^^^^^^^^^^

The ``override_sites`` option inside a configuration override script allows a
dictionary to be provided to map a package name to a new site value. There may
be times where a host may not have access to a specific package site. To have a
host to use a mirror location without having to adjust the package definition,
the site override option can be used. For example, consider a package pulls from
site ``git@example.com:myproject.git``; however, the host ``example.com`` cannot
be access from the host machine. If a mirror location has been setup at
``git@example.org:myproject.git``, the following override can be used:

.. code-block:: python

   override_sites = {
       '<package-name>': 'git@example.org:myproject.git',
   }

Tool overrides
^^^^^^^^^^^^^^

Environment variables can be used to help override external tool invoked by the
releng-tool process. For example, when invoking CMake-based projects, the tool
``cmake`` will be invoked; however, if a builder is running on CentOS and CMake
v3.x is desired, the tool ``cmake3`` needs to be invoked instead. To configure
this, an environment variable can be set to switch which tool to invoke.
Consider the following example:

.. code-block:: shell-session

   $ export RELENG_CMAKE=cmake3
   $ releng-tool
   [cmake3 will be used for cmake projects]

Quirks
^^^^^^

releng-tool also provides a series of configuration quirks to deal with rare
host environment scenarios where releng-tool may be experiencing issues. See
:ref:`configuration quirks <quirks>` for more information.
