Developper reference for building and packaging =============================================== Building a distribution ----------------------- Building a distribution consists of the building of both the compiler & the different engines. The distribution must meet the following goals: * contains only the code we want to distribute (*ie.* no source code) * requires as less installation steps as possible for the user * ability to keep track (version) of distributed files The distribution scripts involve several steps: * compute a distribution version number and inject it in all following steps * compile the compiler: it uses ``ant``/``ivy`` * compile the different engines (*eg.* reference engine, optimized engine) * provide install/setup script * package compiler & engines in archives to be distributed All these steps are scheduled from the script called ``wrap.sh`` located, as everything related to the distribution, in the ``distribution`` directory. .. IMPORTANT:: Please be aware that all steps needed in the release process are **not** automated in scripts. You still need to do manual steps (see below) in order to build a complete release. Failing to do so will lead to incoherence and most probably headaches for solving problems. Invoking ``wrap.sh`` script ^^^^^^^^^^^^^^^^^^^^^^^^^^^ The wrap script accepts several command line parameters:: -r When building a revision to release :) -v Give the version name instead of it being generated -p Build profile for engine (default: Release) -s Skip directly to engine building, no compiler -h This help If the ``-r`` flag is used, then the version used throughout the build uses the pattern **YYYY.MM**. It has the nice property of being easy to compute, to understand and is always increasing. The ``-v`` allows the specification of the full version string. It has priority over ``-r``. If none of ``-v`` and ``-r`` are used, then the pattern used is **YYYY.MM.HHmmSS-DEV**. The ``-s`` can be used to skip the compiler compilation, which is responsible for most of the build time. The remaining steps are still executed. The last parameter, ``-p``, is used to set the cmake build profile used when building the engines. Default is ``Release``, meaning that the code may be optimized and debugging symbols stripped. If you need to build a debugging release, use the ``Debug`` profile. Please note that this profile is only used for building the engines, it has *nothing* to do with the build of the code that may be generated by the bip compiler later. What ``wrap.sh`` does ^^^^^^^^^^^^^^^^^^^^^ It starts by cleaning everything : its own ``build`` directory along with the separate build directories for the difference parts of the compiler. And immediately starts the building of all parts of the compilers. It does that by using ``ant`` with the following targets: * ``clean``: to remove previous build artefacts * ``publish-local-all``: build the compiler The ``publish-local-all`` target is a frontend target that uses ``ivy``. After the compiler has been successfully built, the ``wrap`` script continues by copying the resulting artefacts (jar files) and the frontend script that will be used by the users (*ie.* ``bipc.sh``). At this point, the bip compiler is ready. The ``wrap`` script now compiles the different engines found in the ``Engines/`` directory. The result of the engine compilation is directly a self-contained archive. What ``wrap.sh`` produces ^^^^^^^^^^^^^^^^^^^^^^^^^ After everything has been executed, you can find the distribution in the ``build/`` directory:: build +── bipc-2012.04.110853-DEV.FILES +── bipc_2012.04.110853-DEV.tar.gz +── BIP-optimized-engine-2012.04.110853-DEV_Linux-i686.tar.gz `── BIP-reference-engine-2012.04.110853-DEV_Linux-i686.tar.gz The ``.FILES`` file should be kept as it contains all the filenames included in the compiler distribution archive. It is useful, as the compiler contains several external dependencies whose versions are not encoded in the distribution version. Single archive distribution using ``single-archive-dist.sh`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For even easier distribution, the ``single-archive-dist.sh`` script can be used. It produces a single archive with the compiler and the engines inside. Installation is only a matter of extracting and running a script that sets up the environment correctly. It relies on ``wrap.sh`` and simply rearrange the products of the latter. The script accepts only ``-r`` and ``-v`` command line parameters, which are exactly the same as for ``wrap.sh``. The result of running this script is a single archive called ``bip-full_.tar.gz``. It contains the compiler and the engines. It also has a ``setup.sh`` script that can be used to setup the environment correctly. By default, the script configures the environment for using the reference-engine, but you can give it any engine (provided it is shipped in the archive) name:: $ . ./setup.sh optimized-engine Environment configured for engine: optimized-engine $ . ./setup.sh Environment configured for engine: reference-engine Do not forget the leading ``.`` in the command above. Publishing a distribution ------------------------- Publishing a distribution must always includes the following steps: 1. **compute a new version name**. You must never reuse a previous version name. Never! If you have published a distribution with a huge bug inside, it's already too late, you need a new version. Commonly, a version has a *major* part and a *minor* or *revision* part. The major corresponds to a new release, and the minor can be changed when you want to distribute a new revision for the same release (*eg.* with bug fixes). 2. **build the compiler** using the previously computed version name 3. **build the engines** using the previously computed version name 4. **build the document**. The documentation should always match the distributed software. Never provide outdated documentation. If you can't update the documentation (it's a shame), outdated part must be explicitly marked. 5. **tag** all the different parts of the software, script, documents in the SCM (*ie.* subversion). 6. **publish** ! The scripts presented in previous paragraphs can help for steps 1,2,3. Steps 4,5 and 6 must be done carefully by hand. Manual steps ^^^^^^^^^^^^ Building the documentation """""""""""""""""""""""""" Building the documentation is simple. 2 documentations can be built and published: APIs & user/dev documentations. As of this writing, only the user/dev documentation is published though, as APIs still need some work (they are based on javadoc & doxygen). The user/dev documentation are using `Sphinx `_. First, you need to configure the documentation build to include the correct version number and release name. Edit the file ``source/conf.py`` to include the matching version/name:: # The short X.Y version. version = '2012.04' # The full version, including alpha/beta/rc tags. release = '2012.04 (RC3)' Building is as easy as running:: $ make html latexpdf The targets are self-described and produce static HTML pages and a PDF. The script ``sync-to-www.sh`` provides an easy way to build **and** publish the user/dev documentations. The script also creates a tree-hugger-friendly PDF with 2 pages per side and publishes the example files. Tagging """"""" You must tag **all** parts (compiler, engines, documentation, distribution scripts), even the ones that have not moved since the previous release. use the ``tag`` command:: $ svn tag version-name Of course, replace ``version-name`` by the release version name. You must repeat this operation for **all** svn module that need to be tagged. Publishing """""""""" Publishing means copying files in the web directory. The ``sync-to-www.sh`` moves the documentations in the ``doc/`` subdirectory:: /doc +── examples +── html `── pdf Compiler & engines releases must be copied by hand. Things to keep in mind ---------------------- Unless you know exactly what you are doing (and why!), you should: * **never** commit ``u.v.b.userinterface.cli/src/main/java/u/v/b/userinterface/cli/Version.java`` : it is modified when building a distribution but these modifications should never make their way into the code repository. * **never** commit any ``.project``, ``.classpath`` or any other eclipse dot-files. Having local modifications is also often a sign of wrong configuration (but not always). Be careful to never commit these as this will break other developer setup. * **never** commit ``Documents/sphinx-doc/source/conf.py`` if you've only changed the version/release information.