Chapter 4. The Build Process

Typically configuration header files are #include'd only by the package's source code at build time, or by a package's exported header files if the interface provided by the package may be affected by a configuration option. There should be no need for application code to know the details of individual configuration options, instead the configuration should specifically meet the needs of the application.

There are always exceptions. Application code may want to adapt to configuration options, for example to do different things for ROM and RAM booting systems, or when it is necessary to support several different target boards. This is especially true if the code in question is really re-usable library code which has not been converted to an eCos package, and hence cannot use any CDL facilities.

A major problem here is determining which packages are in the configuration: attempting to #include a header file such as pkgconf/net.h when it is not known for certain that that particular package is part of the configuration will result in compilation errors. The global header file pkgconf/system.h serves to provide such information, so application code can use techniques like the following:

#include <pkgconf/system.h>
#ifdef CYGPKG_NET
# include <pkgconf/net.h>
#endif

This will compile correctly irrespective of the eCos configuration, and subsequent code can use #ifdef or similar directives on CYGPKG_NET or any of the configuration options in that package.

In addition to determining whether or not a package is present, the global configuration header file can also be used to find out the specific version of a package that is being used. This can be useful if a more recent version exports additional functionality. It may also be necessary to adapt to incompatible changes in the exported interface or to changes in behaviour. For each package the configuration system will typically #define three symbols, for example for a V1.3.1 release:

#define CYGNUM_NET_VERSION_MAJOR 1
#define CYGNUM_NET_VERSION_MINOR 3
#define CYGNUM_NET_VERSION_RELEASE 1

There are a number of problems associated with such version #define's. The first restriction is that the package must follow the standard naming conventions, so the package name must be of the form xxxPKG_yyy. The three characters immediately preceding the first underscore must be PKG, and will be replaced with NUM when generating the version #define's. If a package does not follow the naming convention then no version #define's will be generated.

Assuming the package does follow the naming conventions, the configuration tools will always generate three version #define's for the major, minor, and release numbers. The symbol names are obtained from the package name by replacing PKG with NUM and appending _VERSION_MAJOR, _VERSION_MINOR and _VERSION_RELEASE. It is assumed that the resulting symbols will not clash with any configuration option names. The values for the #define's are determined by searching the version string for sequences of digits, optionally preceded by a minus sign. It is possible that some or all of the numbers are absent in any given version string, in which case -1 will be used in the #define. For example, given a version string of V1.12beta, the major version number is 1, the minor number is 12, and the release number is -1. Given a version string of beta all three numbers would be set to -1.

There is special case code for the version current, which typically corresponds to a development version obtained via anonymous CVS or similar means. The configuration system has special built-in knowledge of this version, and will assume it is more recent than any specific release number. The global configuration header defines a special symbol CYGNUM_VERSION_CURRENT, and this will be used as the major version number when version current of a package is used:

#define CYGNUM_VERSION_CURRENT 0x7fffff00
...
#define CYGNUM_INFRA_VERSION_MAJOR CYGNUM_VERSION_CURRENT
#define CYGNUM_INFRA_VERSION_MINOR -1
#define CYGNUM_INFRA_VERSION_RELEASE -1

The large number used for CYGNUM_VERSION_CURRENT should ensure that major version comparisons work as expected, while still allowing for a small amount of arithmetic in case that proves useful.

It should be noted that this implementation of version #define's will not cope with all version number schemes. However for many cases it should suffice.

Some build systems may involve a phase before the header files get exported, to update the build and install trees automatically when there has been a change to the configuration savefile ecos.ecc. This is useful mainly for application developers using the command line tools: it would allow users to create the build tree only once, and after any subsequent configuration changes the tree would be updated automatically by the build system. The facility would be analogous to the --enable-maintainer-mode option provide by the autoconf and automake programs. At present no eCos build system implements this functionality, but it is likely to be added in a future release.

The first compulsory phase involves making sure that there is an up to date set of header files in the install tree. Each package can contain some number of header files defining the exported interface. Applications should only use exported functionality. A package can also contain some number of private header files which are only of interest to the implementation, and which should not be visible to application code. The various packages that go into a particular configuration can be spread all over the component repository. In theory it might be possible to make all the exported header files accessible by having a lengthy -I header file search path, but this would be inconvenient both for building eCos and for building applications. Instead all the relevant header files are copied to a single location, the include subdirectory of the install tree. The process involves the following:

Note

At present there is no defined support in the build system for defining custom build steps that generate exported header files. Any attempt to use the existing custom build step support may fall foul of unexpected header files being deleted automatically by the build system. This limitation will be addressed in a future release of the component framework, and may require changing the priority for exporting header files so that a custom build step can happen first.

Once there are up to date copies of all the exported header files in the build tree, the main build can proceed. Most of this involves compiling source files listed in compile properties in the CDL scripts for the various packages, for example:

cdl_package CYGPKG_ERROR {
  display       "Common error code support"
  compile       strerror.cxx
  …
}

compile properties may appear in the body of a cdl_package, cdl_component, cdl_option or cdl_interface. If the option or other CDL entity is active and enabled, the property takes effect. If the option is inactive or disabled the property is ignored. It is possible for a compile property to list multiple source files, and it is also possible for a given CDL entity to contain multiple compile properties. The following three examples are equivalent:

cdl_option <some_option> {
  …
  compile file1.c file2.c file3.c
}
cdl_option <some_option> {
  …
  compile file1.c
  compile file2.c
  compile file3.c
}
cdl_option <some_option> {
  …
  compile file1.c file2.c
  compile file3.c
}

Packages that follow the directory layout conventions should have a subdirectory src, and the component framework will first look for the specified files there. Failing that it will look for the specified files relative to the package's root directory. For example if a package contains a source file strerror.cxx then the following two lines are equivalent:

compile strerror.cxx
compile src/strerror.cxx

In the first case the component framework will find the file immediately in the packages src subdirectory. In the second case the framework will first look for a file src/src/strerror.cxx, and then for str/strerror.cxx relative to the package's root directory. The result is the same.

The file names may be relative paths, allowing the source code to be split over multiple directories. For example if a package contains a file src/sync/mutex.cxx then the corresponding CDL entry would be:

compile sync/mutex.cxx

All the source files relevant to the current configuration will be identified when the build tree is generated or updated, and added to the appropriate makefile (or its equivalent for other build systems). The actual build will involve a rule of the form:

<object file> : <source file>
        $(CC) -c $(INCLUDE_PATH) $(CFLAGS) -o $@ $<

The component framework has built-in knowledge for processing source files written in C, C++ or assembler. These should have a .c, .cxx and .S suffix respectively. The current implementation has no simple mechanism for extending this with support for other languages or for alternative suffixes, but this should be addressed in a future release.

The compiler command that will be used is something like arm-eabi-gcc. This consists of a command prefix, in this case arm-eabi, and a specific command such as gcc. The command prefix will depend on the target architecture and is controlled by a configuration option in the appropriate HAL package. It will have a sensible default value for the current architecture, but users can modify this option when necessary. The command prefix cannot be changed on a per-package basis, since it is usually essential that all packages are built with a consistent set of tools.

The $(INCLUDE_PATH) header file search path consists of at least the following:

The compiler flags $(CFLAGS) are determined in two steps. First the appropriate HAL package will provide a configuration option defining the global flags. Typically this includes flags that are needed for the target processor, for example -mcpu=arm9, various flags related to warnings, debugging and optimization, and flags such as -finit-priority which are needed by eCos itself. Users can modify the global flags option as required. In addition it is possible for existing flags to be removed from and new flags to be added to the current set on a per-package basis, again by means of user-modifiable configuration options. More details are given below.

Component writers can assume that the build system will perform full header file dependency analysis, including dependencies on configuration headers, but the exact means by which this happens is implementation-defined. Typical application developers are unlikely to modify exported or private header files, but configuration headers are likely to change as the configuration is changed to better meet the needs of the application. Full header file dependency analysis also makes things easier for the component writers themselves.

The current directory used during a compilation is an implementation detail of the build system. However it can be assumed that each package will have its own directory somewhere in the build tree, to prevent file name clashes, that this will be the current directory, and that intermediate object files will end up here.

Once all the compile and make_object properties have been processed and the required object files have been built or rebuilt, these can be collected together in one or more libraries. The archiver will be the ar command corresponding to the current architecture, for example powerpc-eabi-ar. By default al of the object files will end up in a single library libtarget.a. This can be changed on a per-package basis using the library property in the body of the corresponding cdl_package command, for example:

cdl_package <SOME_PACKAGE> {
  …
  library  libSomePackage.a
}

However using different libraries for each package should be avoided. It makes things more difficult for application developers since they now have to link the application code with more libraries, and possibly even change this set of libraries when packages are added to or removed from the configuration. The use of a single library libtarget.a avoids any complications.

It is also possible to change the target library for individual files, using a -library option with the correspondingcompile or make_object property. For example:

compile -library=libSomePackage.a hello.c
make_object -library=libSomePackage.a {
  …
}

Again this should be avoided because it makes application development more difficult. There is one special library which can be used freely, libextras.a, which is used to generate the extras.o file as described below.

The order in which object files end up in a library is not defined. Typically each library will be created directly in the install tree, since there is little point in generating a file in the build tree and then immediately copying it to the install tree.

Package sources files normally get compiled and then added to a library, by default libtarget.a, which is then linked with the application code. Because of the usual rules for linking with libraries, augmented by the use of link-time garbage collection, this means that code will only end up in the final executable if there is a direct or indirect reference to it in the application. Usually this is the desired behaviour: if the application does not make any use of say kernel message boxes, directly or indirectly, then that code should not end up in the final executable taking up valuable memory space.

In a few cases it is desirable for package code to end up in the final executable even if there are no direct or indirect references. For example, device driver functions are often not called directly. Instead the application will access the device via the string "/dev/xyzzy" and call the device functions indirectly. This will be impossible if the functions have been removed at link-time.

Another example involves static C++ objects. It is possible to have a static C++ object, preferably with a suitable constructor priority, where all of the interesting work happens as a side effect of running the constructor. For example a package might include a monitoring thread or a garbage collection thread created from inside such a constructor. Without a reference by the application to the static object the latter will never get linked in, and the package will not function as expected.

A third example would be copyright messages. A package vendor may want to insist that all products shipped using that package include a particular message in memory, even though many users of that package will object to such a restriction.

To meet requirements such as these the build system provides support for a file extras.o, which always gets linked with the application code via the linker script. Because it is an object file rather than a library everything in the file will be linked in. The extras.o file is generated at the end of a build from a library libextras.a, so packages can put functions and variables in suitable source files and add them to that library explicitly:

compile -library=libextras.a xyzzy.c
compile xyzzy_support.c

In this example xyzzy.o will end up in libextras.a, and hence in extras.o and in the final executable. xyzzy_support.o will end up in libtarget.a as usual, and is subject to linker garbage collection.

	Caution
	Some of the details of compiler selection and compiler flags described below are subject to change in future revisions of the component framework, although every reasonable attempt will be made to avoid breaking backwards compatibility.

The build system needs to know what compiler to use, what compiler flags should be used for different stages of the build and so on. Much of this information will vary from target to target, although users should be able to override this when appropriate. There may also be a need for some packages to modify the compiler flags. All platform HAL packages should define a number of options with well-known names, along the following lines (any existing platform HAL package can be consulted for a complete example):

cdl_component CYGBLD_GLOBAL_OPTIONS {
  flavor  none
  parent  CYGPKG_NONE
  …
  cdl_option CYGBLD_GLOBAL_COMMAND_PREFIX {
    flavor  data
    default_value { "arm-eabi" }
    …
  }
  cdl_option CYGBLD_GLOBAL_CFLAGS {
    flavor  data
    default_value "-Wall -g -O2 …"
    …
  }
  cdl_option CYGBLD_GLOBAL_LDFLAGS {
    flavor  data
    default_value "-g -nostdlib -Wl,--gc-sections …"
    …
  }
}

The CYGBLD_GLOBAL_OPTIONS component serves to collect together all global build-related options. It has the flavor none since disabling all of these options would make it impossible to build anything and hence is not useful. It is parented immediately below the root of the configuration hierarchy, thus making sure that it is readily accessible in the graphical configuration tool and, for command line users, in the ecos.ecc save file.

	Note
	Currently the parent property lists a parent of CYGPKG_NONE, rather than an empty string. This could be unfortunate if there was ever a package with that name. The issue will be addressed in a future release of the component framework.

The option CYGBLD_GLOBAL_COMMAND_PREFIX defines which tools should be used for the current target. Typically this is determined by the processor on the target hardware. In some cases a given target board may be able to support several different processors, in which case the default_value expression could select a different toolchain depending on some other option that is used to control which particular processor. CYGBLD_GLOBAL_COMMAND_PREFIX is modifiable rather than calculated, so users can override this when necessary.

Given a command prefix such as arm-eabi, all C source files will be compiled with arm-eabi-gcc, all C++ sources will be built using arm-eabi-g++, and arm-eabi-ar will be used to generate the library. This is in accordance with the usual naming conventions for GNU cross-compilers and similar tools. For the purposes of custom build steps, tokens such as $(CC) will be set to arm-eabi-gcc.

The next option, CYGBLD_GLOBAL_CFLAGS, is used to provide the initial value of $(CFLAGS). Some compiler flags such as -Wall and -g are likely to be used on all targets. Other flags such as -mcpu=arm7tdmi will be target-specific. Again this is a modifiable option, so the user can switch from say -O2 to -Os if desired. The option CYGBLD_GLOBAL_LDFLAGS serves the same purpose for $(LDFLAGS) and linking. It is used primarily when building test cases or possibly for some custom build steps, since building eCos itself generally involves building one or more libraries rather than executables.

Some packages may wish to add certain flags to the global set, or possibly remove some flags. This can be achieved by having appropriately named options in the package, for example:

cdl_component CYGPKG_KERNEL_OPTIONS {
  display "Kernel build options"
  flavor  none
  …
  cdl_option CYGPKG_KERNEL_CFLAGS_ADD {
    display "Additional compiler flags"
    flavor  data
    default_value { "" }
    …
  }
  cdl_option CYGPKG_KERNEL_CFLAGS_REMOVE {
    display "Suppressed compiler flags"
    flavor  data
    default_value { "" }
    …
  }
  cdl_option CYGPKG_KERNEL_LDFLAGS_ADD {
    display "Additional linker flags"
    flavor  data
    default_value { "" }
    …
  }
  cdl_option CYGPKG_KERNEL_LDFLAGS_REMOVE {
    display "Suppressed linker flags"
    flavor  data
    default_value { "" }
    …
  }
}

In this example the kernel does not modify the global compiler flags by default, but it is possible for the users to modify the options if desired. The value of $(CFLAGS) that is used for the compilations and custom build steps in a given package is determined as follows:

$(LDFLAGS) is determined in much the same way.

Note

The way compiler flags are handled at present has numerous limitations that need to be addressed in a future release, although it should suffice for nearly all cases. For the time being custom build steps and in particular the make_object property can be used to work around the limitations. Amongst the issues, there is a specific problem with package encapsulation. For example the math library imposes some stringent requirements on the compiler in order to guarantee exact IEEE behavior, and may need special flags on a per-architecture basis. One way of handling this is to have CYGPKG_LIBM_CFLAGS_ADD and CYGPKG_LIBM_CFLAGS_REMOVE default_value expressions which depend on the target architecture, but such expressions may have to updated for each new architecture. An alternative approach would allow the architectural HAL package to modify the default_value expressions for the math library, but this breaks encapsulation. A third approach would allow some architectural HAL packages to define one or more special options with well-known names, and the math library could check if these options were defined and adjust the default values appropriately. Other packages with floating point requirements could do the same. This approach also has scalability issues, in particular how many such categories of options would be needed? It is not yet clear how best to resolve such issues.

Note

When generating a build tree it would be desirable for the component framework to output details of the tools and compiler flags in a format that can be re-used for application builds, for example a makefile fragment. This would make it easier for application developers to use the same set of flags as were used for building eCos itself, thus avoiding some potential problems with incompatible compiler flags.

Caution

Some of the details of custom build steps as described below are subject to change in future revisions of the component framework, although every reasonable attempt will be made to avoid breaking backwards compatibility. The first line in the make and make_object blocks introduced below defines the make target and its dependencies with the remaining lines specifying the commands. These commands will always be tab-indented in the resulting makefile and as a result all GNUmake commands, such as ifeq(...), will be illegal as these cannot be indented. It is possible to work around this CDL deficiency in some instances using make's shell syntax support. For an example, see the eCos custom build step used to create the mk_romfs command within the CYGPKG_FS_ROM package located in $ECOS_REPOSITORY/packages/fs/rom/<version>/cdl/romfs.cdl.

For most packages simply listing one or more source files in a compile property is sufficient. These files will get built using the appropriate compiler and compiler flags and added to a library, which then gets linked with application code. A package that can be built in this way is likely to be more portable to different targets and build environments, since it avoids build-time dependencies. However some packages have special needs, and the component framework supports custom build steps to allow for these needs. There are two properties related to this, make and make_object, and both take the following form:

make {
  <target_filepath> : <dependency_filepath> …
    <command>
    ...
}

Although this may look like makefile syntax, and although some build environments will indeed involve generating makefiles and running make, this is not guaranteed. It is possible for the component framework to be integrated with some other build system, and custom build steps should be written with that possibility in mind. Each custom build step involves a target, some number of dependency files, and some number of commands. If the target is not up to date with respect to one or more of the dependencies then the commands need to be executed.

By default custom build steps defined in a make_object property have a priority of 100, which means that they will be executed in the same phase as compilations resulting from a compile property. It is possible to change the priority using a property option, for example:

make_object -priority 50 {
  …
}

Specifying a priority smaller than a 100 means that the custom build step happens before the normal compilations. Priorities between 100 and 200 happen after normal compilations but before the libraries are archived together. make_object properties should not specify a priority of 200 or later.

Custom build steps defined in a make property have a default priority of 300, and so they will happen after the libraries have been built. Again this can be changed using a -priority property option.

Linking an application requires the application code, a linker script, the eCos library or libraries, the extras.o file, and some startup code. Depending on the target hardware and how the application gets booted, this startup code may do little more than branching to main(), or it may have to perform a considerable amount of hardware initialization. The startup code generally lives in a file vectors.o which is created by a custom build step in a HAL package. As far as application developers are concered the existence of this file is largely transparent, since the linker script ensures that the file is part of the final executable.

This startup code is not generally of interest to component writers, only to HAL developers who are referred to one of the existing HAL packages for specific details. Other packages are not expected to modify the startup in any way. If a package needs some work performed early on during system initialization, before the application's main entry point gets invoked, this can be achieved using a static object with a suitable constructor priority.

	Note
	It is possible that the extras.o support, in conjunction with appropriate linker script directives, could be used to eliminate the need for a special startup file. The details are not yet clear.

	Caution
	This section is not finished, and the details are subject to change in a future release. Arguably linker script issues should be documented in the HAL documentation rather than in this guide.

Generating the linker script is the responsibility of the various HAL packages that are applicable to a given target. Developers of components other than HAL packages need not be concerned about what is involved. Developers of new HAL packages should use an existing HAL as a template.

	Note
	It may be desirable for some packages to have some control over the linker script, for example to add extra alignment details for a particular section. This can be risky because it can result in subtle portability problems, and the current component framework has no support for any such operations. The issue may be addressed in a future release.