Difference between revisions of "VPP/Build System Deep Dive"

From fd.io
< VPP
Jump to: navigation, search
m (→‎platforms.mk)
m
 
(4 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 +
__NOTOC__
  
−
__TOC__
+
== DEPRECATION NOTICE ==
 +
 
 +
This page is no longer maintained. Please see the [[https://fd.io/docs/vpp/master/gettingstarted/developers/buildsystem/index.html VPP Build System Documentation]]
  
 
== Introduction ==
 
== Introduction ==
  
−
The vpp build system comprises a data-driven main makefile, and a set of makefile fragments. All of hangs together nicely, the result of a set of well-thought-out conventions.  
+
The vpp build system consists of a data-driven main makefile, and a set of makefile fragments. The various parts come together as the result of a set of well-thought-out conventions.  
  
−
The vpp build system stitches together sets of git "repository groups." In a typical product development case, one uses two repository groups: an "open-repo" (including the build system itself) and a "closed-repo" comprising vendor-specific closed source. Each repository group comprises a number of independent git repositories.
+
==== Repository Groups and Source Paths ====
  
−
Current open-vpp workspaces comprise a single repo group, ordinarily called "vpp". Vpp/build-root/build-config.mk defines a key variable - SOURCE_PATH - which names the set of repo groups.
+
The vpp build system brings together sets of git "repository groups." In a typical product development case, one would use two repository groups: a group that we will call for the purpose of this explanation "open-repo" (including the build system itself) and a group that we will call "closed-repo" comprising vendor-specific closed source. Each repository group consists of a number of independent git repositories.
  
−
The vpp build system caters to components built with GNU autoconf / automake. Adding such components is a snap. Dealing with components which use BSD-style raw Makefiles is a bit of a pain. Dealing with toolchain components such as GCC and GLIBC can be seriously painful.
+
The current vpp workspaces comprise a single repository group, ordinarily called "vpp". The file <tt>vpp/build-root/build-config.mk</tt> defines a key variable called <code>SOURCE_PATH</code>. The <code>SOURCE_PATH</code> variable names the set of repository groups. We recommend that you examine that file to understand the repositories that are being built.
  
−
The vpp build system is a single-pass build system. A partial order must exist for any set of components: the set of (a before b) tuples must resolve to an ordered list. If you create a circular dependency of the form; (a,b) (b,c) (c,a), gmake will try to build the target list, but there’s a 0.0% chance that the results will be pleasant. Cut-n-paste mistakes in .../build-data/packages/<oops>.mk can produce the most confusing failures imaginable.
+
==== Dependencies and Components ====
  
−
In a single-pass build system, it’s best to separate libraries and applications which instantiate them. If vpp depends on libfoo.a, and myapp depends on both vpp and libfoo.a, place libfoo.a and myapp in separate components. The build system will build libfoo.a, vpp, and then myapp. If you try to build libfoo.a and myapp from the same component, it won’t work.
+
The vpp build system caters to components built with GNU autoconf / automake. Adding such components is a simple process. Dealing with components which use BSD-style raw Makefiles is a bit more difficult. Dealing with toolchain components such as GCC and GLIBC can be considerably more complicated.
  
−
If you absolutely, positively insist on having myapp and libfoo.a in the same source tree, you can create a pseudo-component in a separate .mk file in .../build-data/packages/. Code “phoneycomponent_source = realcomponent”, and provide manual configure/build/install targets.  
+
The vpp build system is a single-pass build system. A partial order must exist for any set of components: the set of (a before b) tuples must resolve to an ordered list. If you create a circular dependency of the form; (a,b) (b,c) (c,a), gmake will try to build the target list, but there’s a 0.0% chance that the results will be pleasant. Cut-n-paste mistakes in <tt>.../build-data/packages/<oops>.mk</tt> can produce confusing failures.  
  
−
Separate components for myapp, libfoo.a, and vpp are the gold-standard cheap / easy solution. The “mumble_source = realsource” degree of freedom exists to solve intractable circular dependencies: to build gcc-bootstrap, followed by glibc, followed by “real” gcc/g++ [which depend on glibc].
+
In a single-pass build system, it’s best to separate libraries and applications which instantiate them. For example, if vpp depends on <tt>libfoo.a</tt>, and myapp depends on both vpp and <tt>libfoo.a</tt>, it's best to place <tt>libfoo.a</tt> and myapp in separate components. The build system will build <tt>libfoo.a</tt>, vpp, and then (as a separate component) myapp. If you try to build <tt>libfoo.a</tt> and myapp from the same component, it won’t work.
 +
 
 +
If you absolutely, positively insist on having myapp and <tt>libfoo.a</tt> in the same source tree, you can create a pseudo-component in a separate <tt>.mk</tt> file in the <tt>.../build-data/packages/</tt> directory. Define the code <code>phoneycomponent_source = realcomponent</code>, and provide manual configure/build/install targets.
 +
 
 +
Separate components for myapp, <tt>libfoo.a</tt>, and vpp is the best and easiest solution. However, the “mumble_source = realsource” degree of freedom exists to solve intractable circular dependencies, such as: to build gcc-bootstrap, followed by glibc, followed by “real” gcc/g++ [which depends on glibc too].
  
 
== .../vpp/build-root ==
 
== .../vpp/build-root ==
  
−
.../vpp/build-root contains the repository group specification build-config.mk, the main Makefile, and the system-wide set of autoconf/automake variable overrides in config.site. We'll describe these files in some detail. To level-set: the main Makefile and config.site are subtle and complex. It's unlikely that you'll need or want to modify them. Ill-considered changes in either place typically cause difficult bugs.  
+
The .../vpp/build-root directory contains the repository group specification <tt>build-config.mk</tt>, the main Makefile, and the system-wide set of autoconf/automake variable overrides in <tt>config.site</tt>. We'll describe these files in some detail. To be clear about expectations, the main Makefile and <tt>config.site</tt> file are subtle and complex. It's unlikely that you'll need or want to modify them. Poorly planned changes in either place typically cause bugs that are difficult to solve.
  
 
=== .../vpp/build-root/build-config.mk ===
 
=== .../vpp/build-root/build-config.mk ===
  
−
As described above, build-config.mk is completely straightforward: it sets the make variable SOURCE_PATH to a list of repository group absolute paths. If you choose to move a workspace, you must modify it to match "reality 2.0." For example:
+
As described above, the <tt>build-config.mk</tt> file is straightforward: it sets the make variable <code>SOURCE_PATH</code> to a list of repository group absolute paths.  
 +
 
 +
==== The SOURCE_PATH variable ====
 +
If you choose to move a workspace, make sure to modify the paths defined by the <code>SOURCE_PATH</code> variable.  Those paths need to match changes you make in the workspace paths. For example, if you place the vpp directory in the workspace of a user named jsmith, you might need to change the <code>SOURCE_PATH</code> to:
  
−
  SOURCE_PATH = /home/joeuser/workspace/vpp  
+
  SOURCE_PATH = /home/jsmithuser/workspace/vpp
  
 
=== .../vpp/build-root/Makefile ===
 
=== .../vpp/build-root/Makefile ===
  
−
The main Makefile is complex. If you think you need to modify it, please ask for advice before you invest or send money. Seriously.  
+
The main Makefile is complex. If you think you need to modify it, it's a good idea to do some research, or ask for advice before you change it.  
  
−
Pleasant characteristics include: excellent performance, absolutely accurate dependency processing, ccache enablement, timestamp optimizations, git integration, extensibility, builds cross-compilation tool chains, builds embedded Linux distributions. If you need to do so, you can build double-cross tools: compile gdb on x86_64, to run on PowerPC, to debug the Xtensa instruction set.  
+
The main Makefile was organized and designed to provide the following characteristics: excellent performance, absolutely accurate dependency processing, cache enablement, timestamp optimizations, git integration, extensibility, builds with cross-compilation tool chains, and builds with embedded Linux distributions.  
  
−
Unpleasant characteristic: you'll break it if you look cross-eyed at it.
+
If you need to do so, you can build double-cross tools. For example, you could: compile gdb on x86_64, to run on PowerPC, to debug the Xtensa instruction set.  
  
−
==== PLATFORM variable ====
 
  
−
The PLATFORM make / environment variable controls a number of important image characteristics, primarily: cpu architecture, the list of images to build via the "make PLATFORM=xxx install-packages" target. See below for a description of that target. In the vpp case, we use "PLATFORM=vpp".
+
==== The PLATFORM variable ====
  
−
The main Makefile interprets $PLATFORM by attempting to "-include" <repo-group-root>/build-data/platforms.mk:
+
The <code>PLATFORM</code> make/environment variable controls a number of important characteristics, primarily:
 +
 
 +
* cpu architecture, and
 +
* the list of images to build.
 +
 
 +
The list of images to build is specified by the target: <code>make PLATFORM=xxx install-packages</code>. (See below for a description of that target.)
 +
 
 +
In the vpp case, we use <code>"PLATFORM=vpp"</code>.
 +
 
 +
The main Makefile interprets <code>$PLATFORM</code> by attempting to <code>-include</code> the file <tt><repo-group-root>/build-data/platforms.mk</tt>:
  
 
  $(foreach d,$(FULL_SOURCE_PATH), \
 
  $(foreach d,$(FULL_SOURCE_PATH), \
 
   $(eval -include $(d)/platforms.mk))
 
   $(eval -include $(d)/platforms.mk))
  
−
By convention, we don't define platforms in .../<repo-group-root>/build-data/platforms.mk. In the vpp case, we search for platform definition makefile fragments in .../vpp/build-data/platforms.mk, as follows:
+
By convention, we don't define platforms in the <tt>.../<repo-group-root>/build-data/platforms.mk</tt> file. In the vpp case, we search for platform definition makefile fragments in <tt>.../vpp/build-data/platforms.mk</tt>, as follows:
  
 
  $(foreach d,$(SOURCE_PATH_BUILD_DATA_DIRS), \
 
  $(foreach d,$(SOURCE_PATH_BUILD_DATA_DIRS), \
 
   $(eval -include $(d)/platforms/*.mk))
 
   $(eval -include $(d)/platforms/*.mk))
  
−
With vpp, which uses the "vpp" platform as discussed above, we end up "-include"ing .../vpp/build-data/platforms/vpp.mk. With a bunch of secondary "exercise for the reader" variable settings omitted, here is the contents of vpp.mk:
+
With vpp, which uses the "vpp" platform as discussed above, we end up "-include"-ing <tt>.../vpp/build-data/platforms/vpp.mk</tt>. To make it easier to read here in this page, the following snippet omits several variable settings. Here are some of the contents of <tt>vpp.mk</tt>:
  
 
  # vector packet processor
 
  # vector packet processor
Line 79: Line 97:
 
  -fstack-protector -fPIC -pie
 
  -fstack-protector -fPIC -pie
  
−
The variable <platform-name>_arch sets the CPU architecture used to build the per-platform cross-compilation toolchain. With the exception of the "native" architecture - used in our example - the vpp build system produces cross-compiled binaries.  
+
The variable <code><platform-name>_arch</code> sets the CPU architecture used to build the per-platform cross-compilation toolchain. With the exception of the "native" architecture - used in our example - the vpp build system produces cross-compiled binaries.  
  
−
The variable <platform-name>_native_tools lists the required set of self-compiled build tools.
+
The variable <code><platform-name>_native_tools</code> lists the required set of self-compiled build tools.
  
−
The variable <platform-name>_root_packages lists the set of images to build via the "make PLATFORM=<platform-name> TAG=<tag-name> [install-deb | install-rpm]" target.
+
The variable <code><platform-name>_root_packages</code> lists the set of images to build when specifying the target: <code>make PLATFORM=<platform-name> TAG=<tag-name> [install-deb | install-rpm]</code>.
  
−
==== TAG variable ====
 
  
−
The TAG variable indirectly sets CFLAGS and LDFLAGS, as well as the build and install directory names in .../vpp/build-root. Here is the current "vpp" tag definition from .../vpp/build-data/platforms/vpp.mk:
+
==== The TAG variable ====
  
−
Executables built with PLATFORM=vpp TAG=vpp are built in .../vpp/build-root/build-vpp-x86_64, with executables installed in .../vpp/build-root/install-vpp-x86_64
+
The TAG variable indirectly sets <tt>CFLAGS</tt> and <tt>LDFLAGS</tt>, as well as the build and install directory names in the <tt>.../vpp/build-root</tt> directory. Here is the current "vpp" tag definition specified in the  <tt>.../vpp/build-data/platforms/vpp.mk</tt> file:
  
−
==== Makefile important targets ====
+
vpp_debug_TAG_CFLAGS = -g -O0 -DCLIB_DEBUG -DFORTIFY_SOURCE=2 -march=$(MARCH) \
 +
  -fstack-protector-all -fPIC
 +
vpp_debug_TAG_LDFLAGS = -g -O0 -DCLIB_DEBUG -DFORTIFY_SOURCE=2 -march=$(MARCH) \ 
 +
-fstack-protector-all -fPIC
 +
 
 +
vpp_TAG_CFLAGS = -g -O2 -DFORTIFY_SOURCE=2 -march=$(MARCH) \
 +
-fstack-protector -fPIC -pie
 +
vpp_TAG_LDFLAGS = -g -O2 -DFORTIFY_SOURCE=2 -march=$(MARCH) \
 +
-fstack-protector -fPIC -pie
 +
 
 +
Executables built with <code>PLATFORM=vpp TAG=vpp</code> are built in <tt>.../vpp/build-root/build-vpp-x86_64</tt>, with executables installed in <tt>.../vpp/build-root/install-vpp-x86_64</tt>.
 +
 
 +
 
 +
==== Important targets in the Makefile ====
  
 
The main Makefile and the various makefile fragments implement the following user-invoked targets:
 
The main Makefile and the various makefile fragments implement the following user-invoked targets:
Line 160: Line 190:
 
=== config.site ===
 
=== config.site ===
  
−
.../vpp/build-root/config.site contains a set of per-component configuration variable overrides. If you look at e.g. ../vpp/build-tool-x86_64/glibc/config.log, you'll see something like this:
+
The <tt>.../vpp/build-root/config.site</tt> file contains a set of per-component configuration variable overrides. If you look at <tt>../vpp/build-tool-x86_64/glibc/config.log</tt> as an example, you'll see something like this:
  
 
  configure:2233: loading site script /scratch/nightly-builds/latest/vpp/build-root/config.site
 
  configure:2233: loading site script /scratch/nightly-builds/latest/vpp/build-root/config.site
Line 167: Line 197:
 
  | libc_cv_c_cleanup=yes
 
  | libc_cv_c_cleanup=yes
  
−
and so forth.
 
  
−
Settings in config.site keep specifc component "configure" scripts from trying to guess the setting of specific variables which they always screw up. In an ideal world, there would be no need to do this. People who love sausage and respect the law should never watch either one being made.
+
You can see in the above example that the <tt>.../vpp/build-root/config.site</tt> file defines values for certain variables that <tt>glibc</tt> might need. Without those values, <tt>glibc</tt> might assign or assume other default values.
 +
 
 +
The benefit of defining settings in <tt>config.site</tt> is that it keeps specifc component "configure" scripts from trying to guess the values of specific variables. Those guesses (or assumptions) made by component scripts often lead to unpredictable results. In an ideal world, there would be no need to do this. The <tt>config.site</tt> file provides a more predictable alternative than trusting that component scripts are going to make the correct assumptions.
  
 
== .../xxx/build-data ==
 
== .../xxx/build-data ==
Line 177: Line 208:
 
=== platforms.mk ===
 
=== platforms.mk ===
  
−
Each repo group includes platforms.mk, which are included by the main Makefile. vpp/build-data/platforms.mk is not interesting. closed-repo/build-data/platforms.mk accomplishes two tasks. First, it includes vpp/build-data/platforms/*.mk:
+
Each repo group includes the <tt>platforms.mk</tt> file, which is included by the main Makefile. The <tt>vpp/build-data/platforms.mk</tt> file is not complex. The <tt>closed-repo/build-data/platforms.mk</tt> file accomplishes two tasks.  
 +
 
 +
First, it includes <tt>vpp/build-data/platforms/*.mk</tt>:
  
 
  $(foreach d,$(SOURCE_PATH_BUILD_DATA_DIRS), \
 
  $(foreach d,$(SOURCE_PATH_BUILD_DATA_DIRS), \
 
   $(eval -include $(d)/platforms/*.mk))
 
   $(eval -include $(d)/platforms/*.mk))
  
−
This collects the set of platform definition makefile fragments, as discussed [[Build_system_internals#PLATFORM_variable|above]].
+
This collects the set of platform definition makefile fragments, as discussed [[VPP/Build_System_Deep_Dive#The_PLATFORM_variable|above]].
  
−
Second, platforms.mk implements a set of make targets. Certain of these targets could move to the main Makefile at some point in the indefinite future.
+
Second, the <tt>platforms.mk</tt> file implements a set of make targets.
  
 
=== packages/*.mk ===
 
=== packages/*.mk ===
  
−
Each component needs a makefile fragment e.g. so the build system recognizes it. The per-component makefile fragments vary considerably in complexity. For a component built with GNU autoconf / automake which does not depend on other components, the make fragment can be empty. See .../vpp/build-data/packages/vpp.mk for an uncomplicated but fully realistic example.  
+
Each component needs a makefile fragment in order for the build system to recognize it. The per-component makefile fragments vary considerably in complexity. For a component built with GNU autoconf / automake which does not depend on other components, the make fragment can be empty. See <tt>.../vpp/build-data/packages/vpp.mk</tt> for an uncomplicated but fully realistic example.  
  
−
Important variable settings in per-component makefile fragments:
+
Here are some of the important variable settings in per-component makefile fragments:
  
 
{| class="wikitable" border="1"
 
{| class="wikitable" border="1"
Line 205: Line 238:
 
|-
 
|-
 
| xxx_CPPFLAGS
 
| xxx_CPPFLAGS
−
| Adds -I stanzas to CPPFLAGS for components upon which xxx depends. Almost invariably "xxx_CPPFLAGS = $(call installed_includes_fn, dep1 dep2 dep3)", where dep1, dep2, and dep3 are listed in xxx_configure_depend. It is bad manners to set "-g -O3" here. Those settings belong in a [[Vpp Build_system_internals#TAG_variable|TAG]].
+
| Adds -I stanzas to CPPFLAGS for components upon which xxx depends. Almost invariably "xxx_CPPFLAGS = $(call installed_includes_fn, dep1 dep2 dep3)", where dep1, dep2, and dep3 are listed in xxx_configure_depend. It is bad practice to set "-g -O3" here. Those settings belong in a [[VPP/Build_System_Deep_Dive#The_TAG_variable|TAG]].
 
|-
 
|-
 
| xxx_LDFLAGS
 
| xxx_LDFLAGS
Line 211: Line 244:
 
|}
 
|}
  
−
When dealing with "irritating" components built with raw Makefiles which only work when building in the source trees, we use a specific strategy in xxx.mk. Examples include the Intel / 6wind DPDK. The strategy is simple: "OK, be that way. We simply copy the source tree into .../vpp/build-root/build-xxx." This works, but completely defeats dependency processing. This strategy is acceptable '''only''' for 3rd party software which won't need extensive (or preferably any) modifications.
 
  
−
Take a look at .../vpp/build-data/packages/dpdk.mk. When invoked, the dpdk_configure variable copies source code into $(PACKAGE_BUILD_DIR), and performs the BSD equivalent of "autoreconf -i -f" to configure the build area. The rest of the file is similar: a bunch of hand-rolled glue code which manages to make the dpdk act like a good vpp build citizen even though it is not. There are worse examples, notably the historical tail-f "confd" integration.
+
When dealing with "irritating" components built with raw Makefiles which only work when building in the source trees, we use a specific strategy in the <tt>xxx.mk</tt> file. Examples include the Intel / 6wind DPDK.
 +
 
 +
The strategy is simple for those "irritating" components: We copy the source tree into <tt>.../vpp/build-root/build-xxx</tt>. This works, but completely defeats dependency processing. This strategy is acceptable '''only''' for 3rd party software which won't need extensive (or preferably any) modifications.
 +
 
 +
Take a look at <tt>.../vpp/build-data/packages/dpdk.mk</tt>. When invoked, the <code>dpdk_configure</code> variable copies source code into <code>$(PACKAGE_BUILD_DIR)</code>, and performs the BSD equivalent of "autoreconf -i -f" to configure the build area. The rest of the file is similar: a bunch of hand-rolled glue code which manages to make the dpdk act like a good vpp build citizen even though it is not. There are worse examples, notably the historical tail-f "confd" integration.
  
 
== per-component GNU autotools input files ==
 
== per-component GNU autotools input files ==
  
−
This section is not an introduction to the GNU autotools. Folks unfamiliar with the GNU autotools have two choices: stick to programming-by-example, or spend time learning about autotools. A bit of both seems like the way to go. Google is your friend. Here's a pointer to the canonical [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/Autotools-Introduction.html introduction].
+
This section is not an introduction to the GNU autotools. If you are unfamiliar with the GNU autotools you can: stick to programming-by-example, or spend time learning about autotools. A bit of both seems the likely way to get started. There is a lot of information online from various sites on how to use GNU autotools. We recommend that you look at the canonical reference on gnu.org for autotools [http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/Autotools-Introduction.html introduction].
  
 
=== <repo>/<component>/configure.ac ===
 
=== <repo>/<component>/configure.ac ===
  
−
Configure.ac directs the GNU autotools to generate the "configure" script which the main Makefile invokes via its "xxx-configure" target.   
+
The <tt>configure.ac</tt> file directs the GNU autotools to generate the "configure" script which the main Makefile invokes through the <code>xxx-configure</code> target.   
  
−
The configure scripts receives the set of options specified in <repo>/build-data/packages/<component>/<component>.mk> via [[Vpp Build_system_internals#packages.2F.2A.mk|xxx_configure_args]], in addition to --libdir, --prefix, and --host (when cross-compiling).
+
The configure script receives the set of options specified in <tt><repo>/build-data/packages/<component>/<component>.mk></tt> as specified by the [[VPP/Build_System_Deep_Dive#packages.2F.2A.mk|xxx_configure_args]] argument.  When cross-compiling, this is in addition to the arguments <code>--libdir</code>, <code>--prefix</code>, and <code>--host</code>.
  
−
A minimal reasonable example:
+
Here is short but reasonable example:
  
 
  AC_INIT(foo, 1.1)
 
  AC_INIT(foo, 1.1)
Line 237: Line 273:
 
  AC_OUTPUT([Makefile])
 
  AC_OUTPUT([Makefile])
  
−
In rough terms, this says "please build a configure script for version 1.1 of the foo component. Make sure that you can find gcc, gas, gld. The component may build a dynamic library in addition to a command binary." For an uncomplicated new component, this is about all you need to specify.
+
In rough terms, this says "please build a configure script for version 1.1 of the <tt>foo</tt> component. Make sure that you can find gcc, gas, and gld. The component may build a dynamic library in addition to a command binary."  
 +
 
 +
For an uncomplicated new component, the above configuration is about all you need to specify.
  
−
See .../vpp/vpp/configure.ac for a more sophisticated example, which sets automake conditional variables (used to control the set of files compiled) and arranged to add some -Dfoo variable settings to CFLAGS.
+
For a more sophisticated example, look at the <tt>.../vpp/vpp/configure.ac</tt> file. The settings in that file set automake conditional variables (used to control the set of files compiled). It is also arranged to add some <code>-Dfoo</code> variable settings to CFLAGS.
  
−
Less is more. The more tools, libraries, etc. you list in configure.ac, the slower the resulting configure script will run. The vpp build systems is ccache-enabled, timestamp enabled, and tuned to the point where the build system rate-limiter is the amount of time spent running "configure" scripts.
+
In principle, less is more. The more tools, libraries, etc. that you list in the <tt>configure.ac</tt> file, the slower the resulting configure script will run. The vpp build systems is ccache-enabled, timestamp enabled, and tuned to the point where the build system rate-limiter is the amount of time spent running "configure" scripts.

Latest revision as of 22:28, 28 February 2020


DEPRECATION NOTICE

This page is no longer maintained. Please see the [VPP Build System Documentation]

Introduction

The vpp build system consists of a data-driven main makefile, and a set of makefile fragments. The various parts come together as the result of a set of well-thought-out conventions.

Repository Groups and Source Paths

The vpp build system brings together sets of git "repository groups." In a typical product development case, one would use two repository groups: a group that we will call for the purpose of this explanation "open-repo" (including the build system itself) and a group that we will call "closed-repo" comprising vendor-specific closed source. Each repository group consists of a number of independent git repositories.

The current vpp workspaces comprise a single repository group, ordinarily called "vpp". The file vpp/build-root/build-config.mk defines a key variable called SOURCE_PATH. The SOURCE_PATH variable names the set of repository groups. We recommend that you examine that file to understand the repositories that are being built.

Dependencies and Components

The vpp build system caters to components built with GNU autoconf / automake. Adding such components is a simple process. Dealing with components which use BSD-style raw Makefiles is a bit more difficult. Dealing with toolchain components such as GCC and GLIBC can be considerably more complicated.

The vpp build system is a single-pass build system. A partial order must exist for any set of components: the set of (a before b) tuples must resolve to an ordered list. If you create a circular dependency of the form; (a,b) (b,c) (c,a), gmake will try to build the target list, but there’s a 0.0% chance that the results will be pleasant. Cut-n-paste mistakes in .../build-data/packages/<oops>.mk can produce confusing failures.

In a single-pass build system, it’s best to separate libraries and applications which instantiate them. For example, if vpp depends on libfoo.a, and myapp depends on both vpp and libfoo.a, it's best to place libfoo.a and myapp in separate components. The build system will build libfoo.a, vpp, and then (as a separate component) myapp. If you try to build libfoo.a and myapp from the same component, it won’t work.

If you absolutely, positively insist on having myapp and libfoo.a in the same source tree, you can create a pseudo-component in a separate .mk file in the .../build-data/packages/ directory. Define the code phoneycomponent_source = realcomponent, and provide manual configure/build/install targets.

Separate components for myapp, libfoo.a, and vpp is the best and easiest solution. However, the “mumble_source = realsource” degree of freedom exists to solve intractable circular dependencies, such as: to build gcc-bootstrap, followed by glibc, followed by “real” gcc/g++ [which depends on glibc too].

.../vpp/build-root

The .../vpp/build-root directory contains the repository group specification build-config.mk, the main Makefile, and the system-wide set of autoconf/automake variable overrides in config.site. We'll describe these files in some detail. To be clear about expectations, the main Makefile and config.site file are subtle and complex. It's unlikely that you'll need or want to modify them. Poorly planned changes in either place typically cause bugs that are difficult to solve.

.../vpp/build-root/build-config.mk

As described above, the build-config.mk file is straightforward: it sets the make variable SOURCE_PATH to a list of repository group absolute paths.

The SOURCE_PATH variable

If you choose to move a workspace, make sure to modify the paths defined by the SOURCE_PATH variable. Those paths need to match changes you make in the workspace paths. For example, if you place the vpp directory in the workspace of a user named jsmith, you might need to change the SOURCE_PATH to: 

SOURCE_PATH = /home/jsmithuser/workspace/vpp

.../vpp/build-root/Makefile

The main Makefile is complex. If you think you need to modify it, it's a good idea to do some research, or ask for advice before you change it.

The main Makefile was organized and designed to provide the following characteristics: excellent performance, absolutely accurate dependency processing, cache enablement, timestamp optimizations, git integration, extensibility, builds with cross-compilation tool chains, and builds with embedded Linux distributions.

If you need to do so, you can build double-cross tools. For example, you could: compile gdb on x86_64, to run on PowerPC, to debug the Xtensa instruction set.


The PLATFORM variable

The PLATFORM make/environment variable controls a number of important characteristics, primarily:

  • cpu architecture, and
  • the list of images to build.

The list of images to build is specified by the target: make PLATFORM=xxx install-packages. (See below for a description of that target.)

In the vpp case, we use "PLATFORM=vpp".

The main Makefile interprets $PLATFORM by attempting to -include the file <repo-group-root>/build-data/platforms.mk: 

$(foreach d,$(FULL_SOURCE_PATH), \
  $(eval -include $(d)/platforms.mk))

By convention, we don't define platforms in the .../<repo-group-root>/build-data/platforms.mk file. In the vpp case, we search for platform definition makefile fragments in .../vpp/build-data/platforms.mk, as follows: 

$(foreach d,$(SOURCE_PATH_BUILD_DATA_DIRS),	\
  $(eval -include $(d)/platforms/*.mk))

With vpp, which uses the "vpp" platform as discussed above, we end up "-include"-ing .../vpp/build-data/platforms/vpp.mk. To make it easier to read here in this page, the following snippet omits several variable settings. Here are some of the contents of vpp.mk: 

# vector packet processor
vpp_arch = native
vpp_native_tools = vppapigen vppversion
 
vpp_root_packages = vpp vlib vlib-api vnet svm dpdk vpp-api-test \ 
	vpp-japi 
 
vpp_configure_args_vpp = --with-dpdk
vnet_configure_args_vpp = --with-dpdk

# Set these parameters carefully. The vlib_buffer_t is 128 bytes, i.e.
# dpdk_headroom = uiotarball_headroom = vlib_pre_data + 128
dpdk_configure_args_vpp = --with-headroom=256
vlib_configure_args_vpp = --with-pre-data=128

vpp_debug_TAG_CFLAGS = -g -O0 -DCLIB_DEBUG -DFORTIFY_SOURCE=2 -march=$(MARCH) \
  	-fstack-protector-all -fPIC
vpp_debug_TAG_LDFLAGS = -g -O0 -DCLIB_DEBUG -DFORTIFY_SOURCE=2 -march=$(MARCH) \  
	-fstack-protector-all -fPIC
  
vpp_TAG_CFLAGS = -g -O2 -DFORTIFY_SOURCE=2 -march=$(MARCH) \ 
	-fstack-protector -fPIC -pie
vpp_TAG_LDFLAGS = -g -O2 -DFORTIFY_SOURCE=2 -march=$(MARCH) \ 
	-fstack-protector -fPIC -pie

The variable <platform-name>_arch sets the CPU architecture used to build the per-platform cross-compilation toolchain. With the exception of the "native" architecture - used in our example - the vpp build system produces cross-compiled binaries.

The variable <platform-name>_native_tools lists the required set of self-compiled build tools.

The variable <platform-name>_root_packages lists the set of images to build when specifying the target: make PLATFORM=<platform-name> TAG=<tag-name> [install-deb | install-rpm].


The TAG variable

The TAG variable indirectly sets CFLAGS and LDFLAGS, as well as the build and install directory names in the .../vpp/build-root directory. Here is the current "vpp" tag definition specified in the .../vpp/build-data/platforms/vpp.mk file: 

vpp_debug_TAG_CFLAGS = -g -O0 -DCLIB_DEBUG -DFORTIFY_SOURCE=2 -march=$(MARCH) \
  	-fstack-protector-all -fPIC
vpp_debug_TAG_LDFLAGS = -g -O0 -DCLIB_DEBUG -DFORTIFY_SOURCE=2 -march=$(MARCH) \  
	-fstack-protector-all -fPIC
  
vpp_TAG_CFLAGS = -g -O2 -DFORTIFY_SOURCE=2 -march=$(MARCH) \ 
	-fstack-protector -fPIC -pie
vpp_TAG_LDFLAGS = -g -O2 -DFORTIFY_SOURCE=2 -march=$(MARCH) \ 
	-fstack-protector -fPIC -pie

Executables built with PLATFORM=vpp TAG=vpp are built in .../vpp/build-root/build-vpp-x86_64, with executables installed in .../vpp/build-root/install-vpp-x86_64.


Important targets in the Makefile

The main Makefile and the various makefile fragments implement the following user-invoked targets:

Target Environment Variable Settings Notes
bootstrap-tools none Builds the set of native tools needed by the vpp build system to build tool chains and images. Examples include "make", "git", "find", and "tar."
install-tools PLATFORM Builds the tool chain for the indicated <platform>. Often fails once in the middle of building glibc, simply issue "make PLATFORM=xxx install-tools" again. Complaints to the glibc maintainers, effectively "cat complaint > /dev/null".
distclean none Roto-rooters everything in sight: toolchains, images, and so forth. After issuing "make distclean", you'll need to rebuild the bootstrap-tools and the indicated platform toolchain.
wipe-all PLATFORM=xxx [TAG=<tag>] Recursively deletes .../vpp/build-root/build-<tag>-<cpu-arch> and .../vpp/build-root/install-<tag>-<cpu-arch>. The gold standard in recompiling all components listed in the <platform>_root_packages list.
<component>-find-source none Pull a source tree for <component> from the git server. Scan the list of repository groups, looking for .../build-data/packages/<component>.mk to decide which repo group will have the sources for the indicated <component>. Grubs around in .../vpp/build-root/.git/config to decide which git server / git protocol to use.
<component>-pull-all none Update <component> and all components upon which it depends from the git server.
<component>-install PLATFORM and TAG Build and install <component>, and all components upon which <component> depends. Example: to build a vanilla vpp forwarder, use "make PLATFORM=vpp TAG=debug vpp-install". Note the vpp-specific debug and non-debug tags defined in .../vpp/build-data/platforms/vpp.mk.
install-packages PLATFORM and TAG Build and install all components listed in <platform>_root_packages, use compile / link options defined by TAG. Example: to build the production vpp forwarder packages, use "make PLATFORM=vpp TAG=vpp install-packages"
install-deb PLATFORM and TAG Build a Debian package comprising components listed in <platform>_root_packages, using compile / link options defined by TAG. This is a .PHONY target. Issue "make ... install-packages" first to avoid accidentally building a brain-dead debian package.

Additional Makefile environment variables

Variable Setting Notes
is_build_tool=yes Directs Makefile to look in <repo-group-root>/build-root/packages/<component>.mk for instructions to build the indicated <component>. Results in .../vpp/build-root/{build,install}-tools. Related to setting TAG=xxx. Example: "make is_build_tool=yes apigen-install".
BUILD_DEBUG=vx Directs Makefile et al. to make a good-faith effort to show what's going on in excruciating detail. Use it as follows: "make ... BUILD_DEBUG=vx". Fairly effective in Makefile debug situations.
strip_symbols=yes Strip symbols from generated binaries prior to installation and packaging. Typically used with the "install-packages" and "install-deb" targets.
sign_executables=yes Appends public-key encrypted SHA256 signatures to ELF binaries. Used when building embedded Linux systems.

config.site

The .../vpp/build-root/config.site file contains a set of per-component configuration variable overrides. If you look at ../vpp/build-tool-x86_64/glibc/config.log as an example, you'll see something like this: 

configure:2233: loading site script /scratch/nightly-builds/latest/vpp/build-root/config.site
| # glibc needs this for cross compiling 
| libc_cv_forced_unwind=yes
| libc_cv_c_cleanup=yes


You can see in the above example that the .../vpp/build-root/config.site file defines values for certain variables that glibc might need. Without those values, glibc might assign or assume other default values.

The benefit of defining settings in config.site is that it keeps specifc component "configure" scripts from trying to guess the values of specific variables. Those guesses (or assumptions) made by component scripts often lead to unpredictable results. In an ideal world, there would be no need to do this. The config.site file provides a more predictable alternative than trusting that component scripts are going to make the correct assumptions.

.../xxx/build-data

The set of build-data directories contains detailed build and packaging data for components which are defined in each repo group.

platforms.mk

Each repo group includes the platforms.mk file, which is included by the main Makefile. The vpp/build-data/platforms.mk file is not complex. The closed-repo/build-data/platforms.mk file accomplishes two tasks.

First, it includes vpp/build-data/platforms/*.mk: 

$(foreach d,$(SOURCE_PATH_BUILD_DATA_DIRS),	\
  $(eval -include $(d)/platforms/*.mk))

This collects the set of platform definition makefile fragments, as discussed above.

Second, the platforms.mk file implements a set of make targets.

packages/*.mk

Each component needs a makefile fragment in order for the build system to recognize it. The per-component makefile fragments vary considerably in complexity. For a component built with GNU autoconf / automake which does not depend on other components, the make fragment can be empty. See .../vpp/build-data/packages/vpp.mk for an uncomplicated but fully realistic example.

Here are some of the important variable settings in per-component makefile fragments:

Variable Notes
xxx_configure_depend Lists the set of component build dependencies for the xxx component. In plain English: don't try to configure this component until you've successfully built the indicated targets. Almost always, xxx_configure_depend will list a set of "yyy-install" targets. Note the pattern: "variable names contain underscores, make target names contain hyphens"
xxx_configure_args

(optional)

Lists any additional arguments to pass to the xxx component "configure" script. The main Makefile %-configure rule adds the required settings for --libdir, --prefix, and --host (when cross-compiling, aka most of the time)
xxx_CPPFLAGS Adds -I stanzas to CPPFLAGS for components upon which xxx depends. Almost invariably "xxx_CPPFLAGS = $(call installed_includes_fn, dep1 dep2 dep3)", where dep1, dep2, and dep3 are listed in xxx_configure_depend. It is bad practice to set "-g -O3" here. Those settings belong in a TAG.
xxx_LDFLAGS Adds -Wl,-rpath -Wl,depN stanzas to LDFLAGS for components upon which xxx depends. Almost invariably "xxx_CPPFLAGS = $(call installed_lib_fn, dep1 dep2 dep3)", where dep1, dep2, and dep3 are listed in xxx_configure_depend. It is bad manners to set "-liberty-or-death" here. Those settings belong in Makefile.am.


When dealing with "irritating" components built with raw Makefiles which only work when building in the source trees, we use a specific strategy in the xxx.mk file. Examples include the Intel / 6wind DPDK.

The strategy is simple for those "irritating" components: We copy the source tree into .../vpp/build-root/build-xxx. This works, but completely defeats dependency processing. This strategy is acceptable only for 3rd party software which won't need extensive (or preferably any) modifications.

Take a look at .../vpp/build-data/packages/dpdk.mk. When invoked, the dpdk_configure variable copies source code into $(PACKAGE_BUILD_DIR), and performs the BSD equivalent of "autoreconf -i -f" to configure the build area. The rest of the file is similar: a bunch of hand-rolled glue code which manages to make the dpdk act like a good vpp build citizen even though it is not. There are worse examples, notably the historical tail-f "confd" integration.

per-component GNU autotools input files

This section is not an introduction to the GNU autotools. If you are unfamiliar with the GNU autotools you can: stick to programming-by-example, or spend time learning about autotools. A bit of both seems the likely way to get started. There is a lot of information online from various sites on how to use GNU autotools. We recommend that you look at the canonical reference on gnu.org for autotools introduction.

<repo>/<component>/configure.ac

The configure.ac file directs the GNU autotools to generate the "configure" script which the main Makefile invokes through the xxx-configure target.

The configure script receives the set of options specified in <repo>/build-data/packages/<component>/<component>.mk> as specified by the xxx_configure_args argument. When cross-compiling, this is in addition to the arguments --libdir, --prefix, and --host.

Here is short but reasonable example: 

AC_INIT(foo, 1.1)
AM_INIT_AUTOMAKE

AC_PROG_LIBTOOL
AM_PROG_AS
AC_PROG_CC
AM_PROG_CC_C_O 

AC_OUTPUT([Makefile])

In rough terms, this says "please build a configure script for version 1.1 of the foo component. Make sure that you can find gcc, gas, and gld. The component may build a dynamic library in addition to a command binary."

For an uncomplicated new component, the above configuration is about all you need to specify.

For a more sophisticated example, look at the .../vpp/vpp/configure.ac file. The settings in that file set automake conditional variables (used to control the set of files compiled). It is also arranged to add some -Dfoo variable settings to CFLAGS.

In principle, less is more. The more tools, libraries, etc. that you list in the configure.ac file, the slower the resulting configure script will run. The vpp build systems is ccache-enabled, timestamp enabled, and tuned to the point where the build system rate-limiter is the amount of time spent running "configure" scripts.

Retrieved from "https://wiki.fd.io/index.php?title=VPP/Build_System_Deep_Dive&oldid=10426"