rscons/user_guide.md at b5d5fe7a7b1f5bba1003c47c9582dba2776c9ab4

josh/rscons

Fork 0

Josh Holtrop b5d5fe7a7b Add 'sh' script DSL method - close #142

2022-01-17 15:57:34 -05:00

47 KiB

Raw Blame History

#> Overview

Rscons is an open-source build system for developers. It supports the following features:

multi-threaded job execution
auto-configuration
built-in builders for several common operations
out-of-the-box support for C, C++, and D languages
extensibility for other languages or custom builders
compatible with Windows, Linux, OS X, and FreeBSD
colorized output with build progress
build hooks

At its core, Rscons is mainly an engine to:

determine the proper order to perform build operations,
determine whether each build target is up to date or in need of rebuild, and
schedule those build operations across multiple threads as efficiently as possible.

Along the way, Rscons provides a concise syntax for specifying common types of build operations, but also provides an extensible framework for performing custom build operations as well.

Rscons is written in Ruby, and is inspired by SCons and waf.

${remove} WARNING: This user guide is meant to be preprocessed and rendered by a custom script. The markdown source file is not intended to be viewed directly and will not include all intended content. Visit https://holtrop.github.io/rscons/index.html to view the properly rendered version. ${/remove}

Design Principles

Build Correctness

The number one design principle in Rscons is build correctness. This means that a build operation will be performed when Rscons cannot determine that a build target is already up-to-date. A build target will be built whenever:

the target file has been removed or changed since it was last built
the command to build the target file is different from the previous command used to build it
any of the target file's dependency files have changed since the last time the target was built

Importantly, Rscons uses the content of a source (dependency) file to determine whether a rebuild is necessary, not simply the timestamp of the file. This is because relying solely on the timestamp of the file can lead to an incorrect decision being made to not rebuild when a rebuild is necessary.

Build Flexibility

Rscons supports multiple configurations of compilation flags or build options across multiple environments to build output files in different ways according to the user's desire. For example, the same source files can be built into a release executable, but also compiled with different compilation flags or build options into a test executable. Rscons also supports build hooks, which allow the user to further fine-tune the build system's operation. A build hook, for example, can be used to set a build option for only source files coming from a particular source directory.

Build Efficiency

Rscons will automatically determine the number of threads to use based on the host CPU configuration, and will schedule jobs as efficiently as possible across the available threads in order to complete the build operation in as little time as possible. As development occurs and build operations are executed, Rscons makes use of a cache file in order to avoid rebuilding a target when it is already up to date.

Build Directory

Rscons was designed to store temporary build artifacts (for example, object files, dependency files, etc...) and build system metadata in a "build directory". This keeps files generated by the build cleanly separated from user-controlled source files.

In contrast to other build systems or build system generators, rscons executes from the project base directory (up to the user) rather than executing from within the build directory. This keeps any file paths printed by compilers (such as in warning or error messages) accurate relative to the project directory, so that the user does not need to translate any paths to the correct path within a terminal or editor application, for example.

By default a build directory named "build" is used, but this can be overridden by the user by using the -b/--build command-line option.

Getting Started

To use Rscons on your project, you must:

Install the rscons script in your project (See ${#Installation}).
Write the Rsconscript build script for your project (See ${#The Build Script}).
Use the rscons command in your project (See ${#Command-Line Operation}).

#> Installation

Rscons is designed to be distributed as a stand-alone single file script that can be copied into and versioned in a project's source tree. The only requirement to run Rscons is that the system has a Ruby interpreter installed. The latest release can be downloaded from https://github.com/holtrop/rscons/releases. Simply copy the rscons executable script into the desired location within the project to be built (typically the root of the repository) and mark it executable.

Version Control Setup

The following files should be added to source control:

rscons
Rsconscript

Add the following contents to .gitignore (or the equivalent thereof for different version control systems):

/.rscons*
/build/

#> Command-Line Operation

Rscons is typically invoked from the command-line as ./rscons. Rscons supports several build operations:

configure
build
clean
distclean
install
uninstall

##> Configure Operation

The configure operation will initialize the Rscons cache file and build directory. It will also perform any configuration checks requested by the build script. Such configuration checks can include:

verifying operation of a compiler
loading compilation/linker flags from a config program (e.g. pkg-config)
verifying presence of a C/C++ header file
verifying presence of a D import
verifying presence of a library
verifying presence of an executable
any custom user-supplied configuration check

##> Build Operation

If a build operation is requested and a configure operation has not yet been performed, a configure operation will be automatically invoked.

The build operation will execute all builders registered to produce build targets.

If a build operation fails (e.g. due to a compilation failure), Rscons will log the failed commands. By default Rscons does not print the failed commands to the console so that it is easier for the user to focus on the actual compiler failure messages rather than the compilation command itself. However, if the user wishes to see the compilation commands, rscons can be invoked with the -v command-line option to show all complilation commands while building, or, alternatively, following a compilation failure, the user can invoke rscons with the -F option which will not rebuild but will show the failed command log from the previous build operation.

##> Clean Operation

A clean operation will remove all built target files. It will not remove items installed by an install operation. It will not remove the cached configuration options.

##> Distclean Operation

A distclean operation will remove all built target files and all cached configuration options. Generally it will get the project directory back to the state it was in when unpacked before any configuration or build operations took place. It will not removed items installed by an install operation.

##> Install Operation

An install operation will perform a build (and if necessary, first a configure as well). In addition it will execute any Install or InstallDirectory builders to install items into the specified install directory.

##> Uninstall Operation

An uninstall operation will remove any items installed by an install operation. It will not remove all built target files, just the installed copies.

#> The Build Script

Rscons looks for instructions for what to build by reading a build script file called Rsconscript (or Rsconscript.rb). Here is a simple example Rsconscript file:

build do
  Environment.new do |env|
    env.Program("myprog.exe", glob("src/**/*.c"))
  end
end

This Rsconscript file would instruct Rscons to produce a Program target called myprog.exe which is to be built from all C source files found (recursively) under the src directory.

The Rsconscript file is a Ruby script.

##> Build Script Methods

rscons provides several methods that a build script can use.

glob (see ${#Finding Files: The glob Method})
path_append (see ${#PATH Management})
path_components (see ${#PATH Management})
path_prepend (see ${#PATH Management})
path_set (see ${#PATH Management})
rscons (see ${#Using Subsidiary Build Scripts: The rscons Method})
sh (see (${#Executing Commands: The sh Method})

###> Finding Files: The glob Method

The glob method can be used to find files matching the patterns specified. It supports a syntax similar to the Ruby Dir.glob method but operates more deterministically.

Example use:

build do
  Environment.new do |env|
    env.Program("mytests", glob("src/**/*.cc", "test/**/*.cc"))
  end
end

This example would build the mytests executable from all .cc source files found recursively under the src or test directory.

###> PATH Management

rscons provides methods for management of the PATH environment variable.

The path_append and path_prepend methods can be used to append or prepend a path to the PATH environment variable.

path_prepend "i686-elf-gcc/bin"

The path_set method sets the PATH environment variable to the given Array or String.

The path_components method returns an Array of the components in the PATH environment variable.

###> Using Subsidiary Build Scripts: The rscons Method

The rscons build script method can be used to invoke an rscons subprocess to perform an operation using a subsidiary rscons build script. This can be used, for example, when a subproject is imported and a top-level configure or build operation should also perform the same operation in the subproject directory.

The first argument to the rscons method specifies either a directory name, or the path to the subsidiary Rsconscript file to execute. Any additional arguments are passed to rscons when it executes the subsidiary script. rscons will change working directories to the directory containing the subsidiary script when executing it.

For example:

configure do
  rscons "subproject", "configure"
end

build do
  rscons "subproject/Rsconscript", "build"
end

It is also perfectly valid to perform a different operation in the subsidiary script from the one being performed in the top-level script. For example, in a project that requires a particular cross compiler, the top-level configure script could build the necessary cross compiler using a subsidiary build script. This could look something like:

configure do
  rscons "cross/Rsconscript"
  check_c_compiler "i686-elf-gcc"
end

This would build, and if necessary first configure, using the cross/Rsconscript subsidiary build script. Subsidiary build scripts are executed from within the directory containing the build script.

###> Executing Commands: The sh Method

The sh build script method can be used to directly execute commands. The sh method accepts either a single String argument or an Array of Strings. When an Array is given, if the array length is greater than 1, then the command will not be executed and interpreted by the system shell. Otherwise, it will be executed and interpreted by the system shell.

For example:

build do
  # Run "make" in imported "subcomponent" directory.
  sh "cd subcomponent; make"
  # Move a file around.
  sh "mv", "subcomponent/file with spaces.txt", "new_name.txt"
end

If the command fails, rscons will normally print the error and terminate execution. If the :continue option is set, then rscons will not terminate execution. For example:

build do
  # This command will fail and a message will be printed.
  sh "false", continue: true
  # However, due to the :continue option being set, execution will continue.
  sh "echo hi"
end

##> Configuration Operations

A configure block is optional. It can be used to perform various checks and setup operations for a project. Example configure block:

configure do
  check_cxx_compiler
  check_c_header "getopt.h"
end

###> Checking for a Compiler

The following methods can be used within a configure block to check for a working compiler:

check_c_compiler
check_cxx_compiler
check_d_compiler

Each of these methods can take an optional list of compilers to check for. If such a list is supplied, the compilers are tested in the order listed. The first compiler option found which passes a compilation test is used.

Here are example calls which also show the default compiler list for each supported language:

configure do
  check_c_compiler "gcc", "clang"
  check_cxx_compiler "g++", "clang++"
  check_d_compiler "gdc", "ldc2"
end

Global configuration options may be supplied to the compiler checks as well. Example:

configure do
  check_c_compiler "x86_64-elf-gcc", on_fail: "Install x86_64-elf cross toolchain first!"
end

###> Checking for a Header File

The following methods can be used to check for the presence of a header file:

check_c_header will check for a C header to be present
check_cxx_header will check for a C++ header to be present

Each of these methods take the name of the header file to check for as the first argument, and take an optional Hash of arguments as the second argument.

Example calls:

configure do
  check_c_header "getopt.h", set_define: "HAVE_GETOPT_H"
  check_c_header "FreeType2.h"
  check_cxx_header "memory"
end

Options

`:check_cpppath`

Optionally specifies an array of paths to look for the header file in.

###> Checking for a D Import

The check_d_import method can be used to check for the presence of D import.

This method takes the name of the import to check for as the first argument.

Example calls:

configure do
  check_d_import "std.stdio"
  check_d_import "std.numeric"
end

Options

`:check_d_import_path`

Optionally specifies an array of paths to look for the module in.

###> Checking for a Library

The check_lib method can be used to check for the presence of a library.

This method takes the name of the library to check for as the first argument, and take an optional Hash of arguments as the second argument.

Example calls:

configure do
  check_lib "kpty", fail: false, set_define: "HAVE_LIBKPTY"
  check_lib "GL"
end

Options

`:check_libpath`

Optionally specifies an array of paths to look for the library in.

`:use`

If not set, the library will be used by default in all Environment objects. If set, the library will only be used in Environment objects that have a matching :use flag set.

###> Checking for a Program

The check_program method can check for the existence of an executable in the host operating system environment.

Example call:

configure do
  check_program "xxd"
end

###> Checking for a Package Configuration

The check_cfg method can be used to check for the existence of a package as well as import any build options (e.g. include path, defines, libraries to link against, etc...) required to use the package.

This method takes a Hash of options as its only argument.

Example calls:

configure do
  check_cfg package: "zlib"
  check_cfg program: "freetype-config", fail: false, set_define: "HAVE_FREETYPE"
end

Options

`:package`

If the :package option is set to a value, the pkg-config program will be used to look for package configuration flags for the specified package.

`:program`

If the :program option is given, the program specified will be used to look for configuration flags.

`:use`

If not set, the library will be used by default in all Environment objects. If set, the library will only be used in Environment objects that have a matching :use flag set.

###> Custom Configuration Checks

The Rsconscript author can add custom configuration checks to be performed during the rscons configure operation.

Here is an example from build_tests/configure/custom_config_check.rb showing a custom configuration check:

${include build_tests/configure/custom_config_check.rb}

A custom configuration check is created by calling the custom_check method and passing a block. The contents of the block should perform the custom configuration checking logic. This logic can include executing a test command or other arbitrary operations. An argument op is passed to the block. This object is an instance of the ConfigureOp class class and provides several methods that can be used to aid with the custom configuration check. The log_and_test_command method can be used to execute a test command and retrieve its results. The command and its output are also logged to the config.log file. The store_merge, store_append, and store_parse methods can be used to store construction variables for Environments created during the build operation. Finally, the complete method can be used to complete the configuration check and indicate a success or failure.

While performing a custom configuration check, it can sometimes be useful to be able to construct an Environment to use the set of default construction variables as defined so far in the configuration block, for example to expand construction variables to build a test command. The normal Environment class cannot be used within the configure block, however the BasicEnvironment class can be used for such a purpose.

For example, to expand the current ${CCCMD} value:

configure do
  custom_check("Checking something to do with CCCMD") do
    command = BasicEnvironment.new.expand_varref("${CCCMD}")
    # ...
  end
end

###> Global Configuration Check Options

`:fail`

If the :fail option is set to false, then the absence of the package or program requested will not result in the configure option failing. The :fail option defaults to true if the :set_define option is not defined, and defaults to false if the :set_define option is defined.

`:on_fail`

The :on_fail option can be set to a String or a Proc object. If the configuration operation fails (or would fail), the given message is printed or the Proc is called.

Examples:

configure do
  check_c_compiler "special-gcc", on_fail: "First install special gcc!"
end

configure do
  package_hint = lambda do
    puts "The following packages must be installed to build this project:"
    puts "- libsdl2-dev"
    puts "- libsdl2-image-dev"
    puts "- libsdl2-net-dev"
  end
  check_lib "SDL2", on_fail: package_hint
  check_lib "SDL2_image", on_fail: package_hint
  check_lib "SDL2_net", on_fail: package_hint
end

`:set_define`

If set, a build define of the specified String will be added to the CPPDEFINES construction variable array if the requested package is found.

##> Build Operations

The build block is used to create Environments and register build targets. An Rscons build script would not be very useful without a build block.

Here is an example build block demonstrating how to register a build target:

build do
  Environment.new do |env|
    env.Program("myprog.exe", glob("src/**/*.c"))
  end
end

This Rsconscript would build an executable called myprog.exe from all C source files found recursively under the src directory.

###> Environments

An Environment includes:

a name
a collection of construction variables
a collection of build hooks
a collection of user-registered build targets
a build root

All build targets must be registered within an Environment. If the user does not specify a name for the environment, a name will be automatically generated based on the Environment's internal ID, for example "e.1". The Environment's build root is a directory created within the top-level Rscons build directory. It is based on the Environment name. By default it holds all intermediate files generated by Rscons that are needed to produce a user-specified build target. For example, for the Rsconscript:

build do
  Environment.new(name: "myproj") do |env|
    env.Program("myprog.exe", glob("src/**/*.c"))
  end
end

Rscons will place an object file and dependency file corresponding to each C source file under the Environment's build root. Assuming a top-level build directory of "build", the Environment's build root would be "build/myproj". This keeps the intermediate generated build artifacts separate from the source files.

###> Construction Variables

Construction variables are values assigned to keys within an Environment. Construction variables are used by Builders to produce output files. See ${#Default Construction Variables} for a reference of all built-in construction variables.

Example:

build do
  Environment.new do |env|
    env["CCFLAGS"] += %w[-O2 -Wall]
    env["LIBS"] += %w[m]
  end
end

This example modifies the CCFLAGS construction variable to add -O2 and -Wall to the compilation commands used for C and C++ source files. It also instructs the linker to link against the m library.

Construction Variable Naming

uppercase strings - the default construction variables that Rscons uses
strings beginning with "_" - set and used internally by builders
symbols, lowercase strings - reserved as user-defined construction variables

###> Builders

Rscons uses builder objects to produce target output files from source input files. A build target to be built using a builder is registered by calling a method on the Environment object that matches the builder's name. For example, a Program build target is registered by calling the env.Program method.

The general syntax for registering a build target using a builder is:

env.BuilderName(target, sources, vars = {})

The target parameter is the path to the output file or directory. The sources parameter is the path or paths to the input file(s) to be used by the builder. In the target and sources parameters, the user can explicitly refer to a path within the Environment's build root by beginning the path with "^/". The vars parameter is an optional Hash which can include construction variables to be used for this build target. Any construction variable values specified in this parameter will override those assigned to the Environment.

There are several default builders that are built-in to Rscons:

Command, which executes a user-defined command to produce the target.
Copy, which copies files or directories to a specified destination.
CFile, which builds a C or C++ source file from a lex or yacc input file.
Directory, which creates a directory.
Disassemble, which disassembles an object file to a disassembly listing.
Install, which installs files or directories to a specified destination.
InstallDirectory, which creates a directory during an install operation.
Library, which collects object files into a static library archive file.
Object, which compiles source files to produce an object file.
Preprocess, which invokes the C/C++ preprocessor on a source file.
Program, which links object files to produce an executable.
SharedLibrary, which links object files to produce a dynamically loadable library.
SharedObject, which compiles source files to produce an object file, in a way that is able to be used to create a shared library.
Size, which runs the 'size' utility on an executable file.

####> The Command Builder

env.Command(target, sources, "CMD" => command)
# Example
env.Command("user_guide.html", "user_guide.md",
  "CMD" => ["pandoc", "-fmarkdown", "-thtml", "-o${_TARGET}", "${_SOURCES}"],
  "CMD_DESC" => "Generating user guide:")

The Command builder executes a user-defined command in order to produce the desired target file based on the provided source files.

The Command builder supports the following construction variables:

CMD (required) specifies the command to execute (an array of strings). CMD is expanded for variable references, so the tokens ${_TARGET} and ${_SOURCES} can be used, for example.
CMD_DESC (optional) specifies the short text description to print when the builder executes. The given description is followed by the target file name.
CMD_STDOUT (optional) specifies a file to redirect standard output to. CMD_STDOUT is expanded for variable references, so the token ${_TARGET} can be used, for example.

####> The CFile Builder

env.CFile(target, source)
# Example
env.CFile("^/parser/parser.c", "parser.y")

The CFile builder will generate a C or C++ source file from a lex (.l, .ll) or yacc (.y, .yy) input file.

####> The Copy Builder

env.Copy(destination, sources)
# Example
env.Copy("mytests", "^/mytests")
env.Copy("^/dist/share", "share")

The Copy builder can copy files or directories to a target location.

####> The Directory Builder

env.Directory(target)
# Example
env.Directory("^/tests")

The Directory builder can be used to explicitly create a directory. This can also disambiguate whether the target for a subsequent builder (e.g. Copy) refers to a file path or directory path.

####> The Disassemble Builder

env.Disassemble(target, source)
# Example
env.Disassemble("module.dis", "module.o")

The Disassemble builder generates a disassembly listing using objdump from and object file.

####> The Install Builder

env.Install(destination, sources)
# Example
env.Install("${prefix}/bin", "app.exe")
env.Install("${prefix}/share", "share")

The Install builder can install files or directories to their installation target location. Install builders are only processed when the user has requested to perform an install operation from the command line.

####> The InstallDirectory Builder

env.InstallDirectory(target)
# Example
env.InstallDirectory("${prefix}/share")

The InstallDirectory builder can be used to explicitly create a directory. InstallDirectory builders are only processed when the user has requested to perform an install operation from the command line. This can also disambiguate whether the target for a subsequent builder (e.g. Install) refers to a file path or directory path.

####> The Library Builder

env.Library(target, sources)
# Example
env.Library("lib.a", Rscons.glob("src/**/*.c"))

The Library builder creates a static library archive from the given source files.

####> The Object Builder

env.Object(target, sources)
# Example
env.Object("module.o", "module.c")

The Object builder compiles the given sources to an object file. Although it can be called explicitly, it is more commonly implicitly called by the Program builder.

####> The Preprocess Builder

env.Preprocess(target, source)
# Example
env.Preprocess("module-preprocessed.cc", "module.cc")

The Preprocess builder invokes either ${CC} or ${CXX} (depending on if the source contains an extension in ${CXXSUFFIX} or not) and writes the preprocessed output to the target file.

####> The Program Builder

env.Program(target, sources)
# Example
env.Program("myprog", Rscons.glob("src/**/*.cc"))

The Program builder compiles and links the given sources to an executable file. Object files, static library files, or source files can be given as sources. A platform-dependent program suffix will be appended to the target name if one is not specified. This can be controlled with the PROGSUFFIX construction variable.

Direct Mode

The Program builder supports a "direct" mode which is activated by specifying the :direct option. In the direct mode, all source files are passed directly to the compiler together and compiled and linked in one step, rather than being individually compiled to separate object files first. This mode allows taking advantage of any multi-file compilation capabilities of the compiler. However, it also requires recompiling all source files when any one of them has changed.

Example use:

env.Program("myprog", Rscons.glob("src/**/*.c"), direct: true)

####> The SharedLibrary Builder

env.SharedLibrary(target, sources)
# Example
env.SharedLibrary("mydll", Rscons.glob("src/**/*.cc"))

The SharedLibrary builder compiles and links the given sources to a dynamically loadable library. Object files or source files can be given as sources. A platform-dependent prefix and suffix will be appended to the target name if they are not specified by the user. These values can be controlled by overriding the SHLIBPREFIX and SHLIBSUFFIX construction variables.

Direct Mode

The SharedLibrary builder supports a "direct" mode which is activated by specifying the :direct option. In the direct mode, all source files are passed directly to the compiler together and compiled and linked in one step, rather than being individually compiled to separate object files first. This mode allows taking advantage of any multi-file compilation capabilities of the compiler. However, it also requires recompiling all source files when any one of them has changed.

Example use:

env.SharedLibrary("mydll", Rscons.glob("src/**/*.c"), direct: true)

####> The SharedObject Builder

env.SharedObject(target, sources)
# Example
env.SharedObject("lib_module.o", "lib_module.c")

The SharedObject builder compiles the given sources to an object file. Any compilation flags necessary to build the object file in a manner that allows it to be used to create a shared library are added. Although it can be called explicitly, it is more commonly implicitly called by the SharedLibrary builder.

####> The Size Builder

env.Size(target, sources)
# Example
env.Program("program.exe", glob("*.c"))
env.Size("program.size", "program.exe")

The Size builder runs the "size" executable on the given source file and stores its output in the target file. The size executable can be specified with the SIZE construction variable, and flags can be specified with SIZEFLAGS.

###> Phony Targets

rscons supports phony build targets. Normally, a builder produces an output file, and executes whenever the input files or command have changed. A phony build target can be used to register a builder that does not produce an output file. A custom builder can take some action when the input files change even if it does not produce an output file. Such a builder could perform verification or run a test on its source files, possibly failing if some conditions are not met. It could also simply output something to the console, such as an analysis of the source file, whenever it changes. A phony target is signified by passing a Symbol instead of a String as the first parameter (target) to a builder method.

###> Explicit Dependencies

A target can be marked as depending on another file that Rscons would not otherwise know about via the Environment#depends function. For example, to force the linker to re-link a Program output when a linker script changes:

env.Program("a.out", "foo.c", "LDFLAGS" => %w[-T linker_script.ld])
env.depends("a.out", "linker_script.ld")

You can pass multiple dependency files to Environment#depends:

env.depends("my_app", "config/link.ld", "README.txt", *glob("assets/**/*"))

###> Build Hooks

A build hook is a Ruby block that is called whenever Rscons is about to invoke a builder to produce a build target. Rscons also supports post-build hooks which are called after the builder has produced the build target. A build hook can be used to modify construction variables depending on the build target or source file names.

Example:

build do
  Environment.new do |env|
    env["CFLAGS"] << "-Wall"
    env.add_build_hook do |builder|
      # Compile sources from under src/tests without the -Wall flag.
      if builder.sources.first =~ %r{src/tests/}
        builder.vars["CFLAGS"] -= %w[-Wall]
      end
    end
    env.Program("program.exe", glob("src/**/*.c"))
  end
end

This example script would compile all C sources under the src directory with the -Wall flag except for sources under the src/tests directory.

A post-build hook can be added with env.add_post_build_hook. Post-build hooks are only invoked if the build operation was a success.

Build hooks and post-build hooks can register new build targets.

###> Barriers

Normally Rscons will parallelize all builders. A barrier can be used to separate sets of build targets. All build targets registered before the barrier is created will be built before Rscons will schedule any build targets after the barrier. In other words, build targets are not parallelized across a barrier.

env.barrier

##> Extending Rscons

Adding New Languages

The Object and SharedObject builders that ship with Rscons have an API that allows the user to register extra languages that can be suppored by the builders. In fact, the built-in support for assembly, C, C++, and D compilation all make use of this built-in API. To see an example of how this API is used, see the lib/rscons/builders/lang/*.rb files in the Rscons source repository. For example, here is how the C++ language is registered:

Rscons::Builders::Object.register(command: "${CXXCMD}", direct_command: "${CXXCMD:direct}", suffix: "${CXXSUFFIX}", preferred_ld: "${CXX}")
Rscons::Builders::SharedObject.register(command: "${SHCXXCMD}", direct_command: "${SHCXXCMD:direct}", suffix: "${CXXSUFFIX}", preferred_ld: "${SHCXX}")

There are also default construction variables registered to go along with the language registration as specified above. New default construction variables can be registered globally by assigning to the Rscons::DEFAULT_CONSTRUCTION_VARIABLES Hash. For example:

Rscons::DEFAULT_CONSTRUCTION_VARIABLES["CXXCMD"] = %w[${CXX} -c -o ${_TARGET} ${CXXDEPGEN} ${INCPREFIX}${CPPPATH} ${CPPFLAGS} ${CXXFLAGS} ${CCFLAGS} ${_SOURCES}]

###> Adding Custom Builders

It is also possible to extend Rscons with new builders. This is the most flexible method to extend Rscons. Builders can execute a command line program, call another builder, or just use plain Ruby code to produce an output file.

A builder is a class that inherits from the Rscons::Builder base class. Rscons provides a Rscons::Builders namespacing module which contains the built-in builder classes. User-provided custom builder classes can also reside in the Rscons::Builders namespacing module, but this is not required.

####> Adding a Custom Builder to an Environment

The user can add a builder class to an Environment with the env.add_builder method. For example:

class Rscons::Builders::Mine < Rscons::Builder
end

build do
  Environment.new do |env|
    env.add_builder(Rscons::Builders::Mine)
  end
end

Alternatively, the builder author can add the name of the custom builder to the Rscons::DEFAULT_BUILDERS array and then Rscons will automatically add the custom builder to every Environment. This method only works if the custom builder class is contained within the Rscons::Builders namespacing module. For example:

#SpecialBuilder.rb
class Rscons::Builders::Special < Rscons::Builder
end
Rscons::DEFAULT_BUILDERS << :Special

#Rsconscript
load "SpecialBuilder.rb"

build do
  Environment.new do |env|
    # A build target using the "Special" builder can be registered.
    env.Special("target", "source")
  end
end

####> Builder Name

By default, the builder name is taken from the last component of the class name. For example, a class called Rscons::Builders::Mine would be usable in the Rsconscript with env.Mine(). A builder author can override the builder name by defining a class method within the builder class called name. For example, with the following builder definition:

class Rscons::Builders::MySpecialBuilder < Rscons::Builder
  def self.name
    "Special"
  end
end

This builder would be registered in the Rsconscript with env.Special().

####> Custom Builder Constructor

It is optional for a custom builder to provide an initialize method. If an initialize method is provided, it must call super to invoke the base Rscons::Builder class's constructor. A single Hash parameter is passed to the builder constructor. This Hash contains many parameters describing how the build target was registered by the user. The base constructor will set several instance attributes within the builder:

@target will contain the path to the build target
@sources will contain the path(s) to the build source(s)
@cache will contain a reference to the Rscons::Cache object used for the build
@env will contain a reference to the Environment object that registered the build target using the builder
@vars will contain any user-specified construction variable values that should be used for the build operation (overriding any Environment-wide construction variable values)

####> Custom Builder Operation

In order for a builder to perform a build operation, the builder class must implement a the Builder#run() method. Generally, the run() method will use the source file(s) to produce the target file. Here is an example of a trivial builder:

class Rscons::Builders::Custom < Rscons::Builder
  def run(options)
    File.open(@target, "w") do |fh|
      fh.write("Target file created.")
    end
    true
  end
end

Return Value

If the build operation has completed and failed, the run method should return false. In this case, generally the command executed or the builder itself would be expected to output something to $stderr indicating the reason for the build failure. If the build operation has completed successfully, the run method should return true. If the build operation is not yet complete and is waiting on other operations, the run method should return the return value from the Builder#wait_for method. See ${#Custom Builder Parallelization}.

Printing Build Status

A builder should print a status line when it produces a build target. The Builder#print_run_message method can be used to print the builder status line. This method supports a limited markup syntax to identify and color code the build target and/or source(s). Here is our Custom builder example extended to print its status:

class Rscons::Builders::Custom < Rscons::Builder
  def run(options)
    print_run_message("Creating <target>#{@target}<reset> from Custom builder", nil)
    File.open(@target, "w") do |fh|
      fh.write("Target file created.")
    end
    true
  end
end

Custom Builder Cache Usage - Only Rebuild When Necessary

Whenever possible, a builder should keep track of information necessary to know whether the target file(s) need to be rebuilt. The Rscons::Cache object is the mechanism by which to keep track of this information. The Cache object provides two methods: #up_to_date? and #register_build which can be used to check if a built file is still up-to-date, and to register build information for a subsequent check. Here is a Custom builder which combines its source files similar to what the cat command would do:

class Rscons::Builders::Custom < Rscons::Builder
  def run(options)
    unless @cache.up_to_date?(@target, nil, @sources, @env)
      print_run_message("Combining <source>#{Util.short_format_paths(@sources)}<reset> => <target>#{@target}<reset>", nil)
      File.open(@target, "wb") do |fh|
        @sources.each do |source|
          fh.write(File.read(source, mode: "rb"))
        end
      end
      @cache.register_build(@target, nil, @sources, @env)
    end
    true
  end
end

This builder would rebuild the target file and print its run message if the target file or any of the source file(s) were changed, but otherwise would be silent and not re-combine the source files.

Note that generally the same arguments should be passed to @cache.register_build and @cache.up_to_date?.

Custom Builder Parallelization

The Rscons scheduler can parallelize builders to take advantage of multiple processor cores. Taking advantage of this ability to parallelize requires the builder author to author the builder in a particular way. The #run() method of each builder is called from Rscons in the main program thread. However, the builder may execute a subcommand, spawn a thread, or register other builders to execute as a part of doing its job. In any of these cases, the builder's run method should make use of Builder#wait_for to "sleep" until one of the items being waited for has completed.

Using a Ruby Thread to Parallelize a Build Operation

Here is an example of using a Ruby thread to parallelize a build operation:

${include build_tests/custom_builder/wait_for_thread.rb}

It is up to the author of the thread logic to only perform actions that are thread-safe. It is not safe to call other Rscons methods, for example, registering other builders or using the Cache, from a thread other than the one that calls the #run() method.

Executing a Subcommand from a Custom Builder

It is a very common case that a builder will execute a subcommand which produces the build target. This is how most of the built-in Rscons builders execute. A low-level way to handle this is for the builder to construct an instance of the Rscons::Command class and then wait_for the Command object. However, this is a common enough case that Rscons provides a few convenience methods to handle this:

The register_command helper method can be used to create a Command object and wait for it to complete. The standard_command helper does the same thing as register_command but additionally checks the @cache for the target being up to date. The finalize_command helper can be used in conjunction with either of the previous helper methods.

The built-in Rscons builders Command and Disassemble show examples of how to use the standard_command and finalize_command helper methods.

Example (built-in Command builder):

${include lib/rscons/builders/command.rb}

Example (built-in Disassemble builder):

${include lib/rscons/builders/disassemble.rb}

####> Simple custom builders added with add_builder

The add_builder method of the Environment class optionally allows you to define and register a builder by providing a name and action block. This can be useful if the builder you are trying to define is easily expressed as a short ruby procedure. When add_builder is called in this manner a new builder will be registered with the environment with the given name. When this builder is used it will call the provided block in order to build the target.

Example:

${include build_tests/json_to_yaml/Rsconscript}

#> Appendix

Default Construction Variables

${include lib/rscons/default_construction_variables.rb}

##> Example Build Scripts

Example: Building a C Program

build do
  Environment.new do |env|
    env["CFLAGS"] << "-Wall"
    env.Program("program", glob("src/**/*.c"))
  end
end

Example: Building a D Program

build do
  Environment.new do |env|
    env["DFLAGS"] << "-Wall"
    env.Program("program", glob("src/**/*.d"))
  end
end

Example: Cloning an Environment

build do
  main_env = Environment.new do |env|
    env["CFLAGS"] = ["-DSOME_DEFINE", "-O3"]
    env["LIBS"] = ["SDL"]
    env.Program("program", glob("src/**/*.cc"))
  end

  debug_env = main_env.clone do |env|
    env["CFLAGS"] -= ["-O3"]
    env["CFLAGS"] += ["-g", "-O0"]
    env.Program("program-debug", glob("src/**/*.cc"))
  end
end

Example: Custom Builder

class GenerateFoo < Builder
  def run(options)
    target, cache = options.values_at(:target, :cache)
    cache.mkdir_p(File.dirname(target))
    File.open(target, "w") do |fh|
      fh.puts <<EOF
#define GENERATED 42
EOF
    end
    target
  end
end

build do
  Environment.new do |env|
    env.add_builder(GenerateFoo)
    env.GenerateFoo("foo.h", [])
    env.Program("a.out", glob("*.c"))
  end
end

Example: Using different compilation flags for some sources

build do
  Environment.new do |env|
    env["CFLAGS"] = ["-O3", "-Wall"]
    env.add_build_hook do |build_op|
      if build_op[:target] =~ %r{build/third-party}
        build_op[:vars]["CFLAGS"] -= ["-Wall"]
      end
    end
    env.Program("program", glob("**/*.cc"))
  end
end

Example: Creating a static library

build do
  Environment.new do |env|
    env.Library("mylib.a", glob("src/**/*.c"))
  end
end

Example: Creating a C++ parser source from a Yacc/Bison input file

build do
  Environment.new do |env|
    env.CFile("^/parser.tab.cc", "parser.yy")
  end
end

##> ./configure && make

You can make your Rscons-based project more familiar to users of autoconf-generated projects by creating a configure script and a Makefile for the user. Such users may be used to executing:

./configure
make

to build a project. To do this, create a configure script with contents similar to the following:

#!/bin/sh
exec "$(dirname "$0")"/rscons configure "$@"

and make it executable with chmod +x configure.

If you want your users to be able to build/clean a project with make but still make use of Rscons under the hood, you can create a Makefile with contents something like this:

.PHONY: all
all:
	./rscons build

.PHONY: clean
clean:
	./rscons clean

##> YARD API Documentation

See here for Rscons YARD API Documentation.

#> License

Rscons is licensed under the terms of the MIT License:

${include LICENSE.txt}

#> Contributing

Rscons is developed on github.

Issues may be submitted to https://github.com/holtrop/rscons/issues.

Pull requests may be submitted as well:

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

#> Change Log

${changelog}

47 KiB Raw Blame History

Design Principles

Build Correctness

Build Flexibility

Build Efficiency

Build Directory

Getting Started

Version Control Setup

Options

:check_cpppath

Options

:check_d_import_path

Options

:check_libpath

:use

Options

:package

:program

:use

:fail

:on_fail

:set_define

Construction Variable Naming

Direct Mode

Direct Mode

Adding New Languages

Return Value

Printing Build Status

Custom Builder Cache Usage - Only Rebuild When Necessary

Custom Builder Parallelization

Using a Ruby Thread to Parallelize a Build Operation

Executing a Subcommand from a Custom Builder

Default Construction Variables

Example: Building a C Program

Example: Building a D Program

Example: Cloning an Environment

Example: Custom Builder

Example: Using different compilation flags for some sources

Example: Creating a static library

Example: Creating a C++ parser source from a Yacc/Bison input file

47 KiB

Raw Blame History

`:check_cpppath`

`:check_d_import_path`

`:check_libpath`

`:use`

`:package`

`:program`

`:use`

`:fail`

`:on_fail`

`:set_define`