rscons/doc/user_guide.md

#> Overview

Rscons is an open-source build system for developers. It supports the following features:

• multi-threaded job execution
• auto-configuration
• built-in builders for several common operations
• out-of-the-box support for C, C++, and D languages
• extensibility for other languages or custom builders
• compatible with Windows, Linux, and OS X
• colorized output with build progress
• build hooks

At its core, Rscons is mainly an engine to:

• determine the proper order to perform build operations,
• determine whether each build target is up to date or in need of rebuild, and
• schedule those build operations across multiple threads as efficiently as possible.

Along the way, Rscons provides a concise syntax for specifying common types of build operations, but also provides an extensible framework for performing custom build operations as well.

Rscons is written in Ruby, and is inspired by SCons and waf.

Design Principles

Build Correctness

The number one design principle in Rscons is build correctness. This means that a build operation will be performed when Rscons cannot determine that a build target is already up-to-date. A build target will be built whenever:

• the target file has been removed or changed since it was last built
• the command to build the target file is different from the previous command used to build it
• any of the target file's dependency files have changed since the last time the target was built

Importantly, Rscons uses the content of a source (dependency) file to determine whether a rebuild is necessary, not simply the timestamp of the file. This is because relying solely on the timestamp of the file can lead to an incorrect decision being made to not rebuild when a rebuild is necessary.

Build Flexibility

Rscons supports multiple configurations of compilation flags or build options across multiple environments to build output files in different ways according to the user's desire. For example, the same source files can be built into a release executable, but also compiled with different compilation flags or build options into a test executable. Rscons also supports build hooks, which allow the user to further fine-tune the build system's operation. A build hook, for example, can be used to set a build option for only source files coming from a particular source directory.

Build Efficiency

Rscons will automatically determine the number of threads to use based on the host CPU configuration, and will schedule jobs as efficiently as possible across the available threads in order to complete the build operation in as little time as possible. As development occurs and build operations are executed, Rscons makes use of a cache file in order to avoid rebuilding a target when it is already up to date.

Build Directory

Rscons was designed to store temporary build artifacts (for example, object files, dependency files, etc...) in a build directory. This keeps files generated by the build cleanly separated from user-controlled source files.

Getting Started

To use Rscons on your project, you must:

1. Install the rscons script in your project (See ${#Installation}).
2. Write the Rsconscript build script for your project (See ${#The Build Script}).
3. Use the rscons command in your project (See ${#Command-Line Operation}).

#> Installation

Rscons is designed to be distributed as a stand-alone single file script that can be copied into and versioned in a project's source tree. The only dependency required to run Rscons is to have a Ruby interpreter installed. The latest release can be downloaded from https://github.com/holtrop/rscons/releases. Simply copy the rscons executable script into the desired location within the project to be built (typically the root of the repository).

Version Control Setup

The following files should be added to source control:

• rscons
• Rsconscript

Add the following contents to .gitignore (or the equivalent thereof for different version control systems):

/.rscons*
/build/

#> Command-Line Operation

Rscons is typically invoked from the command-line as ./rscons. Rscons supports several build operations:

• configure
• build
• clean
• distclean
• install
• uninstall

##> Configure Operation

The configure operation will initialize the Rscons cache file and build directory. It will also perform any configuration checks requested by the build script. Such configuration checks can include:

• verifying operation of a compiler
• loading compilation/linker flags from a config program (e.g. pkg-config)
• verifying presence of a C/C++ header file
• verifying presence of a D import
• verifying presence of a library
• verifying presence of an executable

##> Build Operation

If a build operation is requested and a configure operation has not yet been performed, a configure operation will be automatically invoked.

The build operation will execute all builders registered to produce build targets.

##> Clean Operation

A clean operation will remove all built target files. It will not remove items installed by an install operation. It will not remove the cached configuration options.

##> Distclean Operation

A distclean operation will remove all built target files and all cached configuration options. Generally it will get the project directory back to the state it was in when unpacked before any configuration or build operations took place. It will not removed items installed by an install operation.

##> Install Operation

An install operation will perform a build (and if necessary, first a configure as well). In addition it will execute any Install or InstallDirectory builders to install items into the specified install directory.

##> Uninstall Operation

An uninstall operation will remove any items installed by an install operation. It will not remove all built target files, just the installed copies.

#> The Build Script

Rscons looks for instructions for what to build by reading a build script file called Rsconscript (or Rsconscript.rb). Here is a simple example Rsconscript file:

build do
  Environment.new do |env|
    env.Program("myprog.exe", glob("src/**/*.c"))
  end
end

This Rsconscript file would instruct Rscons to produce a Program target called myprog.exe which is to be built from all C source files found (recursively) under the src directory.

The Rsconscript file is a Ruby script.

##> Configuration Operations

A configure block is optional. It can be used to perform various checks and setup operations for a project. Example configure block:

configure do
  check_cxx_compiler
  check_c_header "getopt.h"
end

###> Checking for a Compiler

The following methods can be used within a configure block to check for a working compiler:

• check_c_compiler
• check_cxx_compiler
• check_d_compiler

Each of these methods can take an optional list of compilers to check for. If such a list is supplied, the compilers are tested in the order listed.

Here are example calls which also show the default compiler list for each supported language:

configure do
  check_c_compiler "gcc", "clang"
  check_cxx_compiler "g++", "clang++"
  check_d_compiler "gdc", "ldc2"
end

###> Checking for a Header File

The following methods can be used to check for the presence of a header file:

• check_c_header will check for a C header to be present
• check_cxx_header will check for a C++ header to be present

Each of these methods take the name of the header file to check for as the first argument, and take an optional Hash of arguments as the second argument.

Example calls:

configure do
  check_c_header "getopt.h", fail: false, set_define: "HAVE_GETOPT_H"
  check_c_header "FreeType2.h"
  check_cxx_header "memory"
end

Options

:fail

If the :fail option is set to false, then the absence of the header file will not result in the configure option failing.

:set_define

If set, a build define of the specified String will be added to the CPPDEFINES construction variable array if the requested header is found.

###> Checking for a D Import

The check_d_import method can be used to check for the presence of D import.

This method takes the name of the import to check for as the first argument.

Example calls:

configure do
  check_d_import "std.stdio"
  check_d_import "std.numeric"
end

###> Checking for a Library

The check_lib method can be used to check for the presence of a library.

This method takes the name of the library to check for as the first argument, and take an optional Hash of arguments as the second argument.

Example calls:

configure do
  check_lib "kpty", fail: false, set_define: "HAVE_LIBKPTY"
  check_lib "GL"
end

Options

:fail

If the :fail option is set to false, then the absence of the library will not result in the configure option failing.

:set_define

If set, a build define of the specified String will be added to the CPPDEFINES construction variable array if the requested library is found.

###> Checking for a Program

The check_program method can check for the existence of an executable in the host operating system environment.

Example call:

configure do
  check_program "xxd"
end

###> Checking for a Package Configuration

The check_cfg method can be used to check for the existence of a package as well as import any build options (e.g. include path, defines, libraries to link against, etc...) required to use the package.

This method takes a Hash of options as its only argument.

Example calls:

configure do
  check_cfg package: "zlib"
  check_cfg program: "freetype-config", fail: false, set_define: "HAVE_FREETYPE"
end

Options

:package

If the :package option is set to a value, the pkg-config program will be used to look for package configuration flags for the specified package.

:program

If the :program option is given, the program specified will be used to look for configuration flags.

:fail

If the :fail option is set to false, then the absence of the package or program requested will not result in the configure option failing.

:set_define

If set, a build define of the specified String will be added to the CPPDEFINES construction variable array if the requested package is found.

##> Build Operations

The build block is used to create Environments and register build targets. An Rscons build script would not be very useful without a build block.

Here is an example build block demonstrating how to register a build target:

build do
  Environment.new do |env|
    env.Program("myprog.exe", glob("src/**/*.c"))
  end
end

This Rsconscript would build an executable called myprog.exe from all C source files found recursively under the src directory.

###> Environments

An Environment includes:

• a collection of construction variables
• a collection of build hooks
• a collection of user-registered build targets
• a build root

All build targets must be registered within an Environment. The Environment's build root is a directory created within the top-level Rscons build directory. By default it holds all intermediate files generated by Rscons that are needed to produce a user-specified build target. For example, for the Rsconscript:

build do
  Environment.new do |env|
    env.Program("myprog.exe", glob("src/**/*.c"))
  end
end

Rscons will place an object file and dependency file corresponding to each C source file under the Environment's build root. This keeps the intermediate generated build artifacts separate from the source files.

###> Specifying Source Files: The glob Method

The glob method can be used to find files matching the patterns specified. It supports a syntax similar to the Ruby Dir.glob method but operates more deterministically.

Example use:

build do
  Environment.new do |env|
    env.Program("mytests", glob("src/**/*.cc", "test/**/*.cc"))
  end
end

This example would build the mytests executable from all .cc source files found recursively under the src or test directory.

###> Construction Variables

Construction variables are values assigned to keys within an Environment. Construction variables are used by Builders to produce output files. See ${#Default Construction Variables} for a reference of all built-in construction variables.

Example:

build do
  Environment.new do |env|
    env["CCFLAGS"] += %w[-O2 -Wall]
    env["LIBS"] += %w[m]
  end
end

This example modifies the CCFLAGS construction variable to add -O2 and -Wall to the compilation commands used for C and C++ source files. It also instructs the linker to link against the m library.

###> Builders

Rscons uses builder objects to produce target output files from source input files. A build target to be built using a builder is registered by calling a method on the Environment object that matches the builder's name. For example, a Program build target is registered by calling the env.Program method.

The general syntax for registering a build target using a builder is:

env.BuilderName(target, sources, vars = {})

The target parameter is the path to the output file or directory. The sources parameter is the path or paths to the input file(s) to be used by the builder. In the target and sources parameters, the user can explicitly refer to a path within the Environment's build root by beginning the path with "^/". The vars parameter is an optional Hash which can include construction variables to be used for this build target. Any construction variable values specified in this parameter will override those assigned to the Environment.

There are several default builders that are built-in to Rscons:

• Command, which executes a user-defined command to produce the target.
• Copy, which copies files or directories to a specified destination.
• CFile, which builds a C or C++ source file from a lex or yacc input file.
• Directory, which creates a directory.
• Disassemble, which disassembles an object file to a disassembly listing.
• Install, which installs files or directories to a specified destination.
• InstallDirectory, which creates a directory during an install operation.
• Library, which collects object files into a static library archive file.
• Object, which compiles source files to produce an object file.
• Preprocess, which invokes the C/C++ preprocessor on a source file.
• Program, which links object files to produce an executable.
• SharedLibrary, which links object files to produce a dynamically loadable library.
• SharedObject, which compiles source files to produce an object file, in a way that is able to be used to create a shared library.

####> The Command Builder

env.Command(target, sources, "CMD" => command)
# Example
env.Command("docs.html", "docs.md",
  "CMD" => ["pandoc", "-fmarkdown", "-thtml", "-o${_TARGET}", "${_SOURCES}"],
  "CMD_DESC" => "PANDOC")

The Command builder executes a user-defined command in order to produce the desired target file based on the provided source files.

####> The CFile Builder

env.CFile(target, source)
# Example
env.CFile("^/parser/parser.c", "parser.y")

The CFile builder will generate a C or C++ source file from a lex (.l, .ll) or yacc (.y, .yy) input file.

####> The Copy Builder

env.Copy(destination, sources)
# Example
env.Copy("mytests", "^/mytests")
env.Copy("^/dist/share", "share")

The Copy builder can copy files or directories to a target location.

####> The Directory Builder

env.Directory(target)
# Example
env.Directory("^/tests")

The Directory builder can be used to explicitly create a directory. This can also disambiguate whether the target for a subsequent builder (e.g. Copy) refers to a file path or directory path.

####> The Disassemble Builder

env.Disassemble(target, source)
# Example
env.Disassemble("module.dis", "module.o")

The Disassemble builder generates a disassembly listing using objdump from and object file.

####> The Install Builder

env.Install(destination, sources)
# Example
env.Install("${prefix}/bin", "app.exe")
env.Install("${prefix}/share", "share")

The Install builder can install files or directories to their installation target location. Install builders are only processed when the user has requested to perform an install operation from the command line.

####> The InstallDirectory Builder

env.InstallDirectory(target)
# Example
env.InstallDirectory("${prefix}/share")

The InstallDirectory builder can be used to explicitly create a directory. InstallDirectory builders are only processed when the user has requested to perform an install operation from the command line. This can also disambiguate whether the target for a subsequent builder (e.g. Install) refers to a file path or directory path.

####> The Library Builder

env.Library(target, sources)
# Example
env.Library("lib.a", Rscons.glob("src/**/*.c"))

The Library builder creates a static library archive from the given source files.

####> The Object Builder

env.Object(target, sources)
# Example
env.Object("module.o", "module.c")

The Object builder compiles the given sources to an object file. Although it can be called explicitly, it is more commonly implicitly called by the Program builder.

####> The Preprocess Builder

env.Preprocess(target, source)
# Example
env.Preprocess("module-preprocessed.cc", "module.cc")

The Preprocess builder invokes either ${CC} or ${CXX} (depending on if the source contains an extension in ${CXXSUFFIX} or not) and writes the preprocessed output to the target file.

####> The Program Builder

env.Program(target, sources)
# Example
env.Program("myprog", Rscons.glob("src/**/*.cc"))

The Program builder compiles and links the given sources to an executable file. Object files, static library files, or source files can be given as sources. A platform-dependent program suffix will be appended to the target name if one is not specified. This can be controlled with the PROGSUFFIX construction variable.

####> The SharedLibrary Builder

env.SharedLibrary(target, sources)
# Example
env.SharedLibrary("mydll", Rscons.glob("src/**/*.cc"))

The SharedLibrary builder compiles and links the given sources to a dynamically loadable library. Object files or source files can be given as sources. A platform-dependent prefix and suffix will be appended to the target name if they are not specified by the user. These values can be controlled by overriding the SHLIBPREFIX and SHLIBSUFFIX construction variables.

####> The SharedObject Builder

env.SharedObject(target, sources)
# Example
env.SharedObject("lib_module.o", "lib_module.c")

The SharedObject builder compiles the given sources to an object file. Any compilation flags necessary to build the object file in a manner that allows it to be used to create a shared library are added. Although it can be called explicitly, it is more commonly implicitly called by the SharedLibrary builder.

###> Build Hooks

A build hook is a Ruby block that is called whenever Rscons is about to invoke a builder to produce a build target. Rscons also supports post-build hooks which are called after the builder has produced the build target. A build hook can be used to modify construction variables depending on the build target or source file names.

Example:

build do
  Environment.new do |env|
    env["CFLAGS"] << "-Wall"
    env.add_build_hook do |builder|
      # Compile sources from under src/tests without the -Wall flag.
      if builder.sources.first =~ %r{src/tests/}
        builder.vars["CFLAGS"] -= %w[-Wall]
      end
    end
    env.Program("program.exe", glob("src/**/*.c"))
  end
end

This example script would compile all C sources under the src directory with the -Wall flag except for sources under the src/tests directory.

##> Extending Rscons

Adding New Languages

The Object and SharedObject builders that ship with Rscons have an API that allows the user to register extra languages that can be suppored by the builders. In fact, the built-in support for assembly, C, C++, and D compilation all make use of this built-in API. To see an example of how this API is used, see the lib/rscons/builders/lang/*.rb files in the Rscons source repository. For example, here is how the C++ language is registered:

Rscons::Builders::Object.register(command: "${CXXCMD}", direct_command: "${CXXCMD:direct}", suffix: "${CXXSUFFIX}", preferred_ld: "${CXX}")
Rscons::Builders::SharedObject.register(command: "${SHCXXCMD}", direct_command: "${SHCXXCMD:direct}", suffix: "${CXXSUFFIX}", preferred_ld: "${SHCXX}")

There are also default construction variables registered to go along with the language registration as specified above. New default construction variables can be registered globally by assigning to the Rscons::DEFAULT_CONSTRUCTION_VARIABLES Hash. For example:

Rscons::DEFAULT_CONSTRUCTION_VARIABLES["CXXCMD"] = %w[${CXX} -c -o ${_TARGET} ${CXXDEPGEN} ${INCPREFIX}${CPPPATH} ${CPPFLAGS} ${CXXFLAGS} ${CCFLAGS} ${_SOURCES}]

Adding New Builders

It is also possible to extend Rscons with new builders. This is the most flexible method to extend Rscons. Builders can execute a command line program, call another builder, or just use plain Ruby code to produce an output file.

#> Reference

Default Construction Variables

#> License

Rscons is licensed under the terms of the MIT License:

${include LICENSE.txt}

#> Change Log

${changelog}