rscons/doc/user_guide.md

#> Overview

Rscons is an open-source build system for developers.
It supports the following features:

  * multi-threaded job execution
  * auto-configuration
  * built-in builders for several common operations
  * out-of-the-box support for C, C++, and D languages
  * extensibility for other languages or custom builders
  * compatible with Windows, Linux, and OS X
  * colorized output with build progress
  * build hooks

At its core, Rscons is mainly an engine to:

  * determine the proper order to perform build operations,
  * determine whether each build target is up to date or in need of rebuild, and
  * schedule those build operations across multiple threads as efficiently as possible.

Along the way, Rscons provides a concise syntax for specifying common types of
build operations, but also provides an extensible framework for performing
custom build operations as well.

Rscons is written in Ruby, and is inspired by [SCons](https://scons.org/) and [waf](https://waf.io/).

## Design Principles

### Build Correctness

The number one design principle in Rscons is build correctness.
This means that a build operation will be performed when Rscons cannot
determine that a build target is already up-to-date.
A build target will be built whenever:

  * the target file has been removed or changed since it was last built
  * the command to build the target file is different from the previous command
    used to build it
  * any of the target file's dependency files have changed since the last time
    the target was built

Importantly, Rscons uses the content of a source (dependency) file to determine
whether a rebuild is necessary, not simply the timestamp of the file.
This is because relying solely on the timestamp of the file can lead to an
incorrect decision being made to not rebuild when a rebuild is necessary.

### Build Flexibility

Rscons supports multiple configurations of compilation flags or build options
across multiple environments to build output files in different ways according
to the user's desire.
For example, the same source files can be built into a release executable, but
also compiled with different compilation flags or build options into a test
executable.
Rscons also supports build hooks, which allow the user to further fine-tune the
build system's operation.
A build hook, for example, can be used to set a build option for only source
files coming from a particular source directory.

### Build Efficiency

Rscons will automatically determine the number of threads to use based on the
host CPU configuration, and will schedule jobs as efficiently as possible
across the available threads in order to complete the build operation in as
little time as possible.
As development occurs and build operations are executed, Rscons makes use of a
cache file in order to avoid rebuilding a target when it is already up to date.

### Build Directory

Rscons was designed to store temporary build artifacts (for example, object
files, dependency files, etc...) in a `build` directory.
This keeps files generated by the build cleanly separated from user-controlled
source files.

## Getting Started

To use Rscons on your project, you must:

  1. Install the `rscons` script in your project (See ${#Installation}).
  2. Write the `Rsconscript` build script for your project (See ${#The Build Script}).

#> Installation

Rscons is designed to be distributed as a stand-alone single file script that
can be copied into and versioned in a project's source tree.
The only dependency required to run Rscons is to have a Ruby interpreter
installed.
The latest release can be downloaded from [https://github.com/holtrop/rscons/releases](https://github.com/holtrop/rscons/releases).
Simply copy the `rscons` executable script into the desired location within
the project to be built (typically the root of the repository).

## Version Control Setup

The following files should be added to source control:

  * `rscons`
  * `Rsconscript`

Add the following contents to `.gitignore` (or the equivalent thereof for different
version control systems):

```
/.rscons*
/build/
```

#> Command-Line Operation

Rscons is typically invoked from the command-line as `./rscons`.
Rscons supports several build *operations*:

  * configure
  * build
  * clean
  * distclean
  * install
  * uninstall

##> Configure Operation

The `configure` operation will initialize the Rscons cache file and build
directory.
It will also perform any configuration checks requested by the build script.
Such configuration checks can include:

  * verifying operation of a compiler
  * loading compilation/linker flags from a config program (e.g. `pkg-config`)
  * verifying presence of a C/C++ header file
  * verifying presence of a D import
  * verifying presence of a library
  * verifying presence of an executable

##> Build Operation

If a `build` operation is requested and a `configure` operation has not yet
been performed, a `configure` operation will be automatically invoked.

The `build` operation will execute all builders registered to produce build
targets.

##> Clean Operation

A `clean` operation will remove all built target files.
It will not remove items installed by an `install` operation.
It will not remove the cached configuration options.

##> Distclean Operation

A `distclean` operation will remove all built target files and all cached
configuration options.
Generally it will get the project directory back to the state it was in when
unpacked before any configuration or build operations took place.
It will not removed items installed by an `install` operation.

##> Install Operation

An `install` operation will perform a `build` (and if necessary, first a
`configure` as well).
In addition it will execute any `Install` or `InstallDirectory` builders to
install items into the specified install directory.

##> Uninstall Operation

An `uninstall` operation will remove any items installed by an `install`
operation.
It will not remove all built target files, just the installed copies.

#> The Build Script

Rscons looks for instructions for what to build by reading a build script file
called `Rsconscript` (or `Rsconscript.rb`).
Here is a simple example `Rsconscript` file:

```ruby
build do
  Environment.new do |env|
    env.Program("myprog.exe", glob("src/**/*.c"))
  end
end
```

This `Rsconscript` file would instruct Rscons to produce a *Program* target
called `myprog.exe` which is to be built from all C source files found
(recursively) under the `src` directory.

The `Rsconscript` file is a Ruby script.

##> Configuration Operations

A `configure` block is optional.
It can be used to perform various checks and setup operations for a project.
Example `configure` block:

```ruby
configure do
  check_cxx_compiler
  check_c_header "getopt.h"
end
```

###> Checking for a Compiler

The following methods can be used within a `configure` block to check for a
working compiler:

  * `check_c_compiler`
  * `check_cxx_compiler`
  * `check_d_compiler`

Each of these methods can take an optional list of compilers to check for.
If such a list is supplied, the compilers are tested in the order listed.

Here are example calls which also show the default compiler list for each
supported language:

```ruby
configure do
  check_c_compiler "gcc", "clang"
  check_cxx_compiler "g++", "clang++"
  check_d_compiler "gdc", "ldc2"
end
```

###> Checking for a Header File

The following methods can be used to check for the presence of a header file:

  * `check_c_header` will check for a C header to be present
  * `check_cxx_header` will check for a C++ header to be present

Each of these methods take the name of the header file to check for as the
first argument, and take an optional Hash of arguments as the second argument.

Example calls:

```ruby
configure do
  check_c_header "getopt.h", fail: false, set_define: "HAVE_GETOPT_H"
  check_c_header "FreeType2.h"
  check_cxx_header "memory"
end
```

#### Options

##### `:fail`

If the `:fail` option is set to `false`, then the absence of the header file
will not result in the configure option failing.

##### `:set_define`

If set, a build define of the specified String will be added to the
`CPPDEFINES` construction variable array if the requested header is found.

###> Checking for a D Import

The `check_d_import` method can be used to check for the presence of D import.

This method takes the name of the import to check for as the first argument.

Example calls:

```ruby
configure do
  check_d_import "std.stdio"
  check_d_import "std.numeric"
end
```

###> Checking for a Library

The `check_lib` method can be used to check for the presence of a library.

This method takes the name of the library to check for as the first argument,
and take an optional Hash of arguments as the second argument.

Example calls:

```ruby
configure do
  check_lib "kpty", fail: false, set_define: "HAVE_LIBKPTY"
  check_lib "GL"
end
```

#### Options

##### `:fail`

If the `:fail` option is set to `false`, then the absence of the library
will not result in the configure option failing.

##### `:set_define`

If set, a build define of the specified String will be added to the
`CPPDEFINES` construction variable array if the requested library is found.

###> Checking for a Program

The `check_program` method can check for the existence of an executable in the
host operating system environment.

Example call:

```ruby
configure do
  check_program "xxd"
end
```

###> Checking for a Package Configuration

The `check_cfg` method can be used to check for the existence of a package as
well as import any build options (e.g. include path, defines, libraries to link
against, etc...) required to use the package.

This method takes a Hash of options as its only argument.

Example calls:

```ruby
configure do
  check_cfg package: "zlib"
  check_cfg program: "freetype-config", fail: false, set_define: "HAVE_FREETYPE"
end
```

#### Options

##### `:package`

If the `:package` option is set to a value, the `pkg-config` program will be
used to look for package configuration flags for the specified package.

##### `:program`

If the `:program` option is given, the program specified will be used to look
for configuration flags.

##### `:fail`

If the `:fail` option is set to `false`, then the absence of the package or
program requested will not result in the configure option failing.

##### `:set_define`

If set, a build define of the specified String will be added to the
`CPPDEFINES` construction variable array if the requested package is found.

##> Build Operations

The `build` block is used to create Environments and register build targets.
An Rscons build script would not be very useful without a `build` block.

Here is an example `build` block demonstrating how to register a build target:

```ruby
build do
  Environment.new do |env|
    env.Program("myprog.exe", glob("src/**/*.c"))
  end
end
```

This `Rsconscript` would build an executable called `myprog.exe` from all C
source files found recursively under the `src` directory.

###> Environments

An Environment includes:

  - a collection of construction variables
  - a collection of build hooks
  - a collection of user-registered build targets
  - a build root

All build targets must be registered within an `Environment`.
The Environment's build root is a directory created within the top-level
Rscons build directory.
By default it holds all intermediate files generated by Rscons that are needed
to produce a user-specified build target.
For example, for the `Rsconscript`:

```ruby
build do
  Environment.new do |env|
    env.Program("myprog.exe", glob("src/**/*.c"))
  end
end
```

Rscons will place an object file and dependency file corresponding to each C
source file under the Environment's build root.
This keeps the intermediate generated build artifacts separate from the source
files.

###> Specifying Source Files: The glob Method

The `glob` method can be used to find files matching the patterns specified.
It supports a syntax similar to the Ruby [Dir.glob method](https://ruby-doc.org/core-2.5.1/Dir.html#method-c-glob) but operates more deterministically.

Example use:

```ruby
build do
  Environment.new do |env|
    env.Program("mytests", glob("src/**/*.cc", "test/**/*.cc"))
  end
end
```

This example would build the `mytests` executable from all `.cc` source files
found recursively under the `src` or `test` directory.

###> Construction Variables

Construction variables are values assigned to keys within an Environment.
Construction variables are used by Builders to produce output files.
See ${#Default Construction Variables} for a reference of all built-in
construction variables.

Example:

```ruby
build do
  Environment.new do |env|
    env["CCFLAGS"] += %w[-O2 -Wall]
    env["LIBS"] += %w[m]
  end
end
```

This example modifies the `CCFLAGS` construction variable to add `-O2` and
`-Wall` to the compilation commands used for C and C++ source files.
It also instructs the linker to link against the `m` library.

###> Builders

Rscons uses builder objects to produce *target* output files from *source*
input files.
A build target to be built using a builder is registered by calling a method on
the `Environment` object that matches the builder's name.
For example, a `Program` build target is registered by calling the
`env.Program` method.

The general syntax for registering a build target using a builder is:

```ruby
env.BuilderName(target, sources, vars = {})
```

The `target` parameter is the path to the output file or directory.
The `sources` parameter is the path or paths to the input file(s) to be used
by the builder.
In the `target` and `sources` parameters, the user can explicitly refer to a
path within the Environment's build root by beginning the path with "^/".
The `vars` parameter is an optional Hash which can include construction
variables to be used for this build target.
Any construction variable values specified in this parameter will override
those assigned to the Environment.

There are several default builders that are built-in to Rscons:

  * `Command`, which executes a user-defined command to produce the target.
  * `Copy`, which copies files or directories to a specified destination.
  * `CFile`, which builds a C or C++ source file from a lex or yacc input file.
  * `Directory`, which creates a directory.
  * `Disassemble`, which disassembles an object file to a disassembly listing.
  * `Install`, which installs files or directories to a specified destination.
  * `InstallDirectory`, which creates a directory during an install operation.
  * `Library`, which collects object files into a static library archive file.
  * `Object`, which compiles source files to produce an object file.
  * `Preprocess`, which invokes the C/C++ preprocessor on a source file.
  * `Program`, which links object files to produce an executable.
  * `SharedLibrary`, which links object files to produce a dynamically loadable
    library.
  * `SharedObject`, which compiles source files to produce an object file, in a
    way that is able to be used to create a shared library.

####> The Command Builder

```ruby
env.Command(target, sources, "CMD" => command)
# Example
env.Command("docs.html", "docs.md",
  "CMD" => ["pandoc", "-fmarkdown", "-thtml", "-o${_TARGET}", "${_SOURCES}"],
  "CMD_DESC" => "PANDOC")
```

The `Command` builder executes a user-defined command in order to produce the
desired target file based on the provided source files.

####> The CFile Builder

```ruby
env.CFile(target, source)
# Example
env.CFile("^/parser/parser.c", "parser.y")
```

The `CFile` builder will generate a C or C++ source file from a lex (.l, .ll)
or yacc (.y, .yy) input file.

####> The Copy Builder

```ruby
env.Copy(destination, sources)
# Example
env.Copy("mytests", "^/mytests")
env.Copy("^/dist/share", "share")
```

The `Copy` builder can copy files or directories to a target location.

####> The Directory Builder

```ruby
env.Directory(target)
# Example
env.Directory("^/tests")
```

The `Directory` builder can be used to explicitly create a directory.
This can also disambiguate whether the target for a subsequent builder
(e.g. `Copy`) refers to a file path or directory path.

####> The Disassemble Builder

```ruby
env.Disassemble(target, source)
# Example
env.Disassemble("module.dis", "module.o")
```

The `Disassemble` builder generates a disassembly listing using objdump from
and object file.

####> The Install Builder

```ruby
env.Install(destination, sources)
# Example
env.Install("${prefix}/bin", "app.exe")
env.Install("${prefix}/share", "share")
```

The `Install` builder can install files or directories to their installation
target location.
`Install` builders are only processed when the user has requested to perform
an `install` operation from the command line.

####> The InstallDirectory Builder

```ruby
env.InstallDirectory(target)
# Example
env.InstallDirectory("${prefix}/share")
```

The `InstallDirectory` builder can be used to explicitly create a directory.
`InstallDirectory` builders are only processed when the user has requested to
perform an `install` operation from the command line.
This can also disambiguate whether the target for a subsequent builder
(e.g. `Install`) refers to a file path or directory path.

####> The Library Builder

```ruby
env.Library(target, sources)
# Example
env.Library("lib.a", Rscons.glob("src/**/*.c"))
```

The `Library` builder creates a static library archive from the given source
files.

####> The Object Builder

```ruby
env.Object(target, sources)
# Example
env.Object("module.o", "module.c")
```

The `Object` builder compiles the given sources to an object file.
Although it can be called explicitly, it is more commonly implicitly called by
the `Program` builder.

####> The Preprocess Builder

```ruby
env.Preprocess(target, source)
# Example
env.Preprocess("module-preprocessed.cc", "module.cc")
```

The `Preprocess` builder invokes either `${CC}` or `${CXX}` (depending on if
the source contains an extension in `${CXXSUFFIX}` or not) and writes the
preprocessed output to the target file.

####> The Program Builder

```ruby
env.Program(target, sources)
# Example
env.Program("myprog", Rscons.glob("src/**/*.cc"))
```

The `Program` builder compiles and links the given sources to an executable
file.
Object files, static library files, or source files can be given as `sources`.
A platform-dependent program suffix will be appended to the target name if one
is not specified.
This can be controlled with the `PROGSUFFIX` construction variable.

####> The SharedLibrary Builder

```ruby
env.SharedLibrary(target, sources)
# Example
env.SharedLibrary("mydll", Rscons.glob("src/**/*.cc"))
```

The `SharedLibrary` builder compiles and links the given sources to a
dynamically loadable library.
Object files or source files can be given as `sources`.
A platform-dependent prefix and suffix will be appended to the target name if
they are not specified by the user.
These values can be controlled by overriding the `SHLIBPREFIX` and
`SHLIBSUFFIX` construction variables.

####> The SharedObject Builder

```ruby
env.SharedObject(target, sources)
# Example
env.SharedObject("lib_module.o", "lib_module.c")
```

The `SharedObject` builder compiles the given sources to an object file.
Any compilation flags necessary to build the object file in a manner that
allows it to be used to create a shared library are added.
Although it can be called explicitly, it is more commonly implicitly called by
the `SharedLibrary` builder.

###> Build Hooks

A build hook is a Ruby block that is called whenever Rscons is about to invoke
a builder to produce a build target.
Rscons also supports post-build hooks which are called after the builder has
produced the build target.
A build hook can be used to modify construction variables depending on the
build target or source file names.

Example:

```ruby
build do
  Environment.new do |env|
    env["CFLAGS"] << "-Wall"
    env.add_build_hook do |builder|
      # Compile sources from under src/tests without the -Wall flag.
      if builder.sources.first =~ %r{src/tests/}
        builder.vars["CFLAGS"] -= %w[-Wall]
      end
    end
    env.Program("program.exe", glob("src/**/*.c"))
  end
end
```

This example script would compile all C sources under the `src` directory with
the `-Wall` flag except for sources under the `src/tests` directory.

##> Extending Rscons

### Adding New Languages

The `Object` and `SharedObject` builders that ship with Rscons have an API that
allows the user to register extra languages that can be suppored by the
builders.
In fact, the built-in support for assembly, C, C++, and D compilation all make
use of this built-in API.
To see an example of how this API is used, see the
`lib/rscons/builders/lang/*.rb` files in the Rscons source repository.
For example, here is how the C++ language is registered:

```ruby
Rscons::Builders::Object.register(command: "${CXXCMD}", direct_command: "${CXXCMD:direct}", suffix: "${CXXSUFFIX}", preferred_ld: "${CXX}")
Rscons::Builders::SharedObject.register(command: "${SHCXXCMD}", direct_command: "${SHCXXCMD:direct}", suffix: "${CXXSUFFIX}", preferred_ld: "${SHCXX}")
```

There are also default construction variables registered to go along with the
language registration as specified above.
New default construction variables can be registered globally by assigning to
the `Rscons::DEFAULT_CONSTRUCTION_VARIABLES` Hash.
For example:

```ruby
Rscons::DEFAULT_CONSTRUCTION_VARIABLES["CXXCMD"] = %w[${CXX} -c -o ${_TARGET} ${CXXDEPGEN} ${INCPREFIX}${CPPPATH} ${CPPFLAGS} ${CXXFLAGS} ${CCFLAGS} ${_SOURCES}]
```

### Adding New Builders

It is also possible to extend Rscons with new builders.
This is the most flexible method to extend Rscons.
Builders can execute a command line program, call another builder, or just use
plain Ruby code to produce an output file.

#> Reference

## Default Construction Variables

#> License

Rscons is licensed under the terms of the MIT License:

```
${include LICENSE.txt}
```

#> Change Log

${changelog}