#> Overview Rscons is an open-source build system for developers. It supports the following features: * multi-threaded job execution * auto-configuration * built-in builders for several common operations * out-of-the-box support for C, C++, and D languages * extensibility for other languages or custom builders * compatible with Windows, Linux, OS X, and FreeBSD * colorized output with build progress * build hooks * user-defined tasks with dependencies and custom parameters * build variants At its core, Rscons is mainly an engine to: * determine the proper order to perform build steps, * determine whether each build target is up to date or in need of rebuild, and * schedule those build steps across multiple threads as efficiently as possible. Along the way, Rscons provides a concise syntax for specifying common types of build steps, but also provides an extensible framework for performing custom build operations as well. Rscons takes inspiration from: * [SCons](https://scons.org/) * [waf](https://waf.io/) * [rake](https://github.com/ruby/rake) * [CMake](https://cmake.org/) * [Autoconf](https://www.gnu.org/software/autoconf/) Rscons is written in Ruby. The only requirement to run Rscons is that the system has a Ruby interpreter installed. ${remove} WARNING: This user guide is meant to be preprocessed and rendered by a custom script. The markdown source file is not intended to be viewed directly and will not include all intended content. Visit [https://holtrop.github.io/rscons/index.html](https://holtrop.github.io/rscons/index.html) to view the properly rendered version. ${/remove} ## Design Principles ### Build Correctness The number one design principle in Rscons is build correctness. This means that a target will be built when Rscons cannot determine that a build target is already up-to-date. A build target will be built whenever: * the target file has been removed or changed since it was last built * the command to build the target file is different from the previous command used to build it * any of the target file's dependency files have changed since the last time the target was built Importantly, Rscons uses the content of a source (dependency) file to determine whether a rebuild is necessary, not simply the timestamp of the file. This is because relying solely on the timestamp of the file can lead to an incorrect decision being made to not rebuild when a rebuild is necessary. ### Build Flexibility Rscons supports multiple configurations of compilation flags or build options across multiple environments or build variants to build output files in different ways according to the user's desire. For example, the same source files can be built into a release executable, but also compiled with different compilation flags or build options into a test executable. Rscons also supports build hooks, which allow the user to further fine-tune the build system's operation. A build hook, for example, can be used to set a build option for only source files coming from a particular source directory. ### Build Efficiency Rscons will automatically determine the number of threads to use based on the host CPU configuration, and will schedule jobs as efficiently as possible across the available threads in order to complete the build in as little time as possible. As development occurs and builders are executed, Rscons makes use of a cache file in order to avoid rebuilding a target when it is already up to date. ### Build Directory Rscons was designed to store temporary build artifacts (for example, object files, dependency files, etc...) and build system metadata in a "build directory". This keeps files generated by the build cleanly separated from user-controlled source files. In contrast to other build systems or build system generators, rscons executes from the project base directory (up to the user) rather than executing from *within* the build directory. This keeps any file paths printed by compilers (such as in warning or error messages) accurate relative to the project directory, so that the user does not need to translate any paths to the correct path within a terminal or editor application, for example. By default a build directory named "build" is used, but this can be overridden by the user by using the `-b`/`--build` command-line option. ## Getting Started To use Rscons on your project, you must: 1. Install the `rscons` script in your project (See ${#Installation}). 2. Write the `Rsconscript` build script for your project (See ${#The Build Script}). 3. Use the `rscons` command in your project (See ${#Command-Line Operation}). #> Installation Rscons is designed to be distributed as a stand-alone single file script that can be copied into and versioned in a project's source tree. The only requirement to run Rscons is that the system has a Ruby interpreter installed. The latest release can be downloaded from [https://github.com/holtrop/rscons/releases](https://github.com/holtrop/rscons/releases). Simply copy the `rscons` executable script into the desired location within the project to be built (typically the root of the repository) and mark it executable. ## Version Control Setup The following files should be added to source control: * `rscons` * `Rsconscript` Add the following contents to `.gitignore` (or the equivalent thereof for different version control systems): ``` /.rscons* /build/ ``` #> Command-Line Operation Rscons is typically invoked from the command-line as `./rscons`. Usage: ./rscons [global options] [[task] [task options] ...] Global options: -A, --all Show all tasks (even those without descriptions) in task list. Use in conjunction with the -T argument. -b BUILD, --build=BUILD Set build directory (default: build). -e VS, --variants=VS Enable or disable variants. VS is a comma-separated list of variant entries. If the entry begins with "-" the variant is disabled instead of enabled. If the full list begins with "+" or "-" then it modifies the variants that are enabled by default by only enabling or disabling the listed variants. Otherwise, the enabled set of variants is as given and any variants not listed are disabled. The set of enabled variants is remembered from when the project is configured. -f FILE Use FILE as Rsconscript. -F, --show-failure Show failed command log from previous build and exit (does not load build script). -h, --help Show rscons help and exit (does not load build script). -j N, --nthreads=N Set number of threads (local default: 16). -r COLOR, --color=COLOR Set color mode (off, auto, force). -T, --tasks Show task list and parameters and exit (loads build script). By default only tasks with a description are listed. Use -AT to show all tasks whether they have a description or not. -v, --verbose Run verbosely. This causes Rscons to print the full build command used by each builder. --version Show rscons version and exit (does not load build script). The user can list any number of tasks on the command line. Any parameters beginning with a "-" that follow a task are interpreted as task arguments until another parameter is seen that does not begin with "-". For example: ./rscons -v build1 --build1-opt=val1 --flag build2 The above command line is interpreted as follows: * The user is passing the -v global option to run verbosely. * The user requests to run task build1 with task parameters "--build1-opt=val1" and "--flag". * The user requests to run task build2 with no task parameters. If no tasks are specified on the command line, Rscons executes the `default` task. If a task fails due to a command failure (e.g. compilation or linking failed), Rscons will log the failed commands. By default Rscons does not print the failed commands to the console so that it is easier for the user to focus on the actual compiler failure messages rather than the compilation command itself. The user can run `./rscons -F` to see the command that failed on the prior Rscons execution. The user can also invoke Rscons with the `-v` global command-line option which will cause Rscons to print each command it is executing. ## Rscons Operation Phases When Rscons executes, it performs the following phases: - Parse the command line. - This is the last step if --help, --version, or --show-failure is specified. - Load the build script. - This is the last step if --tasks is specified. - Configure the project by running the `configure` task if necessary (the project has not yet been configured, autoconf is set to true, and the user is requesting to execute a task that is marked with autoconf set to true) - Execute user-requested tasks. #> The Build Script Rscons looks for instructions for what to build by reading a build script file called `Rsconscript` (or `Rsconscript.rb`). Here is a simple example `Rsconscript` file: ```ruby env do |env| env.Program("myprog.exe", glob("src/**/*.c")) end ``` This `Rsconscript` file would instruct Rscons to produce a *Program* target called `myprog.exe` which is to be built from all C source files found (recursively) under the `src` directory. The `Rsconscript` file is a Ruby script. ##> Tasks Tasks are a high-level user interface for performing functionality in a build script. Tasks can create Environments that perform compilation/linking steps. Tasks can also execute arbitrary commands or perform any miscellaneous logic. Tasks can have dependencies, which are specified as names of other tasks that should be executed before this task executes. Tasks can have action blocks. When a task is executed, all of its action blocks are called in the order in which they were added. Example: ```ruby task "build" do env do |env| env.Program("^^/proj.elf", glob("src/**/*.c")) end end task "flash", deps: "build" do sh "nrfjprog", "-f", "NRF52", "--program", env.expand("^^/proj.elf") end ``` In this example, the `flash` task depends on the `build` task. So if the project had not yet been built, and the user executes `./rscons flash`, the project would first be built and then flashed to the target. If the `task` method is called again with the name of an already existing task, the task is not overwritten, but rather modified. Any newly specified dependencies are added to the current dependencies. Any action block is appended to the task's list of action blocks to execute when the task is executed. Note that for a simple project, the build script may not need to define any tasks at all and could just make use of the Rscons built-in default task (see ${#Default Task}). ###> Task Parameters Tasks can accept parameters. Parameters are defined by the build script author, and have default values. The user can override parameter values on the command line. Task parameters are defined by passing a parameter constructed with the Rscons `param()` method to the `:params` argument of the `task()` method. The signature of the `param` method is: ```ruby def param(name, value, takes_arg, description) ``` For example: ```ruby task "build", params: [ param("myparam", "defaultvalue", true, "My special parameter"), param("xyz", nil, false, "Enable the xyz feature"), ] do |task, params| env do |env| env["CPPDEFINES"] << "SOMEMACRO=#{params["myparam"]}" if params["flag"] env["CPPDEFINES"] << "ENABLE_FEATURE_XYZ" end end end ``` With the above `Rsconscript`, the user could invoke Rscons as: ./rscons build --myparam=pvalue --xyz This would pass in "pvalue" as the value to the "myparam" parameter, and a truthy value ("--xyz") as the value of the "xyz" parameter. As seen above, task parameter values can be accessed within a task's action block by using the second parameter (`params`) to the action block. Task parameter values can also be accessed with the `Task#[]` method on any task object. This allows accessing the parameter values of any task object, not just the task owning the action block being executed. Example: ```ruby task "one", params: param("flag", nil, false, "Enable a flag") task "two" do puts "Task one's flag #{Task["one"]["flag"] ? "is" : "is not"} set" end ``` Task parameters can also be referenced via construction variables. Each task parameter is stored in a construction variable. The name for the construction variable is created by joining the task name and the parameter name with a ":" character. For example: ```ruby task "build", params: [ param("heap-size", "1024", true, "Set heap size"), ] do env["CPPDEFINES"] << "HEAP_SIZE=${build:heap-size}" env.Program("^/myprog", glob("src/**/*.c")) env.Install("${configure:prefix}/bin/myprog", "^/myprog") end ``` ###> Tasks with Special Meaning Rscons recognizes special meaning for a few tasks: * configure * default * clean * distclean * install * uninstall For each of these tasks, a shortcut method of the same name as the task is provided which is equivalent to calling the `task()` method with the first argument (task name) automatically filled in by the shortcut method. For example: ```ruby default deps: "unpack_compiler" do puts "default task" end ``` is equivalent to: ```ruby task "default", deps: "unpack_compiler" do puts "default task" end ``` ####> Configure Task The `configure` task allows Rscons to perform any one-time setup operations required by a project, for example locating compilers and setting any initial construction variable values based on the host environment in use. It will also perform any configuration checks requested by the build script. Such configuration checks can include: * verifying operation of a compiler * loading compilation/linker flags from a config program (e.g. `pkg-config`) * verifying presence of a C/C++ header file * verifying presence of a D import * verifying presence of a library * verifying presence of an executable * any custom user-supplied configuration check The configure task is implicitly a dependency of every other task unless that task is configured with its `autoconf` option set to `false`. The global build script `autoconf` setting can also be set to `false` to disable automatic invocation of the configure task. For example: ```ruby autoconf false configure do puts "configure" end default do puts "default" end ``` With the above Rsconscript, even if the project has not yet been configured, a configure operation would not take place when the default task is executed. The user would have to explicitly request the configure task from the command line. The build script method `project_name` can be used to set the project name which will be reported to the user during a configure operation. For example: ```ruby project_name "My awesome project" configure do check_d_compiler end ``` See ${#Configuring the Project} for more details on how to make use of the configuration functionality that Rscons provides. ####> Default Task The `default` task is special in that Rscons will execute it if no other task has been requested by the user on the command line. The default task can also be used to declare a dependency on another task that would effectively become the default. For example: ```ruby task "build" do ... end task "flash" do ... end default deps: "build" ``` Then when the user runs `./rscons` the "build" task will be executed. ####> Clean Task The `clean` task is built-in to Rscons. It removes all built target files. It will not remove items installed by an Install builder. It will not remove the cached configuration options. ####> Distclean Task The `distclean` task is built-in to Rscons. It removes all built target files and all cached configuration options. Generally it will get the project directory back to the state it was in when unpacked or checked out, before any configuration or build operations took place. It will not remove items installed by an Install builder. ####> Install Task The `install` task is not built-in to Rscons but rather is just a convention for the build script author to use. The suggested use is for the `install` task to invoke any `Install` or `InstallDirectory` builders to install items into the specified installation directory. The `install` shortcut method can be used. For example: ```ruby install do env.Install("${prefix}/bin", "app.exe") env.Install("${prefix}/share", "share") end ``` ####> Uninstall Task The `uninstall` task is built-in to Rscons. It removes any items installed by an Install builder. It will not remove all built target files, just the installed copies. ##> Configuring the Project Configure task actions can be used to perform various checks and setup operations for a project. Example `configure` action block: ```ruby configure do check_cxx_compiler check_c_header "getopt.h" end ``` If any configure task action blocks are present, they will be execute when the configure operation is performed. This happens if the user requests the `configure` task from the command line. It also happens if all of the following are true: * The project has not yet been configured. * A task that does not have `autoconf` set to `false` is being executed. * The global `autoconf` setting has not been set to `false`. See ${#Configure Task} for more information about `autoconf`. ###> Checking for a Compiler The following methods can be used within a `configure` block to check for a working compiler: * `check_c_compiler` * `check_cxx_compiler` * `check_d_compiler` Each of these methods can take an optional list of compilers to check for. If such a list is supplied, the compilers are tested in the order listed. The first compiler option found which passes a compilation test is used. Here are example calls which also show the default compiler list for each supported language: ```ruby configure do check_c_compiler "gcc", "clang" check_cxx_compiler "g++", "clang++" check_d_compiler "gdc", "ldc2" end ``` Global configuration options may be supplied to the compiler checks as well. Example: ```ruby configure do check_c_compiler "x86_64-elf-gcc", on_fail: "Install x86_64-elf cross toolchain first!" end ``` ###> Checking for a Header File The following methods can be used to check for the presence of a header file: * `check_c_header` will check for a C header to be present * `check_cxx_header` will check for a C++ header to be present Each of these methods take the name of the header file to check for as the first argument, and take an optional Hash of arguments as the second argument. Example calls: ```ruby configure do check_c_header "getopt.h", set_define: "HAVE_GETOPT_H" check_c_header "FreeType2.h" check_cxx_header "memory" end ``` #### Options ##### `:check_cpppath` Optionally specifies an array of paths to look for the header file in. ###> Checking for a D Import The `check_d_import` method can be used to check for the presence of D import. This method takes the name of the import to check for as the first argument. Example calls: ```ruby configure do check_d_import "std.stdio" check_d_import "std.numeric" end ``` #### Options ##### `:check_d_import_path` Optionally specifies an array of paths to look for the module in. ###> Checking for a Library The `check_lib` method can be used to check for the presence of a library. This method takes the name of the library to check for as the first argument, and take an optional Hash of arguments as the second argument. Example calls: ```ruby configure do check_lib "kpty", fail: false, set_define: "HAVE_LIBKPTY" check_lib "GL" end ``` #### Options ##### `:check_libpath` Optionally specifies an array of paths to look for the library in. ##### `:use` If not set, the library will be used by default in all `Environment` objects. If set, the library will only be used in `Environment` objects that have a matching `:use` flag set. ###> Checking for a Program The `check_program` method can check for the existence of an executable in the host operating system environment. Example call: ```ruby configure do check_program "xxd" end ``` ###> Checking for a Package Configuration The `check_cfg` method can be used to check for the existence of a package as well as import any build options (e.g. include path, defines, libraries to link against, etc...) required to use the package. This method takes a Hash of options as its only argument. Example calls: ```ruby configure do check_cfg package: "zlib" check_cfg program: "freetype-config", fail: false, set_define: "HAVE_FREETYPE" end ``` #### Options ##### `:package` If the `:package` option is set to a value, the `pkg-config` program will be used to look for package configuration flags for the specified package. ##### `:program` If the `:program` option is given, the program specified will be used to look for configuration flags. ##### `:use` If not set, the library will be used by default in all `Environment` objects. If set, the library will only be used in `Environment` objects that have a matching `:use` flag set. ###> Custom Configuration Checks The `Rsconscript` author can add custom configuration checks to be performed during the rscons `configure` operation. Here is an example from `build_tests/configure/custom_config_check.rb` showing a custom configuration check: ```ruby ${include build_tests/configure/custom_config_check.rb} ``` A custom configuration check is created by calling the `custom_check` method and passing a block. The contents of the block should perform the custom configuration checking logic. This logic can include executing a test command or other arbitrary operations. An argument `op` is passed to the block. This object is an instance of the [`ConfigureOp` class](../yard/Rscons/ConfigureOp.html) class and provides several methods that can be used to aid with the custom configuration check. The [`log_and_test_command`](../yard/Rscons/ConfigureOp.html#log_and_test_command-instance_method) method can be used to execute a test command and retrieve its results. The command and its output are also logged to the config.log file. The [`store_merge`](../yard/Rscons/ConfigureOp.html#store_merge-instance_method), [`store_append`](../yard/Rscons/ConfigureOp.html#store_append-instance_method), and [`store_parse`](../yard/Rscons/ConfigureOp.html#store_parse-instance_method) methods can be used to store construction variables to be used in Environments created later. Finally, the [`complete`](../yard/Rscons/ConfigureOp.html#complete-instance_method) method can be used to complete the configuration check and indicate a success or failure. While performing a custom configuration check, it can sometimes be useful to be able to construct an Environment to use the set of default construction variables as defined so far in the configuration block, for example to expand construction variables to build a test command. The normal `Environment` class cannot be used within the `configure` block, however the [`BasicEnvironment`](../yard/Rscons/BasicEnvironment.html) class can be used for such a purpose. For example, to expand the current `${CCCMD}` value: ```ruby configure do custom_check("Checking something to do with CCCMD") do command = BasicEnvironment.new.expand_varref("${CCCMD}") # ... end end ``` ###> Global Configuration Check Options #### `:fail` If the `:fail` option is set to `false`, then the absence of the package or program requested will not result in the configure option failing. The `:fail` option defaults to `true` if the `:set_define` option is not defined, and defaults to `false` if the `:set_define` option is defined. #### `:on_fail` The `:on_fail` option can be set to a String or a Proc object. If the configuration operation fails (or would fail), the given message is printed or the Proc is called. Examples: ```ruby configure do check_c_compiler "special-gcc", on_fail: "First install special gcc!" end configure do package_hint = lambda do puts "The following packages must be installed to build this project:" puts "- libsdl2-dev" puts "- libsdl2-image-dev" puts "- libsdl2-net-dev" end check_lib "SDL2", on_fail: package_hint check_lib "SDL2_image", on_fail: package_hint check_lib "SDL2_net", on_fail: package_hint end ``` #### `:set_define` If set, a build define of the specified String will be added to the `CPPDEFINES` construction variable array if the requested package is found. ##> Building Targets Building target files is accomplished by using Environments. Environments can be created at the top level of the build script, or from within a task action block. Environments are created with the [`env`](../yard/Rscons/Script/GlobalDsl.html#env-instance_method) build script method. Here is an example build script that creates an Environment and registers a Program build target: ```ruby env do |env| env.Program("myprog.exe", glob("src/**/*.c")) end ``` This `Rsconscript` would build an executable called `myprog.exe` from all C source files found recursively under the `src` directory. ###> Environments An Environment includes: - a name - a collection of construction variables - a collection of build hooks - a collection of user-registered build targets - a build root directory All build targets must be registered within an `Environment`. If the user does not specify a name for the environment, a name will be automatically generated based on the Environment's internal ID, for example "e.1". The Environment's build root is a directory with the same name as the Environment, created within the top-level Rscons build directory. By default it holds all intermediate files generated by Rscons that are needed to produce a user-specified build target. For example, for the `Rsconscript`: ```ruby env "myproj" do |env| env.Program("myprog.exe", glob("src/**/*.c")) end ``` Rscons will place an object file and dependency file corresponding to each C source file under the Environment's build root. Assuming a top-level build directory of "build", the Environment's build root would be "build/myproj". This keeps the intermediate generated build artifacts separate from the source files. Source and target paths passed to a Builder (e.g. Program) can begin with "^/" to indicate that Rscons should expand those paths to be relative to the Environment's build root. If a source or target path passed to a Builder begins with "^^/", it is expanded to be relative to the Rscons top-level build directory (but outside the Environment's build root). ###> Construction Variables Construction variables are values assigned to keys within an Environment. Construction variables are used by Builders to produce output files. See ${#Default Construction Variables} for a reference of all built-in construction variables. Example: ```ruby env do |env| env["CCFLAGS"] += %w[-O2 -Wall] env["LIBS"] += %w[m] end ``` This example modifies the `CCFLAGS` construction variable to add `-O2` and `-Wall` to the compilation commands used for C and C++ source files. It also instructs the linker to link against the `m` library. #### Construction Variable Naming * uppercase strings - the default construction variables that Rscons uses * strings beginning with "_" - set and used internally by builders * lowercase strings with a ":" as "#{task_name}:#{parameter_name}" - set to task parameter values * symbols, lowercase strings - reserved as user-defined construction variables ###> Builders Rscons uses builder objects to produce *target* output files from *source* input files. A build target to be built using a builder is registered by calling a method on the `Environment` object that matches the builder's name. For example, a `Program` build target is registered by calling the `env.Program` method. The general syntax for registering a build target using a builder is: ```ruby env.BuilderName(target, sources, vars = {}) ``` The `target` parameter is the path to the output file or directory. The `sources` parameter is the path or paths to the input file(s) to be used by the builder. In the `target` and `sources` parameters, the user can explicitly refer to a path within the Environment's build root by beginning the path with "^/", or to a path within the Rscons top-level built directory by beginning with "^^/". The `vars` parameter is an optional Hash which can include construction variables to be used for this build target. Any construction variable values specified in this parameter will override those assigned to the Environment. There are several default builders that are built-in to Rscons: * `Command`, which executes a user-defined command to produce the target. * `Copy`, which copies files or directories to a specified destination. * `CFile`, which builds a C or C++ source file from a lex or yacc input file. * `Directory`, which creates a directory. * `Disassemble`, which disassembles an object file to a disassembly listing. * `Install`, which installs files or directories to a specified destination. * `InstallDirectory`, which creates a directory in an install destination. * `Library`, which collects object files into a static library archive file. * `Object`, which compiles source files to produce an object file. * `Preprocess`, which invokes the C/C++ preprocessor on a source file. * `Program`, which links object files to produce an executable. * `SharedLibrary`, which links object files to produce a dynamically loadable library. * `SharedObject`, which compiles source files to produce an object file, in a way that is able to be used to create a shared library. * `Size`, which runs the 'size' utility on an executable file. ####> The Command Builder ```ruby env.Command(target, sources, "CMD" => command) # Example env.Command("user_guide.html", "user_guide.md", "CMD" => ["pandoc", "-fmarkdown", "-thtml", "-o${_TARGET}", "${_SOURCES}"], "CMD_DESC" => "Generating user guide:") ``` The `Command` builder executes a user-defined command in order to produce the desired target file based on the provided source files. The `Command` builder supports the following construction variables: * `CMD` (required) specifies the command to execute (an array of strings). `CMD` is expanded for variable references, so the tokens `${_TARGET}` and `${_SOURCES}` can be used, for example. * `CMD_DESC` (optional) specifies the short text description to print when the builder executes. The given description is followed by the target file name. * `CMD_STDOUT` (optional) specifies a file to redirect standard output to. `CMD_STDOUT` is expanded for variable references, so the token `${_TARGET}` can be used, for example. ####> The CFile Builder ```ruby env.CFile(target, source) # Example env.CFile("^/parser/parser.c", "parser.y") ``` The `CFile` builder will generate a C or C++ source file from a lex (.l, .ll) or yacc (.y, .yy) input file. ####> The Copy Builder ```ruby env.Copy(destination, sources) # Example env.Copy("mytests", "^/mytests") env.Copy("^/dist/share", "share") ``` The `Copy` builder can copy files or directories to a target location. ####> The Directory Builder ```ruby env.Directory(target) # Example env.Directory("^/tests") ``` The `Directory` builder can be used to explicitly create a directory. This can also disambiguate whether the target for a subsequent builder (e.g. `Copy`) refers to a file path or directory path. ####> The Disassemble Builder ```ruby env.Disassemble(target, source) # Example env.Disassemble("module.dis", "module.o") ``` The `Disassemble` builder generates a disassembly listing using objdump from and object file. ####> The Install Builder ```ruby env.Install(destination, sources) # Example env.Install("${prefix}/bin", "app.exe") env.Install("${prefix}/share", "share") ``` The `Install` builder can install files or directories to their installation target location. It functions almost identically to the `Copy` builder. The only difference relates to the `clean` and `uninstall` tasks. The `clean` task removes targets created by the `Copy` builder but not by the `Install` builder. The `uninstall` task removes targets created by the `Install` builder but not by the `Copy` builder. ####> The InstallDirectory Builder ```ruby env.InstallDirectory(target) # Example env.InstallDirectory("${prefix}/share") ``` The `InstallDirectory` builder can be used to explicitly create a directory in an installation location. This can also disambiguate whether the target for a subsequent builder (e.g. `Install`) refers to a file path or directory path. It functions almost identically to the `Directory` builder. The only difference relates to the `clean` and `uninstall` tasks. The `clean` task removes targets created by the `Directory` builder but not by the `InstallDirectory` builder. The `uninstall` task removes targets created by the `InstallDirectory` builder but not by the `Directory` builder. ####> The Library Builder ```ruby env.Library(target, sources) # Example env.Library("lib.a", Rscons.glob("src/**/*.c")) ``` The `Library` builder creates a static library archive from the given source files. ####> The Object Builder ```ruby env.Object(target, sources) # Example env.Object("module.o", "module.c") ``` The `Object` builder compiles the given sources to an object file. Although it can be called explicitly, it is more commonly implicitly called by the `Program` builder. ####> The Preprocess Builder ```ruby env.Preprocess(target, source) # Example env.Preprocess("module-preprocessed.cc", "module.cc") ``` The `Preprocess` builder invokes either `${CC}` or `${CXX}` (depending on if the source contains an extension in `${CXXSUFFIX}` or not) and writes the preprocessed output to the target file. ####> The Program Builder ```ruby env.Program(target, sources) # Example env.Program("myprog", Rscons.glob("src/**/*.cc")) ``` The `Program` builder compiles and links the given sources to an executable file. Object files, static library files, or source files can be given as `sources`. A platform-dependent program suffix will be appended to the target name if one is not specified. This can be controlled with the `PROGSUFFIX` construction variable. ##### Direct Mode The Program builder supports a "direct" mode which is activated by specifying the `:direct` option. In the direct mode, all source files are passed directly to the compiler together and compiled and linked in one step, rather than being individually compiled to separate object files first. This mode allows taking advantage of any multi-file compilation capabilities of the compiler. However, it also requires recompiling all source files when any one of them has changed. Example use: ```ruby env.Program("myprog", Rscons.glob("src/**/*.c"), direct: true) ``` ####> The SharedLibrary Builder ```ruby env.SharedLibrary(target, sources) # Example env.SharedLibrary("mydll", Rscons.glob("src/**/*.cc")) ``` The `SharedLibrary` builder compiles and links the given sources to a dynamically loadable library. Object files or source files can be given as `sources`. A platform-dependent prefix and suffix will be appended to the target name if they are not specified by the user. These values can be controlled by overriding the `SHLIBPREFIX` and `SHLIBSUFFIX` construction variables. ##### Direct Mode The SharedLibrary builder supports a "direct" mode which is activated by specifying the `:direct` option. In the direct mode, all source files are passed directly to the compiler together and compiled and linked in one step, rather than being individually compiled to separate object files first. This mode allows taking advantage of any multi-file compilation capabilities of the compiler. However, it also requires recompiling all source files when any one of them has changed. Example use: ```ruby env.SharedLibrary("mydll", Rscons.glob("src/**/*.c"), direct: true) ``` ####> The SharedObject Builder ```ruby env.SharedObject(target, sources) # Example env.SharedObject("lib_module.o", "lib_module.c") ``` The `SharedObject` builder compiles the given sources to an object file. Any compilation flags necessary to build the object file in a manner that allows it to be used to create a shared library are added. Although it can be called explicitly, it is more commonly implicitly called by the `SharedLibrary` builder. ####> The Size Builder ```ruby env.Size(target, sources) # Example env.Program("program.exe", glob("*.c")) env.Size("program.size", "program.exe") ``` The `Size` builder runs the "size" executable on the given source file and stores its output in the target file. The size executable can be specified with the `SIZE` construction variable, and flags can be specified with `SIZEFLAGS`. ###> Phony Targets Rscons supports phony build targets. Normally, a builder produces an output file, and executes whenever the input files or command have changed. A phony build target can be used to register a builder that does not produce an output file. A custom builder can take some action when the input files change even if it does not produce an output file. Such a builder could perform verification or run a test on its source files, possibly failing if some conditions are not met. It could also simply output something to the console, such as an analysis of the source file, whenever it changes. A phony target is signified by passing a Symbol instead of a String as the first parameter (target) to a builder method. ###> Explicit Dependencies A target can be marked as depending on another file that Rscons would not otherwise know about via the `Environment#depends` function. For example, to force the linker to re-link a Program output when a linker script changes: ```ruby env.Program("a.out", "foo.c", "LDFLAGS" => %w[-T linker_script.ld]) env.depends("a.out", "linker_script.ld") ``` You can pass multiple dependency files to `Environment#depends`: ```ruby env.depends("my_app", "config/link.ld", "README.txt", *glob("assets/**/*")) ``` ###> Build Hooks A build hook is a Ruby block that is called whenever Rscons is about to invoke a builder to produce a build target. Rscons also supports post-build hooks which are called after the builder has produced the build target. A build hook can be used to modify construction variables depending on the build target or source file names. Example: ```ruby env do |env| env["CFLAGS"] << "-Wall" env.add_build_hook do |builder| # Compile sources from under src/tests without the -Wall flag. if builder.sources.first =~ %r{src/tests/} builder.vars["CFLAGS"] -= %w[-Wall] end end env.Program("program.exe", glob("src/**/*.c")) end ``` This example script would compile all C sources under the `src` directory with the `-Wall` flag except for sources under the `src/tests` directory. A post-build hook can be added with `env.add_post_build_hook`. Post-build hooks are only invoked if the build step was a success. Build hooks and post-build hooks can register new build targets. ###> Barriers Normally Rscons will parallelize all builders executed within an Environment. A barrier can be used to separate sets of build targets. All build targets registered before the barrier is created will be built before Rscons will schedule any build targets after the barrier. In other words, build targets are not parallelized across a barrier. ```ruby env.barrier ``` ##> Variants Rscons supports build variants. Variants can be used to built multiple variations of the same item with a specific change. For example, a desktop application with the same sources could be built to target KDE or GNOME using build variants. It is up to the build script author to define the variants and what effect they have on the build. This build script defines "kde" and "gnome" variants: ```ruby variant "kde" variant "gnome" with_variants do env "prog" do |env| if variant "kde" env["CPPDEFINES"] << "KDE" end if variant "gnome" env["CPPDEFINES"] << "GNOME" end env.Program("^/prog.exe", "src/**/*.cpp") end end ``` The `variant` build script method has two uses: * At the top of the build script, it defines a variant. * Within a `with_variants` block, it queries for whether the given variant is active. The `with_variants` build script method allows the power of variants to be harnessed. It iterates through each enabled variant and calls the given block. In this example, the block would be called twice, once with the "kde" variant active, and the second time with the "gnome" variant active. Each `env()` call creates an Environment, so two environments are created. When an Environment is created within a `with_variants` block, the Environment's name has the active variant(s) appended to the given Environment name (if any), and separated by a "-". In this example, a "prog-kde" Environment would be created with build root build/prog-kde and -DKDE would be passed to the compiler when compiling each source. Next a "prog-gnome" Environment would be created with build root build/prog-gnome and -DGNOME would be passed to the compiler when compiling the sources. Variants are enabled by default, but can be disabled by passing a `false` value to the `:default` option of the `variant` method. For example: ```ruby ${include build_tests/variants/default.rb} ``` The `rscons` command line interface provides a `-e`/`--variants` argument which allows the user to enable a different set of variants from those enabled by default according to the build script author. This argument accepts a comma-separated list of variants to enable. Each entry in the list can begin with "-" to disable the variant instead of enable it. If the list begins with "+" or "-", then the entire given list modifies the defaults given in the build script. Otherwise, it exactly specifies which variants should be enabled, and any variant not listed is disabled. When the project is configured, the set of enabled variants is recorded and remembered for later Rscons invocations. This way, a user working on a single variant of a project does not need to specify the `-e`/`--variants` option on each build operation. The `variant_enabled?` build script method can be called to query whether the given variant is enabled. ###> Variant Groups Variants may be grouped, which allows the build script author to define multiple combinations of desired variations to build with. For example: ```ruby ${include build_tests/variants/multiple_groups.rb} ``` This build script executes the block given to `with_variants` four times and results in four Environments being created: * prog-kde-debug * prog-kde-release * prog-gnome-debug * prog-gnome-release The command `./rscons -e-debug` would build just "prog-kde-release" and "prog-gnome-release". The command `./rscons --variants kde,release` would build just "prog-kde-release". ##> Build Script Methods `rscons` provides several methods that a build script can use. * `autoconf` (see ${#Configure Task}) * `build_dir` which returns the path to the top-level Rscons build directory * `clean` (see ${#Clean Task}) * `configure` (see ${#Configure Task}) * `default` (see ${#Default Task}) * `download` (see ${#Downloading Files: The download Method}) * `distclean` (see ${#Distclean Task}) * `glob` (see ${#Finding Files: The glob Method}) * `install` (see ${#Install Task}) * `param` (see ${#Task Parameters}) * `path_append` (see ${#PATH Management}) * `path_components` (see ${#PATH Management}) * `path_prepend` (see ${#PATH Management}) * `path_set` (see ${#PATH Management}) * `project_name` (see ${#Configure Task}) * `rscons` (see ${#Using Subsidiary Build Scripts: The rscons Method}) * `sh` (see (${#Executing Commands: The sh Method}) * `task` (see ${#Tasks}) * `uninstall` (see ${#Uninstall Task}) * `variant` (see ${#Variants}) * `variant_enabled?` (see ${#Variant Groups}) * `variant_group` (see ${#Variant Groups}) * `with_variants` (see ${#Variant Groups}) Additionally, the following methods from the Ruby [FileUtils](https://ruby-doc.org/stdlib-3.1.0/libdoc/fileutils/rdoc/FileUtils.html) module are made available for the build script to call directly: * `cd` * `chmod` * `chmod_R` * `chown` * `chown_R` * `cp` * `cp_lr` * `cp_r` * `install` * `ln` * `ln_s` * `ln_sf` * `mkdir` * `mkdir_p` * `mv` * `pwd` * `rm` * `rm_f` * `rm_r` * `rm_rf` * `rmdir` * `touch` ###> Finding Files: The glob Method The [`glob`](../yard/Rscons/Script/GlobalDsl.html#glob-instance_method) method can be used to find files matching the patterns specified. It supports a syntax similar to the Ruby [Dir.glob method](https://ruby-doc.org/core-3.1.0/Dir.html#method-c-glob) but operates more deterministically (results are ordered based on file names rather than file system directory ordering). Example use: ```ruby env do |env| env.Program("mytests", glob("src/**/*.cc", "test/**/*.cc")) end ``` This example would build the `mytests` executable from all `.cc` source files found recursively under the `src` or `test` directory. ###> Downloading Files: The download Method The [`download`](../yard/Rscons/Script/GlobalDsl.html#download-instance_method) method can be used to download a file from a given URL. Example use: ```ruby default do download "https://ftp.gnu.org/gnu/gcc/gcc-#{gcc_version}/gcc-#{gcc_version}.tar.xz", "#{build_dir}/gcc-#{gcc_version}.tar.xz", sha256_sum: gcc_checksum end ``` The `download` method downloads the file specified by the URL in the first parameter, and writes it to the local file specified by the second parameter. If the `:sha256sum` option is given, this causes two changes to the default behavior: * If Rscons finds the local file and it already has this checksum, then the file will not be downloaded again. * After downloading, Rscons verifies that the checksum matches what is given and exits with an error if it does not. ###> PATH Management `rscons` provides methods for management of the `PATH` environment variable. The [`path_append`](../yard/Rscons/Script/GlobalDsl.html#path_append-instance_method) and [`path_prepend`](../yard/Rscons/Script/GlobalDsl.html#path_prepend-instance_method) methods can be used to append or prepend a path to the `PATH` environment variable. ```ruby path_prepend "i686-elf-gcc/bin" ``` The [`path_set`](../yard/Rscons/Script/GlobalDsl.html#path_set-instance_method) method sets the `PATH` environment variable to the given Array or String. The [`path_components`](../yard/Rscons/Script/GlobalDsl.html#path_components-instance_method) method returns an Array of the components in the `PATH` environment variable. ###> Using Subsidiary Build Scripts: The rscons Method The [`rscons`](../yard/Rscons/Script/GlobalDsl.html#rscons-instance_method) build script method can be used to invoke an rscons subprocess using a subsidiary rscons build script. This can be used, for example, when a subproject is imported and a top-level `configure` or `build` task should also perform the same task in the subproject directory. The first argument to the `rscons` method specifies either a directory name, or the path to the subsidiary Rsconscript file to execute. Any additional arguments are passed to `rscons` when it executes the subsidiary script. `rscons` will change working directories to the directory containing the subsidiary script when executing it. For example: ```ruby configure do rscons "subproject", "configure" end task "build" do rscons "subproject/Rsconscript", "build" end ``` It is also perfectly valid to perform different task(s) in the subsidiary script from the one being performed in the top-level script. For example, in a project that requires a particular cross compiler, the top-level `configure` script could build the necessary cross compiler using a subsidiary build script. This could look something like: ```ruby configure do rscons "cross/Rsconscript" check_c_compiler "i686-elf-gcc" end ``` This would build, and if necessary first configure, using the cross/Rsconscript subsidiary build script. Subsidiary build scripts are executed from within the directory containing the build script. ###> Executing Commands: The sh Method The [`sh`](../yard/Rscons/Script/GlobalDsl.html#sh-instance_method) build script method can be used to directly execute commands. The `sh` method accepts either a single String argument or an Array of Strings. When an Array is given, if the array length is greater than 1, then the command will not be executed and interpreted by the system shell. Otherwise, it will be executed and interpreted by the system shell. For example: ```ruby default do # Run "make" in imported "subcomponent" directory. sh "cd subcomponent; make" # Move a file around. sh "mv", "subcomponent/file with spaces.txt", "new_name.txt" end ``` If the command fails, rscons will normally print the error and terminate execution. If the `:continue` option is set, then rscons will not terminate execution. For example: ```ruby default do # This command will fail and a message will be printed. sh "false", continue: true # However, due to the :continue option being set, execution will continue. sh "echo hi" end ``` Options that Ruby's `spawn` method accepts can also be passed in to the `sh` method. For example, to execute the given command with a different working directory, the `:chdir` option can be specified: ```ruby default do # Execute 'ls' from within the 'src' directory: sh "ls", chdir: "src" end ``` ##> Extending Rscons ### Adding New Languages The `Object` and `SharedObject` builders that ship with Rscons have an API that allows the user to register extra languages that can be suppored by the builders. In fact, the built-in support for assembly, C, C++, and D compilation all make use of this built-in API. To see an example of how this API is used, see the `lib/rscons/builders/lang/*.rb` files in the Rscons source repository. For example, here is how the C++ language is registered: ```ruby Rscons::Builders::Object.register(command: "${CXXCMD}", direct_command: "${CXXCMD:direct}", suffix: "${CXXSUFFIX}", preferred_ld: "${CXX}") Rscons::Builders::SharedObject.register(command: "${SHCXXCMD}", direct_command: "${SHCXXCMD:direct}", suffix: "${CXXSUFFIX}", preferred_ld: "${SHCXX}") ``` There are also default construction variables registered to go along with the language registration as specified above. New default construction variables can be registered globally by assigning to the `Rscons::DEFAULT_CONSTRUCTION_VARIABLES` Hash. For example: ```ruby Rscons::DEFAULT_CONSTRUCTION_VARIABLES["CXXCMD"] = %w[${CXX} -c -o ${_TARGET} ${CXXDEPGEN} ${INCPREFIX}${CPPPATH} ${CPPFLAGS} ${CXXFLAGS} ${CCFLAGS} ${_SOURCES}] ``` ###> Adding Custom Builders It is also possible to extend Rscons with new builders. This is the most flexible method to extend Rscons. Builders can execute a command line program, call another builder, or just use plain Ruby code to produce an output file. A builder is a class that inherits from the `Rscons::Builder` base class. Rscons provides a `Rscons::Builders` namespacing module which contains the built-in builder classes. User-provided custom builder classes can also reside in the `Rscons::Builders` namespacing module, but this is not required. ####> Adding a Custom Builder to an Environment The user can add a builder class to an Environment with the `env.add_builder` method. For example: ```ruby class Rscons::Builders::Mine < Rscons::Builder end default do env do |env| env.add_builder(Rscons::Builders::Mine) end end ``` Alternatively, the builder author can add the name of the custom builder to the `Rscons::DEFAULT_BUILDERS` array and then Rscons will automatically add the custom builder to every Environment. This method only works if the custom builder class is contained within the `Rscons::Builders` namespacing module. For example: ```ruby #SpecialBuilder.rb class Rscons::Builders::Special < Rscons::Builder end Rscons::DEFAULT_BUILDERS << :Special #Rsconscript load "SpecialBuilder.rb" default do env do |env| # A build target using the "Special" builder can be registered. env.Special("target", "source") end end ``` ####> Builder Name By default, the builder name is taken from the last component of the class name. For example, a class called `Rscons::Builders::Mine` would be usable in the Rsconscript with `env.Mine()`. A builder author can override the builder name by defining a class method within the builder class called `name`. For example, with the following builder definition: ```ruby class Rscons::Builders::MySpecialBuilder < Rscons::Builder def self.name "Special" end end ``` This builder would be registered in the Rsconscript with `env.Special()`. ####> Custom Builder Constructor It is optional for a custom builder to provide an `initialize` method. If an `initialize` method is provided, it must call `super` to invoke the base `Rscons::Builder` class's constructor. A single Hash parameter is passed to the builder constructor. This Hash contains many parameters describing how the build target was registered by the user. The base constructor will set several instance attributes within the builder: * `@target` will contain the path to the build target * `@sources` will contain the path(s) to the build source(s) * `@cache` will contain a reference to the `Rscons::Cache` object used for the build * `@env` will contain a reference to the Environment object that registered the build target using the builder * `@vars` will contain any user-specified construction variable values that should be used for the builder execution (overriding any Environment-wide construction variable values) ####> Custom Builder Operation In order for a builder to run, the builder class must implement a the `Builder#run()` method. Generally, the `run()` method will use the source file(s) to produce the target file. Here is an example of a trivial builder: ```ruby class Rscons::Builders::Custom < Rscons::Builder def run(options) File.open(@target, "w") do |fh| fh.write("Target file created.") end true end end ``` ##### Return Value If the builder has completed and failed, the `run` method should return `false`. In this case, generally the command executed or the builder itself would be expected to output something to `$stderr` indicating the reason for the build failure. If the builder has completed successfully, the `run` method should return `true`. If the builder is not yet complete and is waiting on other steps, the `run` method should return the return value from the `Builder#wait_for` method. See ${#Custom Builder Parallelization}. ##### Printing Build Status A builder should print a status line when it produces a build target. The `Builder#print_run_message` method can be used to print the builder status line. This method supports a limited markup syntax to identify and color code the build target and/or source(s). Here is our Custom builder example extended to print its status: ```ruby class Rscons::Builders::Custom < Rscons::Builder def run(options) print_run_message("Creating #{@target} from Custom builder", nil) File.open(@target, "w") do |fh| fh.write("Target file created.") end true end end ``` ##### Custom Builder Cache Usage - Only Rebuild When Necessary Whenever possible, a builder should keep track of information necessary to know whether the target file(s) need to be rebuilt. The `Rscons::Cache` object is the mechanism by which to keep track of this information. The Cache object provides two methods: `#up_to_date?` and `#register_build` which can be used to check if a built file is still up-to-date, and to register build information for a subsequent check. Here is a Custom builder which combines its source files similar to what the `cat` command would do: ```ruby class Rscons::Builders::Custom < Rscons::Builder def run(options) unless @cache.up_to_date?(@target, nil, @sources, @env) print_run_message("Combining #{Util.short_format_paths(@sources)} => #{@target}", nil) File.open(@target, "wb") do |fh| @sources.each do |source| fh.write(File.read(source, mode: "rb")) end end @cache.register_build(@target, nil, @sources, @env) end true end end ``` This builder would rebuild the target file and print its run message if the target file or any of the source file(s) were changed, but otherwise would be silent and not re-combine the source files. Note that generally the same arguments should be passed to `@cache.register_build` and `@cache.up_to_date?`. ##### Custom Builder Parallelization The Rscons scheduler can parallelize builders to take advantage of multiple processor cores. Taking advantage of this ability to parallelize requires the builder author to author the builder in a particular way. The `#run()` method of each builder is called from Rscons in the main program thread. However, the builder may execute a subcommand, spawn a thread, or register other builders to execute as a part of doing its job. In any of these cases, the builder's `run` method should make use of `Builder#wait_for` to "sleep" until one of the items being waited for has completed. ###### Using a Ruby Thread to Parallelize a Builder Here is an example of using a Ruby thread to parallelize a builder: ```ruby ${include build_tests/custom_builder/wait_for_thread.rb} ``` It is up to the author of the thread logic to only perform actions that are thread-safe. It is not safe to call other Rscons methods, for example, registering other builders or using the Cache, from a thread other than the one that calls the `#run()` method. ###### Executing a Subcommand from a Custom Builder It is a very common case that a builder will execute a subcommand which produces the build target. This is how most of the built-in Rscons builders execute. A low-level way to handle this is for the builder to construct an instance of the `Rscons::Command` class and then `wait_for` the Command object. However, this is a common enough case that Rscons provides a few convenience methods to handle this: * [`Rscons::Builder#register_command`](../yard/Rscons/Builder.html#register_command-instance_method) * [`Rscons::Builder#standard_command`](../yard/Rscons/Builder.html#standard_command-instance_method) * [`Rscons::Builder#finalize_command`](../yard/Rscons/Builder.html#finalize_command-instance_method) The `register_command` helper method can be used to create a Command object and wait for it to complete. The `standard_command` helper does the same thing as `register_command` but additionally checks the `@cache` for the target being up to date. The `finalize_command` helper can be used in conjunction with either of the previous helper methods. The built-in Rscons builders `Command` and `Disassemble` show examples of how to use the `standard_command` and `finalize_command` helper methods. Example (built-in Command builder): ```ruby ${include lib/rscons/builders/command.rb} ``` Example (built-in Disassemble builder): ```ruby ${include lib/rscons/builders/disassemble.rb} ``` ####> Simple custom builders added with add_builder The `add_builder` method of the `Environment` class optionally allows you to define and register a builder by providing a name and action block. This can be useful if the builder you are trying to define is easily expressed as a short ruby procedure. When `add_builder` is called in this manner a new builder will be registered with the environment with the given name. When this builder is used it will call the provided block in order to build the target. Example: ```ruby ${include build_tests/json_to_yaml/Rsconscript} ``` #> Appendix ## Default Construction Variables ```ruby ${include lib/rscons/default_construction_variables.rb} ``` ##> Example Build Scripts ### Example: Building a C Program ```ruby env do |env| env["CFLAGS"] << "-Wall" env.Program("program", glob("src/**/*.c")) end ``` ### Example: Building a D Program ```ruby env do |env| env["DFLAGS"] << "-Wall" env.Program("program", glob("src/**/*.d")) end ``` ### Example: Cloning an Environment ```ruby main_env = env do |env| env["CFLAGS"] = ["-fshort-enums", "-O3"] env["CPPDEFINES"] << "SOME_DEFINE" env["LIBS"] = ["SDL"] env.Program("program", glob("src/**/*.cc")) end test_env = main_env.clone do |env| env["CFLAGS"] -= ["-O3"] env["CFLAGS"] += ["-g", "-O0"] env["CPPDEFINES"] = "ENABLE_TESTS" env.Program("program-test", glob("src/**/*.cc")) end ``` ### Example: Custom Builder ```ruby class GenerateFoo < Builder def run(options) target, cache = options.values_at(:target, :cache) cache.mkdir_p(File.dirname(target)) File.open(target, "w") do |fh| fh.puts < ./configure && make You can make your Rscons-based project more familiar to users of autoconf-generated projects by creating a `configure` script and a `Makefile` for the user. Such users may be used to executing: ``` ./configure make ``` to build a project. To do this, create a `configure` script with contents similar to the following: ``` #!/bin/sh exec "$(dirname "$0")"/rscons configure "$@" ``` and make it executable with `chmod +x configure`. If you want your users to be able to build/clean a project with `make` but still make use of Rscons under the hood, you can create a `Makefile` with contents something like this: ``` .PHONY: all all: ./rscons build .PHONY: clean clean: ./rscons clean ``` ##> YARD API Documentation See [here](../yard/index.html) for Rscons YARD API Documentation. #> License Rscons is licensed under the terms of the MIT License: ``` ${include LICENSE.txt} ``` #> Contributing Rscons is developed on [github](https://github.com/holtrop/rscons). Issues may be submitted to [https://github.com/holtrop/rscons/issues](https://github.com/holtrop/rscons/issues). Pull requests may be submitted as well: 1. Fork it 2. Create your feature branch (`git checkout -b my-new-feature`) 3. Commit your changes (`git commit -am 'Add some feature'`) 4. Push to the branch (`git push origin my-new-feature`) 5. Create new Pull Request #> Change Log ${changelog}