| <!--#include file="header.html" --> |
| |
| <p><h1>Code style</h1></p> |
| |
| <p>The primary goal of toybox is _simple_ code. Keeping the code small is |
| second, with speed and lots of features coming in somewhere after that. |
| (For more on that, see the <a href=design.html>design</a> page.)</p> |
| |
| <p>A simple implementation usually takes up fewer lines of source code, |
| meaning more code can fit on the screen at once, meaning the programmer can |
| see more of it on the screen and thus keep more if in their head at once. |
| This helps code auditing and thus reduces bugs. That said, sometimes being |
| more explicit is preferable to being clever enough to outsmart yourself: |
| don't be so terse your code is unreadable.</p> |
| |
| <p>Toybox source is formatted to be read with 4-space tab stops. Each file |
| starts with a special comment telling vi to set the tab stop to 4. Note that |
| one of the bugs in Ubuntu 7.10 broke vi's ability to parse these comments; you |
| must either rebuild vim from source, or go ":ts=4" yourself each time you load |
| the file.</p> |
| |
| <p>Gotos are allowed for error handling, and for breaking out of |
| nested loops. In general, a goto should only jump forward (not back), and |
| should either jump to the end of an outer loop, or to error handling code |
| at the end of the function. Goto labels are never indented: they override the |
| block structure of the file. Putting them at the left edge makes them easy |
| to spot as overrides to the normal flow of control, which they are.</p> |
| |
| <p><h1>Building Toybox:</h1></p> |
| |
| <p>Toybox is configured using the Kconfig language pioneered by the Linux |
| kernel, and adopted by many other projects (uClibc, OpenEmbedded, etc). |
| This generates a ".config" file containing the selected options, which |
| controls which features to enable when building toybox.</p> |
| |
| <p>Each configuration option has a default value. The defaults indicate the |
| "maximum sane configuration", I.E. if the feature defaults to "n" then it |
| either isn't complete or is a special-purpose option (such as debugging |
| code) that isn't intended for general purpose use.</p> |
| |
| <p>The standard build invocation is:</p> |
| |
| <ul> |
| <li>make defconfig #(or menuconfig)</li> |
| <li>make</li> |
| <li>make install</li> |
| </ul> |
| |
| <p>Type "make help" to see all available build options.</p> |
| |
| <p>The file "configure" contains a number of environment variable definitions |
| which influence the build, such as specifying which compiler to use or where |
| to install the resulting binaries. This file is included by the build, but |
| accepts existing definitions of the environment variables, so it may be sourced |
| or modified by the developer before building and the definitions exported |
| to the environment will take precedence.</p> |
| |
| <p>(To clarify: "configure" describes the build and installation environment, |
| ".config" lists the features selected by defconfig/menuconfig.)</p> |
| |
| <p><h1>Infrastructure:</h1></p> |
| |
| <p>The toybox source code is in following directories:</p> |
| <ul> |
| <li>The <a href="#top">top level directory</a> contains the file main.c (were |
| execution starts), the header file toys.h (included by every command), and |
| other global infrastructure.</li> |
| <li>The <a href="#lib">lib directory</a> contains common functions shared by |
| multiple commands.</li> |
| <li>The <a href="#toys">toys directory</a> contains the C files implementating |
| each command.</li> |
| <li>The <a href="#scripts">scripts directory</a> contains the build and |
| test infrastructure.</li> |
| <li>The <a href="#kconfig">kconfig directory</a> contains the configuration |
| infrastructure implementing menuconfig (copied from the Linux kernel).</li> |
| <li>The <a href="#generated">generated directory</a> contains intermediate |
| files generated from other parts of the source code.</li> |
| </ul> |
| |
| <a name="adding" /> |
| <p><h1>Adding a new command</h1></p> |
| <p>To add a new command to toybox, add a C file implementing that command to |
| the toys directory. No other files need to be modified; the build extracts |
| all the information it needs (such as command line arguments) from specially |
| formatted comments and macros in the C file. (See the description of the |
| <a href="#generated">"generated" directory</a> for details.)</p> |
| |
| <p>An easy way to start a new command is copy the file "hello.c" to |
| the name of the new command, and modify this copy to implement the new command. |
| This file is an example command meant to be used as a "skeleton" for |
| new commands (more or less by turning every instance of "hello" into the |
| name of your command, updating the command line arguments, globals, and |
| help data, and then filling out its "main" function with code that does |
| something interesting). It provides examples of all the build infrastructure |
| (including optional elements like command line argument parsing and global |
| variables that a "hello world" program doesn't strictly need).</p> |
| |
| <p>Here's a checklist of steps to turn hello.c into another command:</p> |
| |
| <ul> |
| <li><p>First "cd toys" and "cp hello.c yourcommand.c". Note that the name |
| of this file is significant, it's the name of the new command you're adding |
| to toybox. Open your new file in your favorite editor.</p></li> |
| |
| <li><p>Change the one line comment at the top of the file (currently |
| "hello.c - A hello world program") to describe your new file.</p></li> |
| |
| <li><p>Change the copyright notice to your name, email, and the current |
| year.</p></li> |
| |
| <li><p>Give a URL to the relevant standards document, or say "Not in SUSv4" if |
| there is no relevant standard. (Currently both lines are there, delete |
| whichever is inappropriate.) The existing link goes to the directory of SUSv4 |
| command line utility standards on the Open Group's website, where there's often |
| a relevant commandname.html file. Feel free to link to other documentation or |
| standards as appropriate.</p></li> |
| |
| <li><p>Update the USE_YOURCOMMAND(NEWTOY(yourcommand,"blah",0)) line. |
| The NEWTOY macro fills out this command's <a href="#toy_list">toy_list</a> |
| structure. The arguments to the NEWTOY macro are:</p> |
| |
| <ol> |
| <li><p>the name used to run your command</p></li> |
| <li><p>the command line argument <a href="#lib_args">option parsing string</a> (NULL if none)</p></li> |
| <li><p>a bitfield of TOYFLAG values |
| (defined in toys.h) providing additional information such as where your |
| command should be installed on a running system, whether to blank umask |
| before running, whether or not the command must run as root (and thus should |
| retain root access if installed SUID), and so on.</p></li> |
| </ol> |
| </li> |
| |
| <li><p>Change the kconfig data (from "config YOURCOMMAND" to the end of the |
| comment block) to supply your command's configuration and help |
| information. The uppper case config symbols are used by menuconfig, and are |
| also what the CFG_ and USE_() macros are generated from (see [TODO]). The |
| help information here is used by menuconfig, and also by the "help" command to |
| describe your new command. (See [TODO] for details.) By convention, |
| unfinished commands default to "n" and finished commands default to "y", |
| so "make defconfig" selects all finished commands. (Note, "finished" means |
| "ready to be used", not that it'll never change again.)<p> |
| |
| <p>Each help block should start with a "usage: yourcommand" line explaining |
| any command line arguments added by this config option. The "help" command |
| outputs this text, and scripts/config2help.c in the build infrastructure |
| collates these usage lines for commands with multiple configuration |
| options when producing generated/help.h.</p> |
| </li> |
| |
| <li><p>Update the DEFINE_GLOBALS() macro to contain your command's global |
| variables, and also change the name "hello" in the #define TT line afterwards |
| to the name of your command. If your command has no global variables, delete |
| this macro (and the #define TT line afterwards). Note that if you specified |
| two-character command line arguments in NEWTOY(), the first few global |
| variables will be initialized by the automatic argument parsing logic, and |
| the type and order of these variables must correspond to the arguments |
| specified in NEWTOY(). See [TODO] for details.</p></li> |
| |
| <li><p>If you didn't delete the DEFINE_GLOBALS macro, change the "#define TT |
| this.hello" line to use your command name in place of the "hello". This is a |
| shortcut to access your global variables as if they were members of the global |
| struct "TT". (Access these members with a period ".", not a right arrow |
| "->".)</p></li> |
| |
| <li><p>Rename hello_main() to yourcommand_main(). This is the main() function |
| where execution of your command starts. See [TODO] to figure out what |
| happened to your command line arguments and how to access them.</p></li> |
| </ul> |
| |
| <p><a name="top" /><h2>Top level directory.</h2></p> |
| |
| <p>This directory contains global infrastructure.</p> |
| |
| <h3>toys.h</h3> |
| <p>Each command #includes "toys.h" as part of its standard prolog.</p> |
| |
| <p>This file sucks in most of the commonly used standard #includes, so |
| individual files can just #include "toys.h" and not have to worry about |
| stdargs.h and so on. Individual commands still need to #include |
| special-purpose headers that may not be present on all systems (and thus would |
| prevent toybox from building that command on such a system with that command |
| enabled). Examples include regex support, any "linux/" or "asm/" headers, mtab |
| support (mntent.h and sys/mount.h), and so on.</p> |
| |
| <p>The toys.h header also defines structures for most of the global variables |
| provided to each command by toybox_main(). These are described in |
| detail in the description for main.c, where they are initialized.</p> |
| |
| <p>The global variables are grouped into structures (and a union) for space |
| savings, to more easily track the amount of memory consumed by them, |
| so that they may be automatically cleared/initialized as needed, and so |
| that access to global variables is more easily distinguished from access to |
| local variables.</p> |
| |
| <h3>main.c</h3> |
| <p>Contains the main() function where execution starts, plus |
| common infrastructure to initialize global variables and select which command |
| to run. The "toybox" multiplexer command also lives here. (This is the |
| only command defined outside of the toys directory.)</p> |
| |
| <p>Execution starts in main() which trims any path off of the first command |
| name and calls toybox_main(), which calls toy_exec(), which calls toy_find() |
| and toy_init() before calling the appropriate command's function from |
| toy_list[] (via toys.which->toy_main()). |
| If the command is "toybox", execution recurses into toybox_main(), otherwise |
| the call goes to the appropriate commandname_main() from a C file in the toys |
| directory.</p> |
| |
| <p>The following global variables are defined in main.c:</p> |
| <ul> |
| <a name="toy_list" /> |
| <li><p><b>struct toy_list toy_list[]</b> - array describing all the |
| commands currently configured into toybox. The first entry (toy_list[0]) is |
| for the "toybox" multiplexer command, which runs all the other built-in commands |
| without symlinks by using its first argument as the name of the command to |
| run and the rest as that command's argument list (ala "./toybox echo hello"). |
| The remaining entries are the commands in alphabetical order (for efficient |
| binary search).</p> |
| |
| <p>This is a read-only array initialized at compile time by |
| defining macros and #including generated/newtoys.h.</p> |
| |
| <p>Members of struct toy_list (defined in "toys.h") include:</p> |
| <ul> |
| <li><p>char *<b>name</b> - the name of this command.</p></li> |
| <li><p>void (*<b>toy_main</b>)(void) - function pointer to run this |
| command.</p></li> |
| <li><p>char *<b>options</b> - command line option string (used by |
| get_optflags() in lib/args.c to intialize toys.optflags, toys.optargs, and |
| entries in the toy's DEFINE_GLOBALS struct). When this is NULL, no option |
| parsing is done before calling toy_main().</p></li> |
| <li><p>int <b>flags</b> - Behavior flags for this command. The following flags are currently understood:</p> |
| |
| <ul> |
| <li><b>TOYFLAG_USR</b> - Install this command under /usr</li> |
| <li><b>TOYFLAG_BIN</b> - Install this command under /bin</li> |
| <li><b>TOYFLAG_SBIN</b> - Install this command under /sbin</li> |
| <li><b>TOYFLAG_NOFORK</b> - This command can be used as a shell builtin.</li> |
| <li><b>TOYFLAG_UMASK</b> - Call umask(0) before running this command.</li> |
| <li><b>TOYFLAG_STAYROOT</b> - Don't drop permissions for this command if toybox is installed SUID root.</li> |
| <li><b>TOYFLAG_NEEDROOT</b> - This command cannot function unless run with root access.</li> |
| </ul> |
| <br> |
| |
| <p>These flags are combined with | (or). For example, to install a command |
| in /usr/bin, or together TOYFLAG_USR|TOYFLAG_BIN.</p> |
| </ul> |
| </li> |
| |
| <li><p><b>struct toy_context toys</b> - global structure containing information |
| common to all commands, initializd by toy_init() and defined in "toys.h". |
| Members of this structure include:</p> |
| <ul> |
| <li><p>struct toy_list *<b>which</b> - a pointer to this command's toy_list |
| structure. Mostly used to grab the name of the running command |
| (toys->which.name).</p> |
| </li> |
| <li><p>int <b>exitval</b> - Exit value of this command. Defaults to zero. The |
| error_exit() functions will return 1 if this is zero, otherwise they'll |
| return this value.</p></li> |
| <li><p>char **<b>argv</b> - "raw" command line options, I.E. the original |
| unmodified string array passed in to main(). Note that modifying this changes |
| "ps" output, and is not recommended. This array is null terminated; a NULL |
| entry indicates the end of the array.</p> |
| <p>Most commands don't use this field, instead the use optargs, optflags, |
| and the fields in the DEFINE_GLOBALS struct initialized by get_optflags().</p> |
| </li> |
| <li><p>unsigned <b>optflags</b> - Command line option flags, set by |
| <a href="#lib_args">get_optflags()</a>. Indicates which of the command line options listed in |
| toys->which.options occurred this time.</p> |
| |
| <p>The rightmost command line argument listed in toys->which.options sets bit |
| 1, the next one sets bit 2, and so on. This means the bits are set in the same |
| order the binary digits would be listed if typed out as a string. For example, |
| the option string "abcd" would parse the command line "-c" to set optflags to 2, |
| "-a" would set optflags to 8, and "-bd" would set optflags to 6 (4|2).</p> |
| |
| <p>Only letters are relevant to optflags. In the string "a*b:c#d", d=1, c=2, |
| b=4, a=8. The punctuation after a letter initializes global variables |
| (see [TODO] DECLARE_GLOBALS() for details).</p> |
| |
| <p>For more information on option parsing, see <a href="#lib_args">get_optflags()</a>.</p> |
| |
| </li> |
| <li><p>char **<b>optargs</b> - Null terminated array of arguments left over |
| after get_optflags() removed all the ones it understood. Note: optarg[0] is |
| the first argument, not the command name. Use toys.which->name for the command |
| name.</p></li> |
| <li><p>int <b>optc</b> - Optarg count, equivalent to argc but for |
| optargs[].<p></li> |
| <li><p>int <b>exithelp</b> - Whether error_exit() should print a usage message |
| via help_main() before exiting. (True during option parsing, defaults to |
| false afterwards.)</p></li> |
| </ul> |
| |
| <li><p><b>union toy_union this</b> - Union of structures containing each |
| command's global variables.</p> |
| |
| <p>Global variables are useful: they reduce the overhead of passing extra |
| command line arguments between functions, they conveniently start prezeroed to |
| save initialization costs, and the command line argument parsing infrastructure |
| can also initialize global variables with its results.</p> |
| |
| <p>But since each toybox process can only run one command at a time, allocating |
| space for global variables belonging to other commands you aren't currently |
| running would be wasteful.</p> |
| |
| <p>Toybox handles this by encapsulating each command's global variables in |
| a structure, and declaring a union of those structures with a single global |
| instance (called "this"). The DEFINE_GLOBALS() macro contains the global |
| variables that should go in the current command's global structure. Each |
| variable can then be accessed as "this.commandname.varname". |
| Generally, the macro TT is #defined to this.commandname so the variable |
| can then be accessed as "TT.variable". See toys/hello.c for an example.</p> |
| |
| <p>A command that needs global variables should declare a structure to |
| contain them all, and add that structure to this union. A command should never |
| declare global variables outside of this, because such global variables would |
| allocate memory when running other commands that don't use those global |
| variables.</p> |
| |
| <p>The first few fields of this structure can be intialized by <a href="#lib_args">get_optargs()</a>, |
| as specified by the options field off this command's toy_list entry. See |
| the get_optargs() description in lib/args.c for details.</p> |
| </li> |
| |
| <li><b>char toybuf[4096]</b> - a common scratch space buffer so |
| commands don't need to allocate their own. Any command is free to use this, |
| and it should never be directly referenced by functions in lib/ (although |
| commands are free to pass toybuf in to a library function as an argument).</li> |
| </ul> |
| |
| <p>The following functions are defined in main.c:</p> |
| <ul> |
| <li><p>struct toy_list *<b>toy_find</b>(char *name) - Return the toy_list |
| structure for this command name, or NULL if not found.</p></li> |
| <li><p>void <b>toy_init</b>(struct toy_list *which, char *argv[]) - fill out |
| the global toys structure, calling get_optargs() if necessary.</p></li> |
| <li><p>void <b>toy_exec</b>(char *argv[]) - Run a built-in command with |
| arguments.</p> |
| <p>Calls toy_find() on argv[0] (which must be just a command name |
| without path). Returns if it can't find this command, otherwise calls |
| toy_init(), toys->which.toy_main(), and exit() instead of returning.</p> |
| |
| <p>Use the library function xexec() to fall back to external executables |
| in $PATH if toy_exec() can't find a built-in command. Note that toy_exec() |
| does not strip paths before searching for a command, so "./command" will |
| never match an internal command.</li> |
| |
| <li><p>void <b>toybox_main</b>(void) - the main function for the multiplexer |
| command (I.E. "toybox"). Given a command name as its first argument, calls |
| toy_exec() on its arguments. With no arguments, it lists available commands. |
| If the first argument starts with "-" it lists each command with its default |
| install path prepended.</p></li> |
| |
| </ul> |
| |
| <h3>Config.in</h3> |
| |
| <p>Top level configuration file in a stylized variant of |
| <a href=http://kernel.org/doc/Documentation/kbuild/kconfig-language.txt>kconfig</a> format. Includes generated/Config.in.</p> |
| |
| <p>These files are directly used by "make menuconfig" to select which commands |
| to build into toybox (thus generating a .config file), and by |
| scripts/config2help.py to create generated/help.h.</p> |
| |
| <h3>Temporary files:</h3> |
| |
| <p>There is one temporary file in the top level source directory:</p> |
| <ul> |
| <li><p><b>.config</b> - Configuration file generated by kconfig, indicating |
| which commands (and options to commands) are currently enabled. Used |
| to make generated/config.h and determine which toys/*.c files to build.</p> |
| |
| <p>You can create a human readable "miniconfig" version of this file using |
| <a href=http://landley.net/aboriginal/new_platform.html#miniconfig>these |
| instructions</a>.</p> |
| </li> |
| </ul> |
| |
| <a name="generated" /> |
| <p>The "generated/" directory contains files generated from other source code |
| in toybox. All of these files can be recreated by the build system, although |
| some (such as generated/help.h) are shipped in release versions to reduce |
| environmental dependencies (I.E. so you don't need python on your build |
| system).</p> |
| |
| <ul> |
| <li><p><b>generated/config.h</b> - list of CFG_SYMBOL and USE_SYMBOL() macros, |
| generated from .config by a sed invocation in the top level Makefile.</p> |
| |
| <p>CFG_SYMBOL is a comple time constant set to 1 for enabled symbols and 0 for |
| disabled symbols. This allows the use of normal if() statements to remove |
| code at compile time via the optimizer's dead code elimination (which removes |
| from the binary any code that cannot be reached). This saves space without |
| cluttering the code with #ifdefs or leading to configuration dependent build |
| breaks. (See the 1992 Usenix paper |
| <a href=http://doc.cat-v.org/henry_spencer/ifdef_considered_harmful.pdf>#ifdef |
| Considered Harmful</a> for more information.)</p> |
| |
| <p>USE_SYMBOL(code) evaluates to the code in parentheses when the symbol |
| is enabled, and nothing when the symbol is disabled. This can be used |
| for things like varargs or variable declarations which can't always be |
| eliminated by a simple test on CFG_SYMBOL. Note that |
| (unlike CFG_SYMBOL) this is really just a variant of #ifdef, and can |
| still result in configuration dependent build breaks. Use with caution.</p> |
| </li> |
| </ul> |
| |
| <p><h2>Directory toys/</h2></p> |
| |
| <h3>toys/Config.in</h3> |
| |
| <p>Included from the top level Config.in, contains one or more |
| configuration entries for each command.</p> |
| |
| <p>Each command has a configuration entry matching the command name (although |
| configuration symbols are uppercase and command names are lower case). |
| Options to commands start with the command name followed by an underscore and |
| the option name. Global options are attached to the "toybox" command, |
| and thus use the prefix "TOYBOX_". This organization is used by |
| scripts/cfg2files to select which toys/*.c files to compile for a given |
| .config.</p> |
| |
| <p>A command with multiple names (or multiple similar commands implemented in |
| the same .c file) should have config symbols prefixed with the name of their |
| C file. I.E. config symbol prefixes are NEWTOY() names. If OLDTOY() names |
| have config symbols they're options (symbols with an underscore and suffix) |
| to the NEWTOY() name. (See toys/toylist.h)</p> |
| |
| <h3>toys/toylist.h</h3> |
| <p>The first half of this file prototypes all the structures to hold |
| global variables for each command, and puts them in toy_union. These |
| prototypes are only included if the macro NEWTOY isn't defined (in which |
| case NEWTOY is defined to a default value that produces function |
| prototypes).</p> |
| |
| <p>The second half of this file lists all the commands in alphabetical |
| order, along with their command line arguments and install location. |
| Each command has an appropriate configuration guard so only the commands that |
| are enabled wind up in the list.</p> |
| |
| <p>The first time this header is #included, it defines structures and |
| produces function prototypes for the commands in the toys directory.</p> |
| |
| |
| <p>The first time it's included, it defines structures and produces function |
| prototypes. |
| This |
| is used to initialize toy_list in main.c, and later in that file to initialize |
| NEED_OPTIONS (to figure out whether the command like parsing logic is needed), |
| and to put the help entries in the right order in toys/help.c.</p> |
| |
| <h3>toys/help.h</h3> |
| |
| <p>#defines two help text strings for each command: a single line |
| command_help and an additinal command_help_long. This is used by help_main() |
| in toys/help.c to display help for commands.</p> |
| |
| <p>Although this file is generated from Config.in help entries by |
| scripts/config2help.py, it's shipped in release tarballs so you don't need |
| python on the build system. (If you check code out of source control, or |
| modify Config.in, then you'll need python installed to rebuild it.)</p> |
| |
| <p>This file contains help for all commands, regardless of current |
| configuration, but only the currently enabled ones are entered into help_data[] |
| in toys/help.c.</p> |
| |
| <a name="lib"> |
| <h2>Directory lib/</h2> |
| |
| <p>TODO: document lots more here.</p> |
| |
| <p>lib: llist, getmountlist(), error_msg/error_exit, xmalloc(), |
| strlcpy(), xexec(), xopen()/xread(), xgetcwd(), xabspath(), find_in_path(), |
| itoa().</p> |
| |
| <h3>lib/portability.h</h3> |
| |
| <p>This file is automatically included from the top of toys.h, and smooths |
| over differences between platforms (hardware targets, compilers, C libraries, |
| operating systems, etc).</p> |
| |
| <p>This file provides SWAP macros (SWAP_BE16(x) and SWAP_LE32(x) and so on).</p> |
| |
| <p>A macro like SWAP_LE32(x) means "The value in x is stored as a little |
| endian 32 bit value, so perform the translation to/from whatever the native |
| 32-bit format is". You do the swap once on the way in, and once on the way |
| out. If your target is already little endian, the macro is a NOP.</p> |
| |
| <p>The SWAP macros come in BE and LE each with 16, 32, and 64 bit versions. |
| In each case, the name of the macro refers to the _external_ representation, |
| and converts to/from whatever your native representation happens to be (which |
| can vary depending on what you're currently compiling for).</p> |
| |
| <a name="lib_args"><h3>lib/args.c</h3> |
| |
| <p>Toybox's main.c automatically parses command line options before calling the |
| command's main function. Option parsing starts in get_optflags(), which stores |
| results in the global structures "toys" (optflags and optargs) and "this".</p> |
| |
| <p>The option parsing infrastructure stores a bitfield in toys.optflags to |
| indicate which options the current command line contained. Arguments |
| attached to those options are saved into the command's global structure |
| ("this"). Any remaining command line arguments are collected together into |
| the null-terminated array toys.optargs, with the length in toys.optc. (Note |
| that toys.optargs does not contain the current command name at position zero, |
| use "toys.which->name" for that.) The raw command line arguments get_optflags() |
| parsed are retained unmodified in toys.argv[].</p> |
| |
| <p>Toybox's option parsing logic is controlled by an "optflags" string, using |
| a format reminiscent of getopt's optargs but has several important differences. |
| Toybox does not use the getopt() |
| function out of the C library, get_optflags() is an independent implementation |
| which doesn't permute the original arguments (and thus doesn't change how the |
| command is displayed in ps and top), and has many features not present in |
| libc optargs() (such as the ability to describe long options in the same string |
| as normal options).</p> |
| |
| <p>Each command's NEWTOY() macro has an optflags string as its middle argument, |
| which sets toy_list.options for that command to tell get_optflags() what |
| command line arguments to look for, and what to do with them. |
| If a command has no option |
| definition string (I.E. the argument is NULL), option parsing is skipped |
| for that command, which must look at the raw data in toys.argv to parse its |
| own arguments. (If no currently enabled command uses option parsing, |
| get_optflags() is optimized out of the resulting binary by the compiler's |
| --gc-sections option.)</p> |
| |
| <p>You don't have to free the option strings, which point into the environment |
| space (I.E. the string data is not copied). A TOYFLAG_NOFORK command |
| that uses the linked list type "*" should free the list objects but not |
| the data they point to, via "llist_free(TT.mylist, NULL);". (If it's not |
| NOFORK, exit() will free all the malloced data anyway unless you want |
| to implement a CONFIG_TOYBOX_FREE cleanup for it.)</p> |
| |
| <h4>Optflags format string</h4> |
| |
| <p>Note: the optflags option description string format is much more |
| concisely described by a large comment at the top of lib/args.c.</p> |
| |
| <p>The general theory is that letters set optflags, and punctuation describes |
| other actions the option parsing logic should take.</p> |
| |
| <p>For example, suppose the command line <b>command -b fruit -d walrus -a 42</b> |
| is parsed using the optflags string "<b>a#b:c:d</b>". (I.E. |
| toys.which->options="a#b:c:d" and argv = ["command", "-b", "fruit", "-d", |
| "walrus", "-a", "42"]). When get_optflags() returns, the following data is |
| available to command_main(): |
| |
| <ul> |
| <li><p>In <b>struct toys</b>: |
| <ul> |
| <li>toys.optflags = 13; // -a = 8 | -b = 4 | -d = 1</li> |
| <li>toys.optargs[0] = "walrus"; // leftover argument</li> |
| <li>toys.optargs[1] = NULL; // end of list</li> |
| <li>toys.optc=1; // there was 1 leftover argument</li> |
| <li>toys.argv[] = {"-b", "fruit", "-d", "walrus", "-a", "42"}; // The original command line arguments |
| </ul> |
| <p></li> |
| |
| <li><p>In <b>union this</b> (treated as <b>long this[]</b>): |
| <ul> |
| <li>this[0] = NULL; // -c didn't get an argument this time, so get_optflags() didn't change it and toys_init() zeroed "this" during setup.)</li> |
| <li>this[1] = (long)"fruit"; // argument to -b</li> |
| <li>this[2] = 42; // argument to -a</li> |
| </ul> |
| </p></li> |
| </ul> |
| |
| <p>If the command's globals are:</p> |
| |
| <blockquote><pre> |
| DECLARE_GLOBALS( |
| char *c; |
| char *b; |
| long a; |
| ) |
| #define TT this.command |
| </pre></blockquote> |
| <p>That would mean TT.c == NULL, TT.b == "fruit", and TT.a == 42. (Remember, |
| each entry that receives an argument must be a long or pointer, to line up |
| with the array position. Right to left in the optflags string corresponds to |
| top to bottom in DECLARE_GLOBALS().</p> |
| |
| <p><b>long toys.optflags</b></p> |
| |
| <p>Each option in the optflags string corresponds to a bit position in |
| toys.optflags, with the same value as a corresponding binary digit. The |
| rightmost argument is (1<<0), the next to last is (1<<1) and so on. If |
| the option isn't encountered while parsing argv[], its bit remains 0.</p> |
| |
| <p>For example, |
| the optflags string "abcd" would parse the command line argument "-c" to set |
| optflags to 2, "-a" would set optflags to 8, "-bd" would set optflags to |
| 6 (I.E. 4|2), and "-a -c" would set optflags to 10 (2|8).</p> |
| |
| <p>Only letters are relevant to optflags, punctuation is skipped: in the |
| string "a*b:c#d", d=1, c=2, b=4, a=8. The punctuation after a letter |
| usually indicate that the option takes an argument.</p> |
| |
| <p>Since toys.optflags is an unsigned int, it only stores 32 bits. (Which is |
| the amount a long would have on 32-bit platforms anyway; 64 bit code on |
| 32 bit platforms is too expensive to require in common code used by almost |
| all commands.) Bit positions beyond the 1<<31 aren't recorded, but |
| parsing higher options can still set global variables.</p> |
| |
| <p><b>Automatically setting global variables from arguments (union this)</b></p> |
| |
| <p>The following punctuation characters may be appended to an optflags |
| argument letter, indicating the option takes an additional argument:</p> |
| |
| <ul> |
| <li><b>:</b> - plus a string argument, keep most recent if more than one.</li> |
| <li><b>*</b> - plus a string argument, appended to a linked list.</li> |
| <li><b>@</b> - plus an occurrence counter (stored in a long)</li> |
| <li><b>#</b> - plus a signed long argument. |
| <li><b>.</b> - plus a floating point argument (if CFG_TOYBOX_FLOAT).</li> |
| <ul>The following can be appended to a float or double: |
| <li><b><123</b> - error if argument is less than this</li> |
| <li><b>>123</b> - error if argument is greater than this</li> |
| <li><b>=123</b> - default value if argument not supplied</li> |
| </ul> |
| <ul><li>Option parsing only understands <>= after . when CFG_TOYBOX_FLOAT |
| is enabled. (Otherwise the code to determine where floating point constants |
| end drops out. When disabled, it can reserve a global data slot for the |
| argument so offsets won't change, but will never fill it out.). You can handle |
| this by using the USE_BLAH() macros with C string concatenation, ala: |
| "abc." USE_TOYBOX_FLOAT("<1.23>4.56=7.89") "def"</li></ul> |
| </ul> |
| |
| <p>Arguments may occur with or without a space (I.E. "-a 42" or "-a42"). |
| The command line argument "-abc" may be interepreted many different ways: |
| the optflags string "cba" sets toys.optflags = 7, "c:ba" sets toys.optflags=4 |
| and saves "ba" as the argument to -c, and "cb:a" sets optflags to 6 and saves |
| "c" as the argument to -b.</p> |
| |
| <p>Options which have an argument fill in the corresponding slot in the global |
| union "this" (see generated/globals.h), treating it as an array of longs |
| with the rightmost saved in this[0]. Again using "a*b:c#d", "-c 42" would set |
| this[0]=42; and "-b 42" would set this[1]="42"; each slot is left NULL if |
| the corresponding argument is not encountered.</p> |
| |
| <p>This behavior is useful because the LP64 standard ensures long and pointer |
| are the same size. C99 guarantees structure members will occur in memory |
| in the same order they're declared, and that padding won't be inserted between |
| consecutive variables of register size. Thus the first few entries can |
| be longs or pointers corresponding to the saved arguments.</p> |
| |
| <p><b>char *toys.optargs[]</b></p> |
| |
| <p>Command line arguments in argv[] which are not consumed by option parsing |
| (I.E. not recognized either as -flags or arguments to -flags) will be copied |
| to toys.optargs[], with the length of that array in toys.optc. |
| (When toys.optc is 0, no unrecognized command line arguments remain.) |
| The order of entries is preserved, and as with argv[] this new array is also |
| terminated by a NULL entry.</p> |
| |
| <p>Option parsing can require a minimum or maximum number of optargs left |
| over, by adding "<1" (read "at least one") or ">9" ("at most nine") to the |
| start of the optflags string.</p> |
| |
| <p>The special argument "--" terminates option parsing, storing all remaining |
| arguments in optargs. The "--" itself is consumed.</p> |
| |
| <p><b>Other optflags control characters</b></p> |
| |
| <p>The following characters may occur at the start of each command's |
| optflags string, before any options that would set a bit in toys.optflags:</p> |
| |
| <ul> |
| <li><b>^</b> - stop at first nonoption argument (for nice, xargs...)</li> |
| <li><b>?</b> - allow unknown arguments (pass non-option arguments starting |
| with - through to optargs instead of erroring out).</li> |
| <li><b>&</b> - the first argument has imaginary dash (ala tar/ps. If given twice, all arguments have imaginary dash.)</li> |
| <li><b><</b> - must be followed by a decimal digit indicating at least this many leftover arguments are needed in optargs (default 0)</li> |
| <li><b>></b> - must be followed by a decimal digit indicating at most this many leftover arguments allowed (default MAX_INT)</li> |
| </ul> |
| |
| <p>The following characters may be appended to an option character, but do |
| not by themselves indicate an extra argument should be saved in this[]. |
| (Technically any character not recognized as a control character sets an |
| optflag, but letters are never control characters.)</p> |
| |
| <ul> |
| <li><b>^</b> - stop parsing options after encountering this option, everything else goes into optargs.</li> |
| <li><b>|</b> - this option is required. If more than one marked, only one is required.</li> |
| <li><b>+X</b> enabling this option also enables option X (switch bit on).</li> |
| <li><b>~X</b> enabling this option disables option X (switch bit off).</li> |
| <li><b>!X</b> this option cannot be used in combination with X (die with error).</li> |
| <li><b>[yz]</b> this option requires at least one of y or z to also be enabled.</li> |
| </ul> |
| |
| <p>The following may be appended to a float or double:</p> |
| |
| <ul> |
| <li><b><123</b> - error if argument is less than this</li> |
| <li><b>>123</b> - error if argument is greater than this</li> |
| <li><b>=123</b> - default value if argument not supplied</li> |
| </ul> |
| |
| <p>Option parsing only understands <>= after . when CFG_TOYBOX_FLOAT |
| is enabled. (Otherwise the code to determine where floating point constants |
| end drops out. When disabled, it can reserve a global data slot for the |
| argument so offsets won't change, but will never fill it out.). You can handle |
| this by using the USE_BLAH() macros with C string concatenation, ala:</p> |
| |
| <blockquote>"abc." USE_TOYBOX_FLOAT("<1.23>4.56=7.89") "def"</blockquote> |
| |
| <p><b>--longopts</b></p> |
| |
| <p>The optflags string can contain long options, which are enclosed in |
| parentheses. They may be appended to an existing option character, in |
| which case the --longopt is a synonym for that option, ala "a:(--fred)" |
| which understands "-a blah" or "--fred blah" as synonyms.</p> |
| |
| <p>Longopts may also appear before any other options in the optflags string, |
| in which case they have no corresponding short argument, but instead set |
| their own bit based on position. So for "(walrus)#(blah)xy:z" "command |
| --walrus 42" would set toys.optflags = 16 (-z = 1, -y = 2, -x = 4, --blah = 8) |
| and would assign this[1] = 42;</p> |
| |
| <p>A short option may have multiple longopt synonyms, "a(one)(two)", but |
| each "bare longopt" (ala "(one)(two)abc" before any option characters) |
| always sets its own bit (although you can group them with +X).</p> |
| |
| <h2>Directory scripts/</h2> |
| |
| <h3>scripts/cfg2files.sh</h3> |
| |
| <p>Run .config through this filter to get a list of enabled commands, which |
| is turned into a list of files in toys via a sed invocation in the top level |
| Makefile. |
| </p> |
| |
| <h2>Directory kconfig/</h2> |
| |
| <p>Menuconfig infrastructure copied from the Linux kernel. See the |
| Linux kernel's Documentation/kbuild/kconfig-language.txt</p> |
| |
| <a name="generated"> |
| <h2>Directory generated/</h2> |
| |
| <p>All the files in this directory except the README are generated by the |
| build. (See scripts/make.sh)</p> |
| |
| <ul> |
| <li><p><b>config.h</b> - CFG_COMMAND and USE_COMMAND() macros set by menuconfig via .config.</p></li> |
| |
| <li><p><b>Config.in</b> - Kconfig entries for each command. Included by top level Config.in. The help text in here is used to generated help.h</p></li> |
| |
| <li><p><b>help.h</b> - Help text strings for use by "help" command. Building |
| this file requires python on the host system, so the prebuilt file is shipped |
| in the build tarball to avoid requiring python to build toybox.</p></li> |
| |
| <li><p><b>newtoys.h</b> - List of NEWTOY() or OLDTOY() macros for all available |
| commands. Associates command_main() functions with command names, provides |
| option string for command line parsing (<a href="#lib_args">see lib/args.c</a>), |
| specifies where to install each command and whether toysh should fork before |
| calling it.</p></li> |
| </ul> |
| |
| <p>Everything in this directory is a derivative file produced from something |
| else. The entire directory is deleted by "make distclean".</p> |
| <!--#include file="footer.html" --> |