blob: 45822b1f752bd43ba0b78850afe515058a7b5d56 [file] [log] [blame]
Rob Landley4e68de12007-12-13 07:00:27 -06001<!--#include file="header.html" -->
2
Rob Landley5a0660f2007-12-27 21:36:44 -06003<p><h1>Code style</h1></p>
4
5<p>Toybox source is formatted to be read with 4-space tab stops. Each file
6starts with a special comment telling vi to set the tab stop to 4. Note that
7one of the bugs in Ubuntu 7.10 broke vi's ability to parse these comments; you
8must either rebuild vim from source, or go ":ts=4" yourself each time you load
9the file.</p>
10
11<p>Gotos are allowed for error handling, and for breaking out of
12nested loops. In general, a goto should only jump forward (not back), and
13should either jump to the end of an outer loop, or to error handling code
14at the end of the function. Goto labels are never indented: they override the
15block structure of the file. Putting them at the left edge makes them easy
16to spot as overrides to the normal flow of control, which they are.</p>
17
Rob Landley6882ee82008-02-12 18:41:34 -060018<p>The primary goal of toybox is _simple_ code. Small is second,
19speed and lots of features come in somewhere after that. Note that
20environmental dependencies are a type of complexity, so needing other packages
Rob Landley66a69d92012-01-16 01:44:17 -060021to build or run is a big downside. For example, don't use curses when you can
Rob Landley6882ee82008-02-12 18:41:34 -060022output ansi escape sequences instead.</p>
23
Rob Landley4e68de12007-12-13 07:00:27 -060024<p><h1>Infrastructure:</h1></p>
25
Rob Landley7c04f012008-01-20 19:00:16 -060026<p>The toybox source code is in following directories:</p>
27<ul>
28<li>The <a href="#top">top level directory</a> contains the file main.c (were
29execution starts), the header file toys.h (included by every command), and
30other global infrastructure.</li>
31<li>The <a href="#lib">lib directory</a> contains common functions shared by
32multiple commands.</li>
33<li>The <a href="#toys">toys directory</a> contains the C files implementating
34each command.</li>
35<li>The <a href="#scripts">scripts directory</a> contains the build and
36test infrastructure.</li>
37<li>The <a href="#kconfig">kconfig directory</a> contains the configuration
38infrastructure implementing menuconfig (copied from the Linux kernel).</li>
39<li>The <a href="#generated">generated directory</a> contains intermediate
40files generated from other parts of the source code.</li>
41</ul>
Rob Landley4e68de12007-12-13 07:00:27 -060042
Rob Landleybbe500e2012-02-26 21:53:15 -060043<a name="adding" />
Rob Landley7c04f012008-01-20 19:00:16 -060044<p><h1>Adding a new command</h1></p>
45<p>To add a new command to toybox, add a C file implementing that command to
46the toys directory. No other files need to be modified; the build extracts
Rob Landley6882ee82008-02-12 18:41:34 -060047all the information it needs (such as command line arguments) from specially
Rob Landley7c04f012008-01-20 19:00:16 -060048formatted comments and macros in the C file. (See the description of the
49<a href="#generated">generated directory</a> for details.)</p>
Rob Landley4e68de12007-12-13 07:00:27 -060050
Rob Landley7c04f012008-01-20 19:00:16 -060051<p>An easy way to start a new command is copy the file "hello.c" to
52the name of the new command, and modify this copy to implement the new command.
Rob Landley6882ee82008-02-12 18:41:34 -060053This file is an example command meant to be used as a "skeleton" for
Rob Landley7c04f012008-01-20 19:00:16 -060054new commands (more or less by turning every instance of "hello" into the
55name of your command, updating the command line arguments, globals, and
56help data, and then filling out its "main" function with code that does
Rob Landley6882ee82008-02-12 18:41:34 -060057something interesting). It provides examples of all the build infrastructure
58(including optional elements like command line argument parsing and global
59variables that a "hello world" program doesn't strictly need).</p>
Rob Landley7c04f012008-01-20 19:00:16 -060060
61<p>Here's a checklist of steps to turn hello.c into another command:</p>
62
63<ul>
64<li><p>First "cd toys" and "cp hello.c yourcommand.c". Note that the name
65of this file is significant, it's the name of the new command you're adding
66to toybox. Open your new file in your favorite editor.</p></li>
67
68<li><p>Change the one line comment at the top of the file (currently
69"hello.c - A hello world program") to describe your new file.</p></li>
70
71<li><p>Change the copyright notice to your name, email, and the current
72year.</p></li>
73
Rob Landley66a69d92012-01-16 01:44:17 -060074<li><p>Give a URL to the relevant standards document, or say "Not in SUSv4" if
Rob Landley7c04f012008-01-20 19:00:16 -060075there is no relevant standard. (Currently both lines are there, delete
Rob Landley66a69d92012-01-16 01:44:17 -060076whichever is inappropriate.) The existing link goes to the directory of SUSv4
Rob Landley7c04f012008-01-20 19:00:16 -060077command line utility standards on the Open Group's website, where there's often
78a relevant commandname.html file. Feel free to link to other documentation or
79standards as appropriate.</p></li>
80
Rob Landley66a69d92012-01-16 01:44:17 -060081<li><p>Update the USE_YOURCOMMAND(NEWTOY(yourcommand,"blah",0)) line.
82The NEWTOY macro fills out this command's <a href="#toy_list">toy_list</a>
83structure. The arguments to the NEWTOY macro are:</p>
84
85<ol>
86<li><p>the name used to run your command</p></li>
87<li><p>the command line argument <a href="#lib_args">option parsing string</a> (NULL if none)</p></li>
88<li><p>a bitfield of TOYFLAG values
89(defined in toys.h) providing additional information such as where your
90command should be installed on a running system, whether to blank umask
91before running, whether or not the command must run as root (and thus should
92retain root access if installed SUID), and so on.</p></li>
93</ol>
94</li>
Rob Landley7c04f012008-01-20 19:00:16 -060095
96<li><p>Change the kconfig data (from "config YOURCOMMAND" to the end of the
97comment block) to supply your command's configuration and help
98information. The uppper case config symbols are used by menuconfig, and are
99also what the CFG_ and USE_() macros are generated from (see [TODO]). The
100help information here is used by menuconfig, and also by the "help" command to
101describe your new command. (See [TODO] for details.) By convention,
Rob Landley66a69d92012-01-16 01:44:17 -0600102unfinished commands default to "n" and finished commands default to "y",
103so "make defconfig" selects all finished commands. (Note, "finished" means
104"ready to be used", not that it'll never change again.)<p>
105
106<p>Each help block should start with a "usage: yourcommand" line explaining
107any command line arguments added by this config option. The "help" command
108outputs this text, and scripts/config2help.c in the build infrastructure
109collates these usage lines for commands with multiple configuration
110options when producing generated/help.h.</p>
111</li>
Rob Landley7c04f012008-01-20 19:00:16 -0600112
113<li><p>Update the DEFINE_GLOBALS() macro to contain your command's global
114variables, and also change the name "hello" in the #define TT line afterwards
115to the name of your command. If your command has no global variables, delete
116this macro (and the #define TT line afterwards). Note that if you specified
117two-character command line arguments in NEWTOY(), the first few global
118variables will be initialized by the automatic argument parsing logic, and
119the type and order of these variables must correspond to the arguments
120specified in NEWTOY(). See [TODO] for details.</p></li>
121
Rob Landley6882ee82008-02-12 18:41:34 -0600122<li><p>If you didn't delete the DEFINE_GLOBALS macro, change the "#define TT
123this.hello" line to use your command name in place of the "hello". This is a
124shortcut to access your global variables as if they were members of the global
125struct "TT". (Access these members with a period ".", not a right arrow
126"->".)</p></li>
Rob Landley7c04f012008-01-20 19:00:16 -0600127
128<li><p>Rename hello_main() to yourcommand_main(). This is the main() function
Rob Landley6882ee82008-02-12 18:41:34 -0600129where execution of your command starts. See [TODO] to figure out what
Rob Landley7c04f012008-01-20 19:00:16 -0600130happened to your command line arguments and how to access them.</p></li>
131</ul>
132
133<p><a name="top" /><h2>Top level directory.</h2></p>
134
Rob Landley66a69d92012-01-16 01:44:17 -0600135<p>This directory contains global infrastructure.</p>
136
137<h3>toys.h</h3>
138<p>Each command #includes "toys.h" as part of its standard prolog.</p>
139
140<p>This file sucks in most of the commonly used standard #includes, so
141individual files can just #include "toys.h" and not have to worry about
142stdargs.h and so on. Individual commands still need to #include
143special-purpose headers that may not be present on all systems (and thus would
144prevent toybox from building that command on such a system with that command
145enabled). Examples include regex support, any "linux/" or "asm/" headers, mtab
146support (mntent.h and sys/mount.h), and so on.</p>
147
148<p>The toys.h header also defines structures for most of the global variables
149provided to each command by toybox_main(). These are described in
150detail in the description for main.c, where they are initialized.</p>
151
152<p>The global variables are grouped into structures (and a union) for space
153savings, to more easily track the amount of memory consumed by them,
154so that they may be automatically cleared/initialized as needed, and so
155that access to global variables is more easily distinguished from access to
156local variables.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600157
158<h3>main.c</h3>
159<p>Contains the main() function where execution starts, plus
160common infrastructure to initialize global variables and select which command
Rob Landley6882ee82008-02-12 18:41:34 -0600161to run. The "toybox" multiplexer command also lives here. (This is the
Rob Landley7c04f012008-01-20 19:00:16 -0600162only command defined outside of the toys directory.)</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600163
Rob Landley6882ee82008-02-12 18:41:34 -0600164<p>Execution starts in main() which trims any path off of the first command
165name and calls toybox_main(), which calls toy_exec(), which calls toy_find()
Rob Landley66a69d92012-01-16 01:44:17 -0600166and toy_init() before calling the appropriate command's function from
167toy_list[] (via toys.which->toy_main()).
Rob Landley6882ee82008-02-12 18:41:34 -0600168If the command is "toybox", execution recurses into toybox_main(), otherwise
169the call goes to the appropriate commandname_main() from a C file in the toys
170directory.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600171
Rob Landley6882ee82008-02-12 18:41:34 -0600172<p>The following global variables are defined in main.c:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600173<ul>
Rob Landley66a69d92012-01-16 01:44:17 -0600174<a name="toy_list" />
175<li><p><b>struct toy_list toy_list[]</b> - array describing all the
Rob Landley4e68de12007-12-13 07:00:27 -0600176commands currently configured into toybox. The first entry (toy_list[0]) is
177for the "toybox" multiplexer command, which runs all the other built-in commands
178without symlinks by using its first argument as the name of the command to
179run and the rest as that command's argument list (ala "./toybox echo hello").
180The remaining entries are the commands in alphabetical order (for efficient
181binary search).</p>
182
183<p>This is a read-only array initialized at compile time by
Rob Landley6882ee82008-02-12 18:41:34 -0600184defining macros and #including generated/newtoys.h.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600185
Rob Landley66a69d92012-01-16 01:44:17 -0600186<p>Members of struct toy_list (defined in "toys.h") include:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600187<ul>
188<li><p>char *<b>name</b> - the name of this command.</p></li>
189<li><p>void (*<b>toy_main</b>)(void) - function pointer to run this
190command.</p></li>
191<li><p>char *<b>options</b> - command line option string (used by
192get_optflags() in lib/args.c to intialize toys.optflags, toys.optargs, and
Rob Landley66a69d92012-01-16 01:44:17 -0600193entries in the toy's DEFINE_GLOBALS struct). When this is NULL, no option
194parsing is done before calling toy_main().</p></li>
Rob Landley6882ee82008-02-12 18:41:34 -0600195<li><p>int <b>flags</b> - Behavior flags for this command. The following flags are currently understood:</p>
196
197<ul>
198<li><b>TOYFLAG_USR</b> - Install this command under /usr</li>
199<li><b>TOYFLAG_BIN</b> - Install this command under /bin</li>
200<li><b>TOYFLAG_SBIN</b> - Install this command under /sbin</li>
201<li><b>TOYFLAG_NOFORK</b> - This command can be used as a shell builtin.</li>
202<li><b>TOYFLAG_UMASK</b> - Call umask(0) before running this command.</li>
Rob Landley66a69d92012-01-16 01:44:17 -0600203<li><b>TOYFLAG_STAYROOT</b> - Don't drop permissions for this command if toybox is installed SUID root.</li>
204<li><b>TOYFLAG_NEEDROOT</b> - This command cannot function unless run with root access.</li>
Rob Landley6882ee82008-02-12 18:41:34 -0600205</ul>
206<br>
207
208<p>These flags are combined with | (or). For example, to install a command
209in /usr/bin, or together TOYFLAG_USR|TOYFLAG_BIN.</p>
210</ul>
Rob Landley4e68de12007-12-13 07:00:27 -0600211</li>
212
Rob Landley66a69d92012-01-16 01:44:17 -0600213<li><p><b>struct toy_context toys</b> - global structure containing information
214common to all commands, initializd by toy_init() and defined in "toys.h".
215Members of this structure include:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600216<ul>
217<li><p>struct toy_list *<b>which</b> - a pointer to this command's toy_list
218structure. Mostly used to grab the name of the running command
219(toys->which.name).</p>
220</li>
221<li><p>int <b>exitval</b> - Exit value of this command. Defaults to zero. The
222error_exit() functions will return 1 if this is zero, otherwise they'll
223return this value.</p></li>
224<li><p>char **<b>argv</b> - "raw" command line options, I.E. the original
225unmodified string array passed in to main(). Note that modifying this changes
Rob Landley66a69d92012-01-16 01:44:17 -0600226"ps" output, and is not recommended. This array is null terminated; a NULL
227entry indicates the end of the array.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600228<p>Most commands don't use this field, instead the use optargs, optflags,
Rob Landley66a69d92012-01-16 01:44:17 -0600229and the fields in the DEFINE_GLOBALS struct initialized by get_optflags().</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600230</li>
231<li><p>unsigned <b>optflags</b> - Command line option flags, set by
Rob Landley66a69d92012-01-16 01:44:17 -0600232<a href="#lib_args">get_optflags()</a>. Indicates which of the command line options listed in
Rob Landley6882ee82008-02-12 18:41:34 -0600233toys->which.options occurred this time.</p>
234
235<p>The rightmost command line argument listed in toys->which.options sets bit
2361, the next one sets bit 2, and so on. This means the bits are set in the same
237order the binary digits would be listed if typed out as a string. For example,
238the option string "abcd" would parse the command line "-c" to set optflags to 2,
239"-a" would set optflags to 8, and "-bd" would set optflags to 6 (4|2).</p>
240
241<p>Only letters are relevant to optflags. In the string "a*b:c#d", d=1, c=2,
242b=4, a=8. The punctuation after a letter initializes global variables
243(see [TODO] DECLARE_GLOBALS() for details).</p>
244
Rob Landley66a69d92012-01-16 01:44:17 -0600245<p>For more information on option parsing, see <a href="#lib_args">get_optflags()</a>.</p>
Rob Landley6882ee82008-02-12 18:41:34 -0600246
247</li>
Rob Landley4e68de12007-12-13 07:00:27 -0600248<li><p>char **<b>optargs</b> - Null terminated array of arguments left over
249after get_optflags() removed all the ones it understood. Note: optarg[0] is
250the first argument, not the command name. Use toys.which->name for the command
251name.</p></li>
Rob Landley6882ee82008-02-12 18:41:34 -0600252<li><p>int <b>optc</b> - Optarg count, equivalent to argc but for
253optargs[].<p></li>
Rob Landley4e68de12007-12-13 07:00:27 -0600254<li><p>int <b>exithelp</b> - Whether error_exit() should print a usage message
255via help_main() before exiting. (True during option parsing, defaults to
256false afterwards.)</p></li>
Rob Landley66a69d92012-01-16 01:44:17 -0600257</ul>
Rob Landley4e68de12007-12-13 07:00:27 -0600258
Rob Landley66a69d92012-01-16 01:44:17 -0600259<li><p><b>union toy_union this</b> - Union of structures containing each
Rob Landley4e68de12007-12-13 07:00:27 -0600260command's global variables.</p>
261
Rob Landley6882ee82008-02-12 18:41:34 -0600262<p>Global variables are useful: they reduce the overhead of passing extra
263command line arguments between functions, they conveniently start prezeroed to
264save initialization costs, and the command line argument parsing infrastructure
265can also initialize global variables with its results.</p>
266
267<p>But since each toybox process can only run one command at a time, allocating
268space for global variables belonging to other commands you aren't currently
269running would be wasteful.</p>
270
271<p>Toybox handles this by encapsulating each command's global variables in
Rob Landley66a69d92012-01-16 01:44:17 -0600272a structure, and declaring a union of those structures with a single global
273instance (called "this"). The DEFINE_GLOBALS() macro contains the global
274variables that should go in the current command's global structure. Each
275variable can then be accessed as "this.commandname.varname".
Rob Landley6882ee82008-02-12 18:41:34 -0600276Generally, the macro TT is #defined to this.commandname so the variable
Rob Landley66a69d92012-01-16 01:44:17 -0600277can then be accessed as "TT.variable". See toys/hello.c for an example.</p>
Rob Landley6882ee82008-02-12 18:41:34 -0600278
Rob Landley66a69d92012-01-16 01:44:17 -0600279<p>A command that needs global variables should declare a structure to
Rob Landley4e68de12007-12-13 07:00:27 -0600280contain them all, and add that structure to this union. A command should never
281declare global variables outside of this, because such global variables would
282allocate memory when running other commands that don't use those global
283variables.</p>
284
Rob Landley66a69d92012-01-16 01:44:17 -0600285<p>The first few fields of this structure can be intialized by <a href="#lib_args">get_optargs()</a>,
Rob Landley4e68de12007-12-13 07:00:27 -0600286as specified by the options field off this command's toy_list entry. See
287the get_optargs() description in lib/args.c for details.</p>
288</li>
289
Rob Landley81b899d2007-12-18 02:02:47 -0600290<li><b>char toybuf[4096]</b> - a common scratch space buffer so
Rob Landley4e68de12007-12-13 07:00:27 -0600291commands don't need to allocate their own. Any command is free to use this,
292and it should never be directly referenced by functions in lib/ (although
293commands are free to pass toybuf in to a library function as an argument).</li>
294</ul>
295
Rob Landley6882ee82008-02-12 18:41:34 -0600296<p>The following functions are defined in main.c:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600297<ul>
298<li><p>struct toy_list *<b>toy_find</b>(char *name) - Return the toy_list
299structure for this command name, or NULL if not found.</p></li>
Rob Landley81b899d2007-12-18 02:02:47 -0600300<li><p>void <b>toy_init</b>(struct toy_list *which, char *argv[]) - fill out
301the global toys structure, calling get_optargs() if necessary.</p></li>
Rob Landley6882ee82008-02-12 18:41:34 -0600302<li><p>void <b>toy_exec</b>(char *argv[]) - Run a built-in command with
303arguments.</p>
304<p>Calls toy_find() on argv[0] (which must be just a command name
Rob Landley4e68de12007-12-13 07:00:27 -0600305without path). Returns if it can't find this command, otherwise calls
Rob Landley6882ee82008-02-12 18:41:34 -0600306toy_init(), toys->which.toy_main(), and exit() instead of returning.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600307
Rob Landley6882ee82008-02-12 18:41:34 -0600308<p>Use the library function xexec() to fall back to external executables
309in $PATH if toy_exec() can't find a built-in command. Note that toy_exec()
310does not strip paths before searching for a command, so "./command" will
311never match an internal command.</li>
312
313<li><p>void <b>toybox_main</b>(void) - the main function for the multiplexer
314command (I.E. "toybox"). Given a command name as its first argument, calls
315toy_exec() on its arguments. With no arguments, it lists available commands.
316If the first argument starts with "-" it lists each command with its default
317install path prepended.</p></li>
Rob Landley4e68de12007-12-13 07:00:27 -0600318
319</ul>
320
321<h3>Config.in</h3>
322
323<p>Top level configuration file in a stylized variant of
Rob Landley6882ee82008-02-12 18:41:34 -0600324<a href=http://kernel.org/doc/Documentation/kbuild/kconfig-language.txt>kconfig</a> format. Includes generated/Config.in.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600325
326<p>These files are directly used by "make menuconfig" to select which commands
327to build into toybox (thus generating a .config file), and by
Rob Landley6882ee82008-02-12 18:41:34 -0600328scripts/config2help.py to create generated/help.h.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600329
330<h3>Temporary files:</h3>
331
Rob Landley6882ee82008-02-12 18:41:34 -0600332<p>There is one temporary file in the top level source directory:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600333<ul>
334<li><p><b>.config</b> - Configuration file generated by kconfig, indicating
335which commands (and options to commands) are currently enabled. Used
Rob Landley6882ee82008-02-12 18:41:34 -0600336to make generated/config.h and determine which toys/*.c files to build.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600337
Rob Landley6882ee82008-02-12 18:41:34 -0600338<p>You can create a human readable "miniconfig" version of this file using
Rob Landley66a69d92012-01-16 01:44:17 -0600339<a href=http://landley.net/aboriginal/new_platform.html#miniconfig>these
Rob Landley6882ee82008-02-12 18:41:34 -0600340instructions</a>.</p>
341</li>
342</ul>
343
344<p>The "generated/" directory contains files generated from other source code
345in toybox. All of these files can be recreated by the build system, although
346some (such as generated/help.h) are shipped in release versions to reduce
347environmental dependencies (I.E. so you don't need python on your build
348system).</p>
349
350<ul>
351<li><p><b>generated/config.h</b> - list of CFG_SYMBOL and USE_SYMBOL() macros,
Rob Landley4e68de12007-12-13 07:00:27 -0600352generated from .config by a sed invocation in the top level Makefile.</p>
353
354<p>CFG_SYMBOL is a comple time constant set to 1 for enabled symbols and 0 for
Rob Landley6882ee82008-02-12 18:41:34 -0600355disabled symbols. This allows the use of normal if() statements to remove
356code at compile time via the optimizer's dead code elimination (which removes
357from the binary any code that cannot be reached). This saves space without
Rob Landley4e68de12007-12-13 07:00:27 -0600358cluttering the code with #ifdefs or leading to configuration dependent build
359breaks. (See the 1992 Usenix paper
Rob Landleyb6063de2012-01-29 13:54:13 -0600360<a href=http://doc.cat-v.org/henry_spencer/ifdef_considered_harmful.pdf>#ifdef
Rob Landley4e68de12007-12-13 07:00:27 -0600361Considered Harmful</a> for more information.)</p>
362
363<p>USE_SYMBOL(code) evaluates to the code in parentheses when the symbol
364is enabled, and nothing when the symbol is disabled. This can be used
365for things like varargs or variable declarations which can't always be
Rob Landley6882ee82008-02-12 18:41:34 -0600366eliminated by a simple test on CFG_SYMBOL. Note that
Rob Landley4e68de12007-12-13 07:00:27 -0600367(unlike CFG_SYMBOL) this is really just a variant of #ifdef, and can
368still result in configuration dependent build breaks. Use with caution.</p>
369</li>
370</ul>
371
Rob Landley81b899d2007-12-18 02:02:47 -0600372<p><h2>Directory toys/</h2></p>
Rob Landley4e68de12007-12-13 07:00:27 -0600373
374<h3>toys/Config.in</h3>
375
376<p>Included from the top level Config.in, contains one or more
377configuration entries for each command.</p>
378
Rob Landley81b899d2007-12-18 02:02:47 -0600379<p>Each command has a configuration entry matching the command name (although
380configuration symbols are uppercase and command names are lower case).
Rob Landley4e68de12007-12-13 07:00:27 -0600381Options to commands start with the command name followed by an underscore and
Rob Landley66a69d92012-01-16 01:44:17 -0600382the option name. Global options are attached to the "toybox" command,
Rob Landley4e68de12007-12-13 07:00:27 -0600383and thus use the prefix "TOYBOX_". This organization is used by
Rob Landley81b899d2007-12-18 02:02:47 -0600384scripts/cfg2files to select which toys/*.c files to compile for a given
385.config.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600386
Rob Landley66a69d92012-01-16 01:44:17 -0600387<p>A command with multiple names (or multiple similar commands implemented in
Rob Landley4e68de12007-12-13 07:00:27 -0600388the same .c file) should have config symbols prefixed with the name of their
389C file. I.E. config symbol prefixes are NEWTOY() names. If OLDTOY() names
390have config symbols they're options (symbols with an underscore and suffix)
391to the NEWTOY() name. (See toys/toylist.h)</p>
392
393<h3>toys/toylist.h</h3>
Rob Landley81b899d2007-12-18 02:02:47 -0600394<p>The first half of this file prototypes all the structures to hold
Rob Landleyda09b7f2007-12-20 06:29:59 -0600395global variables for each command, and puts them in toy_union. These
396prototypes are only included if the macro NEWTOY isn't defined (in which
397case NEWTOY is defined to a default value that produces function
398prototypes).</p>
Rob Landley81b899d2007-12-18 02:02:47 -0600399
Rob Landleyda09b7f2007-12-20 06:29:59 -0600400<p>The second half of this file lists all the commands in alphabetical
401order, along with their command line arguments and install location.
402Each command has an appropriate configuration guard so only the commands that
403are enabled wind up in the list.</p>
404
405<p>The first time this header is #included, it defines structures and
406produces function prototypes for the commands in the toys directory.</p>
407
408
409<p>The first time it's included, it defines structures and produces function
410prototypes.
411 This
Rob Landley81b899d2007-12-18 02:02:47 -0600412is used to initialize toy_list in main.c, and later in that file to initialize
413NEED_OPTIONS (to figure out whether the command like parsing logic is needed),
414and to put the help entries in the right order in toys/help.c.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600415
416<h3>toys/help.h</h3>
417
418<p>#defines two help text strings for each command: a single line
419command_help and an additinal command_help_long. This is used by help_main()
420in toys/help.c to display help for commands.</p>
421
422<p>Although this file is generated from Config.in help entries by
423scripts/config2help.py, it's shipped in release tarballs so you don't need
424python on the build system. (If you check code out of source control, or
425modify Config.in, then you'll need python installed to rebuild it.)</p>
426
427<p>This file contains help for all commands, regardless of current
428configuration, but only the currently enabled ones are entered into help_data[]
429in toys/help.c.</p>
430
Rob Landley81b899d2007-12-18 02:02:47 -0600431<h2>Directory lib/</h2>
Rob Landley4e68de12007-12-13 07:00:27 -0600432
Rob Landley7c04f012008-01-20 19:00:16 -0600433<p>lib: llist, getmountlist(), error_msg/error_exit, xmalloc(),
434strlcpy(), xexec(), xopen()/xread(), xgetcwd(), xabspath(), find_in_path(),
435itoa().</p>
436
Rob Landley66a69d92012-01-16 01:44:17 -0600437<a name="lib_args"><h3>lib/args.c</h3>
Rob Landley7c04f012008-01-20 19:00:16 -0600438
Rob Landley66a69d92012-01-16 01:44:17 -0600439<p>Toybox's main.c automatically parses command line options before calling the
440command's main function. Option parsing starts in get_optflags(), which stores
441results in the global structures "toys" (optflags and optargs) and "this".</p>
442
443<p>The option parsing infrastructure stores a bitfield in toys.optflags to
444indicate which options the current command line contained. Arguments
445attached to those options are saved into the command's global structure
446("this"). Any remaining command line arguments are collected together into
447the null-terminated array toys.optargs, with the length in toys.optc. (Note
448that toys.optargs does not contain the current command name at position zero,
449use "toys.which->name" for that.) The raw command line arguments get_optflags()
450parsed are retained unmodified in toys.argv[].</p>
451
452<p>Toybox's option parsing logic is controlled by an "optflags" string, using
453a format reminiscent of getopt's optargs but has several important differences.
454Toybox does not use the getopt()
455function out of the C library, get_optflags() is an independent implementation
456which doesn't permute the original arguments (and thus doesn't change how the
457command is displayed in ps and top), and has many features not present in
458libc optargs() (such as the ability to describe long options in the same string
459as normal options).</p>
460
461<p>Each command's NEWTOY() macro has an optflags string as its middle argument,
462which sets toy_list.options for that command to tell get_optflags() what
463command line arguments to look for, and what to do with them.
464If a command has no option
465definition string (I.E. the argument is NULL), option parsing is skipped
466for that command, which must look at the raw data in toys.argv to parse its
467own arguments. (If no currently enabled command uses option parsing,
468get_optflags() is optimized out of the resulting binary by the compiler's
469--gc-sections option.)</p>
470
471<p>You don't have to free the option strings, which point into the environment
472space (I.E. the string data is not copied). A TOYFLAG_NOFORK command
473that uses the linked list type "*" should free the list objects but not
474the data they point to, via "llist_free(TT.mylist, NULL);". (If it's not
475NOFORK, exit() will free all the malloced data anyway unless you want
476to implement a CONFIG_TOYBOX_FREE cleanup for it.)</p>
477
478<h4>Optflags format string</h4>
479
480<p>Note: the optflags option description string format is much more
481concisely described by a large comment at the top of lib/args.c.</p>
482
483<p>The general theory is that letters set optflags, and punctuation describes
484other actions the option parsing logic should take.</p>
485
486<p>For example, suppose the command line <b>command -b fruit -d walrus -a 42</b>
487is parsed using the optflags string "<b>a#b:c:d</b>". (I.E.
488toys.which->options="a#b:c:d" and argv = ["command", "-b", "fruit", "-d",
489"walrus", "-a", "42"]). When get_optflags() returns, the following data is
490available to command_main():
491
492<ul>
493<li><p>In <b>struct toys</b>:
494<ul>
495<li>toys.optflags = 13; // -a = 8 | -b = 4 | -d = 1</li>
496<li>toys.optargs[0] = "walrus"; // leftover argument</li>
497<li>toys.optargs[1] = NULL; // end of list</li>
498<li>toys.optc=1; // there was 1 leftover argument</li>
499<li>toys.argv[] = {"-b", "fruit", "-d", "walrus", "-a", "42"}; // The original command line arguments
500</ul>
501<p></li>
502
503<li><p>In <b>union this</b> (treated as <b>long this[]</b>):
504<ul>
505<li>this[0] = NULL; // -c didn't get an argument this time, so get_optflags() didn't change it and toys_init() zeroed "this" during setup.)</li>
506<li>this[1] = (long)"fruit"; // argument to -b</li>
507<li>this[2] = 42; // argument to -a</li>
508</ul>
509</p></li>
510</ul>
511
512<p>If the command's globals are:</p>
513
514<blockquote><pre>
515DECLARE_GLOBALS(
516 char *c;
517 char *b;
518 long a;
519)
520#define TT this.command
521</pre></blockquote>
522<p>That would mean TT.c == NULL, TT.b == "fruit", and TT.a == 42. (Remember,
523each entry that receives an argument must be a long or pointer, to line up
524with the array position. Right to left in the optflags string corresponds to
525top to bottom in DECLARE_GLOBALS().</p>
526
527<p><b>long toys.optflags</b></p>
528
529<p>Each option in the optflags string corresponds to a bit position in
530toys.optflags, with the same value as a corresponding binary digit. The
531rightmost argument is (1<<0), the next to last is (1<<1) and so on. If
Rob Landleyb4a0efa2012-02-06 21:15:19 -0600532the option isn't encountered while parsing argv[], its bit remains 0.</p>
533
534<p>For example,
Rob Landley66a69d92012-01-16 01:44:17 -0600535the optflags string "abcd" would parse the command line argument "-c" to set
536optflags to 2, "-a" would set optflags to 8, "-bd" would set optflags to
5376 (I.E. 4|2), and "-a -c" would set optflags to 10 (2|8).</p>
538
539<p>Only letters are relevant to optflags, punctuation is skipped: in the
540string "a*b:c#d", d=1, c=2, b=4, a=8. The punctuation after a letter
541usually indicate that the option takes an argument.</p>
542
Rob Landleyb4a0efa2012-02-06 21:15:19 -0600543<p>Since toys.optflags is an unsigned int, it only stores 32 bits. (Which is
544the amount a long would have on 32-bit platforms anyway; 64 bit code on
54532 bit platforms is too expensive to require in common code used by almost
546all commands.) Bit positions beyond the 1<<31 aren't recorded, but
547parsing higher options can still set global variables.</p>
548
Rob Landley66a69d92012-01-16 01:44:17 -0600549<p><b>Automatically setting global variables from arguments (union this)</b></p>
550
551<p>The following punctuation characters may be appended to an optflags
552argument letter, indicating the option takes an additional argument:</p>
553
554<ul>
555<li><b>:</b> - plus a string argument, keep most recent if more than one.</li>
556<li><b>*</b> - plus a string argument, appended to a linked list.</li>
Rob Landley66a69d92012-01-16 01:44:17 -0600557<li><b>@</b> - plus an occurrence counter (stored in a long)</li>
Rob Landleyb6063de2012-01-29 13:54:13 -0600558<li><b>#</b> - plus a signed long argument.
559<li><b>.</b> - plus a floating point argument (if CFG_TOYBOX_FLOAT).</li>
560<ul>The following can be appended to a float or double:
561<li><b>&lt;123</b> - error if argument is less than this</li>
562<li><b>&gt;123</b> - error if argument is greater than this</li>
563<li><b>=123</b> - default value if argument not supplied</li>
564</ul>
565<ul><li>Option parsing only understands <>= after . when CFG_TOYBOX_FLOAT
566is enabled. (Otherwise the code to determine where floating point constants
567end drops out. When disabled, it can reserve a global data slot for the
568argument so offsets won't change, but will never fill it out.). You can handle
569this by using the USE_BLAH() macros with C string concatenation, ala:
570"abc." USE_TOYBOX_FLOAT("<1.23>4.56=7.89") "def"</li></ul>
Rob Landley66a69d92012-01-16 01:44:17 -0600571</ul>
572
573<p>Arguments may occur with or without a space (I.E. "-a 42" or "-a42").
574The command line argument "-abc" may be interepreted many different ways:
575the optflags string "cba" sets toys.optflags = 7, "c:ba" sets toys.optflags=4
576and saves "ba" as the argument to -c, and "cb:a" sets optflags to 6 and saves
577"c" as the argument to -b.</p>
578
579<p>Options which have an argument fill in the corresponding slot in the global
580union "this" (see generated/globals.h), treating it as an array of longs
581with the rightmost saved in this[0]. Again using "a*b:c#d", "-c 42" would set
582this[0]=42; and "-b 42" would set this[1]="42"; each slot is left NULL if
583the corresponding argument is not encountered.</p>
584
585<p>This behavior is useful because the LP64 standard ensures long and pointer
Rob Landleyb4a0efa2012-02-06 21:15:19 -0600586are the same size. C99 guarantees structure members will occur in memory
587in the same order they're declared, and that padding won't be inserted between
Rob Landley66a69d92012-01-16 01:44:17 -0600588consecutive variables of register size. Thus the first few entries can
589be longs or pointers corresponding to the saved arguments.</p>
590
591<p><b>char *toys.optargs[]</b></p>
592
593<p>Command line arguments in argv[] which are not consumed by option parsing
594(I.E. not recognized either as -flags or arguments to -flags) will be copied
595to toys.optargs[], with the length of that array in toys.optc.
596(When toys.optc is 0, no unrecognized command line arguments remain.)
597The order of entries is preserved, and as with argv[] this new array is also
598terminated by a NULL entry.</p>
599
600<p>Option parsing can require a minimum or maximum number of optargs left
601over, by adding "<1" (read "at least one") or ">9" ("at most nine") to the
602start of the optflags string.</p>
603
604<p>The special argument "--" terminates option parsing, storing all remaining
605arguments in optargs. The "--" itself is consumed.</p>
606
607<p><b>Other optflags control characters</b></p>
608
609<p>The following characters may occur at the start of each command's
610optflags string, before any options that would set a bit in toys.optflags:</p>
611
612<ul>
613<li><b>^</b> - stop at first nonoption argument (for nice, xargs...)</li>
614<li><b>?</b> - allow unknown arguments (pass non-option arguments starting
615with - through to optargs instead of erroring out).</li>
616<li><b>&amp;</b> - the first argument has imaginary dash (ala tar/ps. If given twice, all arguments have imaginary dash.)</li>
617<li><b>&lt;</b> - must be followed by a decimal digit indicating at least this many leftover arguments are needed in optargs (default 0)</li>
618<li><b>&gt;</b> - must be followed by a decimal digit indicating at most this many leftover arguments allowed (default MAX_INT)</li>
619</ul>
620
621<p>The following characters may be appended to an option character, but do
622not by themselves indicate an extra argument should be saved in this[].
623(Technically any character not recognized as a control character sets an
624optflag, but letters are never control characters.)</p>
625
626<ul>
627<li><b>^</b> - stop parsing options after encountering this option, everything else goes into optargs.</li>
628<li><b>|</b> - this option is required. If more than one marked, only one is required.</li>
629<li><b>+X</b> enabling this option also enables option X (switch bit on).</li>
630<li><b>~X</b> enabling this option disables option X (switch bit off).</li>
631<li><b>!X</b> this option cannot be used in combination with X (die with error).</li>
632<li><b>[yz]</b> this option requires at least one of y or z to also be enabled.</li>
633</ul>
634
Rob Landleyb6063de2012-01-29 13:54:13 -0600635<p>The following may be appended to a float or double:</p>
636
637<ul>
638<li><b>&lt;123</b> - error if argument is less than this</li>
639<li><b>&gt;123</b> - error if argument is greater than this</li>
640<li><b>=123</b> - default value if argument not supplied</li>
641</ul>
642
643<p>Option parsing only understands <>= after . when CFG_TOYBOX_FLOAT
644is enabled. (Otherwise the code to determine where floating point constants
645end drops out. When disabled, it can reserve a global data slot for the
646argument so offsets won't change, but will never fill it out.). You can handle
647this by using the USE_BLAH() macros with C string concatenation, ala:</p>
648
649<blockquote>"abc." USE_TOYBOX_FLOAT("<1.23>4.56=7.89") "def"</blockquote>
650
Rob Landley66a69d92012-01-16 01:44:17 -0600651<p><b>--longopts</b></p>
652
653<p>The optflags string can contain long options, which are enclosed in
654parentheses. They may be appended to an existing option character, in
655which case the --longopt is a synonym for that option, ala "a:(--fred)"
656which understands "-a blah" or "--fred blah" as synonyms.</p>
657
658<p>Longopts may also appear before any other options in the optflags string,
659in which case they have no corresponding short argument, but instead set
660their own bit based on position. So for "(walrus)#(blah)xy:z" "command
661--walrus 42" would set toys.optflags = 16 (-z = 1, -y = 2, -x = 4, --blah = 8)
662and would assign this[1] = 42;</p>
663
664<p>A short option may have multiple longopt synonyms, "a(one)(two)", but
665each "bare longopt" (ala "(one)(two)abc" before any option characters)
666always sets its own bit (although you can group them with +X).</p>
Rob Landley7c04f012008-01-20 19:00:16 -0600667
Rob Landley81b899d2007-12-18 02:02:47 -0600668<h2>Directory scripts/</h2>
Rob Landley4e68de12007-12-13 07:00:27 -0600669
670<h3>scripts/cfg2files.sh</h3>
671
672<p>Run .config through this filter to get a list of enabled commands, which
673is turned into a list of files in toys via a sed invocation in the top level
674Makefile.
675</p>
676
Rob Landley81b899d2007-12-18 02:02:47 -0600677<h2>Directory kconfig/</h2>
Rob Landley4e68de12007-12-13 07:00:27 -0600678
679<p>Menuconfig infrastructure copied from the Linux kernel. See the
680Linux kernel's Documentation/kbuild/kconfig-language.txt</p>
681
Rob Landley66a69d92012-01-16 01:44:17 -0600682<a name="generated">
683<h2>Directory generated/</h2>
684
685<p>All the files in this directory except the README are generated by the
686build. (See scripts/make.sh)</p>
687
688<ul>
689<li><p><b>config.h</b> - CFG_COMMAND and USE_COMMAND() macros set by menuconfig via .config.</p></li>
690
691<li><p><b>Config.in</b> - Kconfig entries for each command. Included by top level Config.in. The help text in here is used to generated help.h</p></li>
692
693<li><p><b>help.h</b> - Help text strings for use by "help" command. Building
694this file requires python on the host system, so the prebuilt file is shipped
695in the build tarball to avoid requiring python to build toybox.</p></li>
696
697<li><p><b>newtoys.h</b> - List of NEWTOY() or OLDTOY() macros for all available
698commands. Associates command_main() functions with command names, provides
699option string for command line parsing (<a href="#lib_args">see lib/args.c</a>),
700specifies where to install each command and whether toysh should fork before
701calling it.</p></li>
702</ul>
703
704<p>Everything in this directory is a derivative file produced from something
705else. The entire directory is deleted by "make distclean".</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600706<!--#include file="footer.html" -->