blob: 725bebdc64ecc2c7f9a7818899c2d5433b6f07f9 [file] [log] [blame]
Rob Landley4e68de12007-12-13 07:00:27 -06001<!--#include file="header.html" -->
2
Rob Landley27048342013-08-18 14:24:59 -05003<p><h1><a name="style" /><a href="#style">Code style</a></h1></p>
Rob Landleye7c9a6d2012-02-28 06:34:09 -06004
5<p>The primary goal of toybox is _simple_ code. Keeping the code small is
Rob Landleyed6ed622012-03-06 20:49:03 -06006second, with speed and lots of features coming in somewhere after that.
7(For more on that, see the <a href=design.html>design</a> page.)</p>
Rob Landleye7c9a6d2012-02-28 06:34:09 -06008
9<p>A simple implementation usually takes up fewer lines of source code,
10meaning more code can fit on the screen at once, meaning the programmer can
11see more of it on the screen and thus keep more if in their head at once.
Rob Landleyed6ed622012-03-06 20:49:03 -060012This helps code auditing and thus reduces bugs. That said, sometimes being
13more explicit is preferable to being clever enough to outsmart yourself:
14don't be so terse your code is unreadable.</p>
Rob Landley5a0660f2007-12-27 21:36:44 -060015
Rob Landley7aa651a2012-11-13 17:14:08 -060016<p>Toybox source uses two spaces per indentation level, and wraps at 80
17columns.</p>
Rob Landley5a0660f2007-12-27 21:36:44 -060018
19<p>Gotos are allowed for error handling, and for breaking out of
20nested loops. In general, a goto should only jump forward (not back), and
21should either jump to the end of an outer loop, or to error handling code
22at the end of the function. Goto labels are never indented: they override the
23block structure of the file. Putting them at the left edge makes them easy
24to spot as overrides to the normal flow of control, which they are.</p>
25
Rob Landley27048342013-08-18 14:24:59 -050026<p><h1><a name="building" /><a href="#building">Building Toybox</a></h1></p>
Rob Landleye7c9a6d2012-02-28 06:34:09 -060027
28<p>Toybox is configured using the Kconfig language pioneered by the Linux
29kernel, and adopted by many other projects (uClibc, OpenEmbedded, etc).
30This generates a ".config" file containing the selected options, which
Rob Landley7aa651a2012-11-13 17:14:08 -060031controls which features are included when compiling toybox.</p>
Rob Landleye7c9a6d2012-02-28 06:34:09 -060032
33<p>Each configuration option has a default value. The defaults indicate the
34"maximum sane configuration", I.E. if the feature defaults to "n" then it
35either isn't complete or is a special-purpose option (such as debugging
36code) that isn't intended for general purpose use.</p>
37
38<p>The standard build invocation is:</p>
39
40<ul>
41<li>make defconfig #(or menuconfig)</li>
42<li>make</li>
43<li>make install</li>
44</ul>
45
46<p>Type "make help" to see all available build options.</p>
47
48<p>The file "configure" contains a number of environment variable definitions
49which influence the build, such as specifying which compiler to use or where
50to install the resulting binaries. This file is included by the build, but
51accepts existing definitions of the environment variables, so it may be sourced
52or modified by the developer before building and the definitions exported
53to the environment will take precedence.</p>
54
55<p>(To clarify: "configure" describes the build and installation environment,
56".config" lists the features selected by defconfig/menuconfig.)</p>
Rob Landley6882ee82008-02-12 18:41:34 -060057
Rob Landley27048342013-08-18 14:24:59 -050058<p><h1><a name="running"><a href="#running">Running a command</a></h1></p>
59
60<h2>main</h2>
61
62<p>The toybox main() function is at the end of main.c at the top level. It has
63two possible codepaths, only one of which is configured into any given build
64of toybox.</p>
65
66<p>If CONFIG_SINGLE is selected, toybox is configured to contain only a single
67command, so most of the normal setup can be skipped. In this case the
68multiplexer isn't used, instead main() calls toy_singleinit() (also in main.c)
69to set up global state and parse command line arguments, calls the command's
70main function out of toy_list (in the CONFIG_SINGLE case the array has a single entry, no need to search), and if the function returns instead of exiting
71it flushes stdout (detecting error) and returns toys.exitval.</p>
72
73<p>When CONFIG_SINGLE is not selected, main() uses basename() to find the
74name it was run as, shifts its argument list one to the right so it lines up
75with where the multiplexer function expects it, and calls toybox_main(). This
76leverages the multiplexer command's infrastructure to find and run the
77appropriate command. (A command name starting with "toybox" will
78recursively call toybox_main(); you can go "./toybox toybox toybox toybox ls"
79if you want to...)</p>
80
81<h2>toybox_main</h2>
82
83<p>The toybox_main() function is also in main,c. It handles a possible
84--help option ("toybox --help ls"), prints the list of available commands if no
85arguments were provided to the multiplexer (or with full path names if any
86other option is provided before a command name, ala "toybox --list").
87Otherwise it calls toy_exec() on its argument list.</p>
88
89<p>Note that the multiplexer is the first entry in toy_list (the rest of the
90list is sorted alphabetically to allow binary search), so toybox_main can
91cheat and just grab the first entry to quickly set up its context without
92searching. Since all command names go through the multiplexer at least once
93in the non-TOYBOX_SINGLE case, this avoids a redundant search of
94the list.</p>
95
96<p>The toy_exec() function is also in main.c. It performs toy_find() to
97perform a binary search on the toy_list array to look up the command's
98entry by name and saves it in the global variable which, calls toy_init()
99to parse command line arguments and set up global state (using which->options),
100and calls the appropriate command's main() function (which->toy_main). On
101return it flushes all pending ansi FILE * I/O, detects if stdout had an
102error, and then calls xexit() (which uses toys.exitval).</p>
103
104<p><h1><a name="infrastructure" /><a href="#infrastructure">Infrastructure</a></h1></p>
Rob Landley4e68de12007-12-13 07:00:27 -0600105
Rob Landley7c04f012008-01-20 19:00:16 -0600106<p>The toybox source code is in following directories:</p>
107<ul>
108<li>The <a href="#top">top level directory</a> contains the file main.c (were
109execution starts), the header file toys.h (included by every command), and
110other global infrastructure.</li>
111<li>The <a href="#lib">lib directory</a> contains common functions shared by
Rob Landleyc8d0da52012-07-15 17:47:08 -0500112multiple commands:</li>
113<ul>
114<li><a href="#lib_lib">lib/lib.c</a></li>
115<li><a href="#lib_llist">lib/llist.c</a></li>
116<li><a href="#lib_args">lib/args.c</a></li>
117<li><a href="#lib_dirtree">lib/dirtree.c</a></li>
118</ul>
Rob Landley7c04f012008-01-20 19:00:16 -0600119<li>The <a href="#toys">toys directory</a> contains the C files implementating
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500120each command. Currently it contains three subdirectories:
121posix, lsb, and other.</li>
Rob Landley7c04f012008-01-20 19:00:16 -0600122<li>The <a href="#scripts">scripts directory</a> contains the build and
123test infrastructure.</li>
124<li>The <a href="#kconfig">kconfig directory</a> contains the configuration
125infrastructure implementing menuconfig (copied from the Linux kernel).</li>
126<li>The <a href="#generated">generated directory</a> contains intermediate
127files generated from other parts of the source code.</li>
128</ul>
Rob Landley4e68de12007-12-13 07:00:27 -0600129
Rob Landleybbe500e2012-02-26 21:53:15 -0600130<a name="adding" />
Rob Landley7c04f012008-01-20 19:00:16 -0600131<p><h1>Adding a new command</h1></p>
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500132<p>To add a new command to toybox, add a C file implementing that command under
Rob Landley7c04f012008-01-20 19:00:16 -0600133the toys directory. No other files need to be modified; the build extracts
Rob Landley6882ee82008-02-12 18:41:34 -0600134all the information it needs (such as command line arguments) from specially
Rob Landley7c04f012008-01-20 19:00:16 -0600135formatted comments and macros in the C file. (See the description of the
Rob Landleye7c9a6d2012-02-28 06:34:09 -0600136<a href="#generated">"generated" directory</a> for details.)</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600137
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500138<p>Currently there are three subdirectories under "toys", one for commands
139defined by the POSIX standard, one for commands defined by the Linux Standard
140Base, and one for all other commands. (This is just for developer convenience
141sorting them, the directories are otherwise functionally identical.)</p>
142
143<p>An easy way to start a new command is copy the file "toys/other/hello.c" to
Rob Landley7c04f012008-01-20 19:00:16 -0600144the name of the new command, and modify this copy to implement the new command.
Rob Landley6882ee82008-02-12 18:41:34 -0600145This file is an example command meant to be used as a "skeleton" for
Rob Landley7c04f012008-01-20 19:00:16 -0600146new commands (more or less by turning every instance of "hello" into the
147name of your command, updating the command line arguments, globals, and
148help data, and then filling out its "main" function with code that does
Rob Landley6882ee82008-02-12 18:41:34 -0600149something interesting). It provides examples of all the build infrastructure
150(including optional elements like command line argument parsing and global
151variables that a "hello world" program doesn't strictly need).</p>
Rob Landley7c04f012008-01-20 19:00:16 -0600152
153<p>Here's a checklist of steps to turn hello.c into another command:</p>
154
155<ul>
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500156<li><p>First "cd toys/other" and "cp hello.c yourcommand.c". Note that the name
Rob Landley7c04f012008-01-20 19:00:16 -0600157of this file is significant, it's the name of the new command you're adding
158to toybox. Open your new file in your favorite editor.</p></li>
159
160<li><p>Change the one line comment at the top of the file (currently
161"hello.c - A hello world program") to describe your new file.</p></li>
162
163<li><p>Change the copyright notice to your name, email, and the current
164year.</p></li>
165
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500166<li><p>Give a URL to the relevant standards document, where applicable.
167(Sample links to SUSv4 and LSB are provided, feel free to link to other
168documentation or standards as appropriate.)</p></li>
Rob Landley7c04f012008-01-20 19:00:16 -0600169
Rob Landley66a69d92012-01-16 01:44:17 -0600170<li><p>Update the USE_YOURCOMMAND(NEWTOY(yourcommand,"blah",0)) line.
171The NEWTOY macro fills out this command's <a href="#toy_list">toy_list</a>
172structure. The arguments to the NEWTOY macro are:</p>
173
174<ol>
175<li><p>the name used to run your command</p></li>
176<li><p>the command line argument <a href="#lib_args">option parsing string</a> (NULL if none)</p></li>
177<li><p>a bitfield of TOYFLAG values
178(defined in toys.h) providing additional information such as where your
179command should be installed on a running system, whether to blank umask
180before running, whether or not the command must run as root (and thus should
181retain root access if installed SUID), and so on.</p></li>
182</ol>
183</li>
Rob Landley7c04f012008-01-20 19:00:16 -0600184
185<li><p>Change the kconfig data (from "config YOURCOMMAND" to the end of the
186comment block) to supply your command's configuration and help
187information. The uppper case config symbols are used by menuconfig, and are
188also what the CFG_ and USE_() macros are generated from (see [TODO]). The
189help information here is used by menuconfig, and also by the "help" command to
190describe your new command. (See [TODO] for details.) By convention,
Rob Landley66a69d92012-01-16 01:44:17 -0600191unfinished commands default to "n" and finished commands default to "y",
192so "make defconfig" selects all finished commands. (Note, "finished" means
193"ready to be used", not that it'll never change again.)<p>
194
195<p>Each help block should start with a "usage: yourcommand" line explaining
196any command line arguments added by this config option. The "help" command
197outputs this text, and scripts/config2help.c in the build infrastructure
198collates these usage lines for commands with multiple configuration
199options when producing generated/help.h.</p>
200</li>
Rob Landley7c04f012008-01-20 19:00:16 -0600201
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500202<li><p>Change the "#define FOR_hello" line to "#define FOR_yourcommand" right
203before the "#include <toys.h>". (This selects the appropriate FLAG_ macros and
204does a "#define TT this.yourcommand" so you can access the global variables
205out of the space-saving union of structures. If you aren't using any command
206flag bits and aren't defining a GLOBAL block, you can delete this line.)</p></li>
Rob Landley7c04f012008-01-20 19:00:16 -0600207
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500208<li><p>Update the GLOBALS() macro to contain your command's global
209variables. If your command has no global variables, delete this macro.</p>
210
211<p>Variables in the GLOBALS() block are are stored in a space saving
212<a href="#toy_union">union of structures</a> format, which may be accessed
213using the TT macro as if TT were a global structure (so TT.membername).
214If you specified two-character command line arguments in
215NEWTOY(), the first few global variables will be initialized by the automatic
216argument parsing logic, and the type and order of these variables must
217correspond to the arguments specified in NEWTOY().
218(See <a href="#lib_args">lib/args.c</a> for details.)</p></li>
Rob Landley7c04f012008-01-20 19:00:16 -0600219
220<li><p>Rename hello_main() to yourcommand_main(). This is the main() function
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500221where execution of your command starts. Your command line options are
222already sorted into this.optflags, this.optargs, this.optc, and the GLOBALS()
223as appropriate by the time this function is called. (See
224<a href="#lib_args">get_optflags()</a> for details.</p></li>
Rob Landley7c04f012008-01-20 19:00:16 -0600225</ul>
226
Rob Landley85a32412013-12-27 06:53:15 -0600227<a name="headers" /><h2>Headers.</h2>
228
229<p>Commands generally don't have their own headers. If it's common code
230it can live in lib/, if it isn't put it in the command's .c file. (The line
231between implementing multiple commands in a C file via OLDTOY() to share
232infrastructure and moving that shared infrastructure to lib/ is a judgement
233call. Try to figure out which is simplest.)</p>
234
235<p>The top level toys.h should #include all the standard (posix) headers
236that any command uses. (Partly this is friendly to ccache and partly this
237makes the command implementations shorter.) Individual commands should only
238need to include nonstandard headers that might prevent that command from
239building in some context we'd care about (and thus requiring that command to
240be disabled to avoid a build break).</p>
241
242<p>Target-specific stuff (differences between compiler versions, libc versions,
243or operating systems) should be confined to lib/portability.h and
244lib/portability.c. (There's even some minimal compile-time environment probing
245that writes data to generated/portability.h, see scripts/genconfig.sh.)</p>
246
247<p>Only include linux/*.h headers from individual commands (not from other
248headers), and only if you really need to. Data that varies per architecture
249is a good reason to include a header. If you just need a couple constants
250that haven't changed since the 1990's, it's ok to #define them yourself or
251just use the constant inline with a comment explaining what it is. (A
252#define that's only used once isn't really helping.)</p>
253
Rob Landley7c04f012008-01-20 19:00:16 -0600254<p><a name="top" /><h2>Top level directory.</h2></p>
255
Rob Landley66a69d92012-01-16 01:44:17 -0600256<p>This directory contains global infrastructure.</p>
257
258<h3>toys.h</h3>
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500259<p>Each command #includes "toys.h" as part of its standard prolog. It
260may "#define FOR_commandname" before doing so to get some extra entries
261specific to this command.</p>
Rob Landley66a69d92012-01-16 01:44:17 -0600262
263<p>This file sucks in most of the commonly used standard #includes, so
264individual files can just #include "toys.h" and not have to worry about
265stdargs.h and so on. Individual commands still need to #include
266special-purpose headers that may not be present on all systems (and thus would
267prevent toybox from building that command on such a system with that command
268enabled). Examples include regex support, any "linux/" or "asm/" headers, mtab
269support (mntent.h and sys/mount.h), and so on.</p>
270
271<p>The toys.h header also defines structures for most of the global variables
272provided to each command by toybox_main(). These are described in
273detail in the description for main.c, where they are initialized.</p>
274
275<p>The global variables are grouped into structures (and a union) for space
276savings, to more easily track the amount of memory consumed by them,
277so that they may be automatically cleared/initialized as needed, and so
278that access to global variables is more easily distinguished from access to
279local variables.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600280
281<h3>main.c</h3>
282<p>Contains the main() function where execution starts, plus
283common infrastructure to initialize global variables and select which command
Rob Landley6882ee82008-02-12 18:41:34 -0600284to run. The "toybox" multiplexer command also lives here. (This is the
Rob Landley7c04f012008-01-20 19:00:16 -0600285only command defined outside of the toys directory.)</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600286
Rob Landley6882ee82008-02-12 18:41:34 -0600287<p>Execution starts in main() which trims any path off of the first command
288name and calls toybox_main(), which calls toy_exec(), which calls toy_find()
Rob Landley66a69d92012-01-16 01:44:17 -0600289and toy_init() before calling the appropriate command's function from
290toy_list[] (via toys.which->toy_main()).
Rob Landley6882ee82008-02-12 18:41:34 -0600291If the command is "toybox", execution recurses into toybox_main(), otherwise
292the call goes to the appropriate commandname_main() from a C file in the toys
293directory.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600294
Rob Landley6882ee82008-02-12 18:41:34 -0600295<p>The following global variables are defined in main.c:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600296<ul>
Rob Landley66a69d92012-01-16 01:44:17 -0600297<a name="toy_list" />
298<li><p><b>struct toy_list toy_list[]</b> - array describing all the
Rob Landley4e68de12007-12-13 07:00:27 -0600299commands currently configured into toybox. The first entry (toy_list[0]) is
300for the "toybox" multiplexer command, which runs all the other built-in commands
301without symlinks by using its first argument as the name of the command to
302run and the rest as that command's argument list (ala "./toybox echo hello").
303The remaining entries are the commands in alphabetical order (for efficient
304binary search).</p>
305
306<p>This is a read-only array initialized at compile time by
Rob Landley6882ee82008-02-12 18:41:34 -0600307defining macros and #including generated/newtoys.h.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600308
Rob Landley66a69d92012-01-16 01:44:17 -0600309<p>Members of struct toy_list (defined in "toys.h") include:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600310<ul>
311<li><p>char *<b>name</b> - the name of this command.</p></li>
312<li><p>void (*<b>toy_main</b>)(void) - function pointer to run this
313command.</p></li>
314<li><p>char *<b>options</b> - command line option string (used by
315get_optflags() in lib/args.c to intialize toys.optflags, toys.optargs, and
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500316entries in the toy's GLOBALS struct). When this is NULL, no option
Rob Landley66a69d92012-01-16 01:44:17 -0600317parsing is done before calling toy_main().</p></li>
Rob Landley6882ee82008-02-12 18:41:34 -0600318<li><p>int <b>flags</b> - Behavior flags for this command. The following flags are currently understood:</p>
319
320<ul>
321<li><b>TOYFLAG_USR</b> - Install this command under /usr</li>
322<li><b>TOYFLAG_BIN</b> - Install this command under /bin</li>
323<li><b>TOYFLAG_SBIN</b> - Install this command under /sbin</li>
324<li><b>TOYFLAG_NOFORK</b> - This command can be used as a shell builtin.</li>
325<li><b>TOYFLAG_UMASK</b> - Call umask(0) before running this command.</li>
Rob Landley66a69d92012-01-16 01:44:17 -0600326<li><b>TOYFLAG_STAYROOT</b> - Don't drop permissions for this command if toybox is installed SUID root.</li>
327<li><b>TOYFLAG_NEEDROOT</b> - This command cannot function unless run with root access.</li>
Rob Landley6882ee82008-02-12 18:41:34 -0600328</ul>
329<br>
330
331<p>These flags are combined with | (or). For example, to install a command
332in /usr/bin, or together TOYFLAG_USR|TOYFLAG_BIN.</p>
333</ul>
Rob Landley4e68de12007-12-13 07:00:27 -0600334</li>
335
Rob Landley66a69d92012-01-16 01:44:17 -0600336<li><p><b>struct toy_context toys</b> - global structure containing information
337common to all commands, initializd by toy_init() and defined in "toys.h".
338Members of this structure include:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600339<ul>
340<li><p>struct toy_list *<b>which</b> - a pointer to this command's toy_list
341structure. Mostly used to grab the name of the running command
342(toys->which.name).</p>
343</li>
344<li><p>int <b>exitval</b> - Exit value of this command. Defaults to zero. The
345error_exit() functions will return 1 if this is zero, otherwise they'll
346return this value.</p></li>
347<li><p>char **<b>argv</b> - "raw" command line options, I.E. the original
348unmodified string array passed in to main(). Note that modifying this changes
Rob Landley66a69d92012-01-16 01:44:17 -0600349"ps" output, and is not recommended. This array is null terminated; a NULL
350entry indicates the end of the array.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600351<p>Most commands don't use this field, instead the use optargs, optflags,
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500352and the fields in the GLOBALS struct initialized by get_optflags().</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600353</li>
354<li><p>unsigned <b>optflags</b> - Command line option flags, set by
Rob Landley66a69d92012-01-16 01:44:17 -0600355<a href="#lib_args">get_optflags()</a>. Indicates which of the command line options listed in
Rob Landley6882ee82008-02-12 18:41:34 -0600356toys->which.options occurred this time.</p>
357
358<p>The rightmost command line argument listed in toys->which.options sets bit
3591, the next one sets bit 2, and so on. This means the bits are set in the same
360order the binary digits would be listed if typed out as a string. For example,
361the option string "abcd" would parse the command line "-c" to set optflags to 2,
362"-a" would set optflags to 8, and "-bd" would set optflags to 6 (4|2).</p>
363
364<p>Only letters are relevant to optflags. In the string "a*b:c#d", d=1, c=2,
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500365b=4, a=8. Punctuation after a letter initializes global variables at the
366start of the GLOBALS() block (see <a href="#toy_union">union toy_union this</a>
367for details).</p>
368
369<p>The build infrastructure creates FLAG_ macros for each option letter,
370corresponding to the bit position, so you can check (toys.optflags & FLAG_x)
371to see if a flag was specified. (The correct set of FLAG_ macros is selected
372by defining FOR_mycommand before #including toys.h. The macros live in
373toys/globals.h which is generated by scripts/make.sh.)</p>
Rob Landley6882ee82008-02-12 18:41:34 -0600374
Rob Landley66a69d92012-01-16 01:44:17 -0600375<p>For more information on option parsing, see <a href="#lib_args">get_optflags()</a>.</p>
Rob Landley6882ee82008-02-12 18:41:34 -0600376
377</li>
Rob Landley4e68de12007-12-13 07:00:27 -0600378<li><p>char **<b>optargs</b> - Null terminated array of arguments left over
379after get_optflags() removed all the ones it understood. Note: optarg[0] is
380the first argument, not the command name. Use toys.which->name for the command
381name.</p></li>
Rob Landley6882ee82008-02-12 18:41:34 -0600382<li><p>int <b>optc</b> - Optarg count, equivalent to argc but for
383optargs[].<p></li>
Rob Landley4e68de12007-12-13 07:00:27 -0600384<li><p>int <b>exithelp</b> - Whether error_exit() should print a usage message
385via help_main() before exiting. (True during option parsing, defaults to
386false afterwards.)</p></li>
Rob Landley66a69d92012-01-16 01:44:17 -0600387</ul>
Rob Landley4e68de12007-12-13 07:00:27 -0600388
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500389<a name="toy_union" />
Rob Landley66a69d92012-01-16 01:44:17 -0600390<li><p><b>union toy_union this</b> - Union of structures containing each
Rob Landley4e68de12007-12-13 07:00:27 -0600391command's global variables.</p>
392
Rob Landley6882ee82008-02-12 18:41:34 -0600393<p>Global variables are useful: they reduce the overhead of passing extra
394command line arguments between functions, they conveniently start prezeroed to
395save initialization costs, and the command line argument parsing infrastructure
396can also initialize global variables with its results.</p>
397
398<p>But since each toybox process can only run one command at a time, allocating
399space for global variables belonging to other commands you aren't currently
400running would be wasteful.</p>
401
402<p>Toybox handles this by encapsulating each command's global variables in
Rob Landley66a69d92012-01-16 01:44:17 -0600403a structure, and declaring a union of those structures with a single global
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500404instance (called "this"). The GLOBALS() macro contains the global
Rob Landley66a69d92012-01-16 01:44:17 -0600405variables that should go in the current command's global structure. Each
406variable can then be accessed as "this.commandname.varname".
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500407If you #defined FOR_commandname before including toys.h, the macro TT is
408#defined to this.commandname so the variable can then be accessed as
409"TT.variable". See toys/hello.c for an example.</p>
Rob Landley6882ee82008-02-12 18:41:34 -0600410
Rob Landley66a69d92012-01-16 01:44:17 -0600411<p>A command that needs global variables should declare a structure to
Rob Landley4e68de12007-12-13 07:00:27 -0600412contain them all, and add that structure to this union. A command should never
413declare global variables outside of this, because such global variables would
414allocate memory when running other commands that don't use those global
415variables.</p>
416
Rob Landley66a69d92012-01-16 01:44:17 -0600417<p>The first few fields of this structure can be intialized by <a href="#lib_args">get_optargs()</a>,
Rob Landley4e68de12007-12-13 07:00:27 -0600418as specified by the options field off this command's toy_list entry. See
419the get_optargs() description in lib/args.c for details.</p>
420</li>
421
Rob Landley81b899d2007-12-18 02:02:47 -0600422<li><b>char toybuf[4096]</b> - a common scratch space buffer so
Rob Landley4e68de12007-12-13 07:00:27 -0600423commands don't need to allocate their own. Any command is free to use this,
424and it should never be directly referenced by functions in lib/ (although
425commands are free to pass toybuf in to a library function as an argument).</li>
426</ul>
427
Rob Landley6882ee82008-02-12 18:41:34 -0600428<p>The following functions are defined in main.c:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600429<ul>
430<li><p>struct toy_list *<b>toy_find</b>(char *name) - Return the toy_list
431structure for this command name, or NULL if not found.</p></li>
Rob Landley81b899d2007-12-18 02:02:47 -0600432<li><p>void <b>toy_init</b>(struct toy_list *which, char *argv[]) - fill out
433the global toys structure, calling get_optargs() if necessary.</p></li>
Rob Landley6882ee82008-02-12 18:41:34 -0600434<li><p>void <b>toy_exec</b>(char *argv[]) - Run a built-in command with
435arguments.</p>
436<p>Calls toy_find() on argv[0] (which must be just a command name
Rob Landley4e68de12007-12-13 07:00:27 -0600437without path). Returns if it can't find this command, otherwise calls
Rob Landley6882ee82008-02-12 18:41:34 -0600438toy_init(), toys->which.toy_main(), and exit() instead of returning.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600439
Rob Landley6882ee82008-02-12 18:41:34 -0600440<p>Use the library function xexec() to fall back to external executables
441in $PATH if toy_exec() can't find a built-in command. Note that toy_exec()
442does not strip paths before searching for a command, so "./command" will
443never match an internal command.</li>
444
445<li><p>void <b>toybox_main</b>(void) - the main function for the multiplexer
446command (I.E. "toybox"). Given a command name as its first argument, calls
447toy_exec() on its arguments. With no arguments, it lists available commands.
448If the first argument starts with "-" it lists each command with its default
449install path prepended.</p></li>
Rob Landley4e68de12007-12-13 07:00:27 -0600450
451</ul>
452
453<h3>Config.in</h3>
454
455<p>Top level configuration file in a stylized variant of
Rob Landley6882ee82008-02-12 18:41:34 -0600456<a href=http://kernel.org/doc/Documentation/kbuild/kconfig-language.txt>kconfig</a> format. Includes generated/Config.in.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600457
458<p>These files are directly used by "make menuconfig" to select which commands
459to build into toybox (thus generating a .config file), and by
Rob Landley6882ee82008-02-12 18:41:34 -0600460scripts/config2help.py to create generated/help.h.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600461
462<h3>Temporary files:</h3>
463
Rob Landley6882ee82008-02-12 18:41:34 -0600464<p>There is one temporary file in the top level source directory:</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600465<ul>
466<li><p><b>.config</b> - Configuration file generated by kconfig, indicating
467which commands (and options to commands) are currently enabled. Used
Rob Landley6882ee82008-02-12 18:41:34 -0600468to make generated/config.h and determine which toys/*.c files to build.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600469
Rob Landley6882ee82008-02-12 18:41:34 -0600470<p>You can create a human readable "miniconfig" version of this file using
Rob Landley66a69d92012-01-16 01:44:17 -0600471<a href=http://landley.net/aboriginal/new_platform.html#miniconfig>these
Rob Landley6882ee82008-02-12 18:41:34 -0600472instructions</a>.</p>
473</li>
474</ul>
475
Rob Landleye7c9a6d2012-02-28 06:34:09 -0600476<a name="generated" />
Rob Landley6882ee82008-02-12 18:41:34 -0600477<p>The "generated/" directory contains files generated from other source code
478in toybox. All of these files can be recreated by the build system, although
479some (such as generated/help.h) are shipped in release versions to reduce
480environmental dependencies (I.E. so you don't need python on your build
481system).</p>
482
483<ul>
484<li><p><b>generated/config.h</b> - list of CFG_SYMBOL and USE_SYMBOL() macros,
Rob Landley4e68de12007-12-13 07:00:27 -0600485generated from .config by a sed invocation in the top level Makefile.</p>
486
487<p>CFG_SYMBOL is a comple time constant set to 1 for enabled symbols and 0 for
Rob Landley6882ee82008-02-12 18:41:34 -0600488disabled symbols. This allows the use of normal if() statements to remove
489code at compile time via the optimizer's dead code elimination (which removes
490from the binary any code that cannot be reached). This saves space without
Rob Landley4e68de12007-12-13 07:00:27 -0600491cluttering the code with #ifdefs or leading to configuration dependent build
492breaks. (See the 1992 Usenix paper
Rob Landleyb6063de2012-01-29 13:54:13 -0600493<a href=http://doc.cat-v.org/henry_spencer/ifdef_considered_harmful.pdf>#ifdef
Rob Landley4e68de12007-12-13 07:00:27 -0600494Considered Harmful</a> for more information.)</p>
495
496<p>USE_SYMBOL(code) evaluates to the code in parentheses when the symbol
497is enabled, and nothing when the symbol is disabled. This can be used
498for things like varargs or variable declarations which can't always be
Rob Landley6882ee82008-02-12 18:41:34 -0600499eliminated by a simple test on CFG_SYMBOL. Note that
Rob Landley4e68de12007-12-13 07:00:27 -0600500(unlike CFG_SYMBOL) this is really just a variant of #ifdef, and can
501still result in configuration dependent build breaks. Use with caution.</p>
502</li>
503</ul>
504
Rob Landley81b899d2007-12-18 02:02:47 -0600505<p><h2>Directory toys/</h2></p>
Rob Landley4e68de12007-12-13 07:00:27 -0600506
507<h3>toys/Config.in</h3>
508
509<p>Included from the top level Config.in, contains one or more
510configuration entries for each command.</p>
511
Rob Landley81b899d2007-12-18 02:02:47 -0600512<p>Each command has a configuration entry matching the command name (although
513configuration symbols are uppercase and command names are lower case).
Rob Landley4e68de12007-12-13 07:00:27 -0600514Options to commands start with the command name followed by an underscore and
Rob Landley66a69d92012-01-16 01:44:17 -0600515the option name. Global options are attached to the "toybox" command,
Rob Landley4e68de12007-12-13 07:00:27 -0600516and thus use the prefix "TOYBOX_". This organization is used by
Rob Landley81b899d2007-12-18 02:02:47 -0600517scripts/cfg2files to select which toys/*.c files to compile for a given
518.config.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600519
Rob Landley66a69d92012-01-16 01:44:17 -0600520<p>A command with multiple names (or multiple similar commands implemented in
Rob Landley4e68de12007-12-13 07:00:27 -0600521the same .c file) should have config symbols prefixed with the name of their
522C file. I.E. config symbol prefixes are NEWTOY() names. If OLDTOY() names
523have config symbols they're options (symbols with an underscore and suffix)
524to the NEWTOY() name. (See toys/toylist.h)</p>
525
526<h3>toys/toylist.h</h3>
Rob Landley81b899d2007-12-18 02:02:47 -0600527<p>The first half of this file prototypes all the structures to hold
Rob Landleyda09b7f2007-12-20 06:29:59 -0600528global variables for each command, and puts them in toy_union. These
529prototypes are only included if the macro NEWTOY isn't defined (in which
530case NEWTOY is defined to a default value that produces function
531prototypes).</p>
Rob Landley81b899d2007-12-18 02:02:47 -0600532
Rob Landleyda09b7f2007-12-20 06:29:59 -0600533<p>The second half of this file lists all the commands in alphabetical
534order, along with their command line arguments and install location.
535Each command has an appropriate configuration guard so only the commands that
536are enabled wind up in the list.</p>
537
538<p>The first time this header is #included, it defines structures and
539produces function prototypes for the commands in the toys directory.</p>
540
541
542<p>The first time it's included, it defines structures and produces function
543prototypes.
544 This
Rob Landley81b899d2007-12-18 02:02:47 -0600545is used to initialize toy_list in main.c, and later in that file to initialize
546NEED_OPTIONS (to figure out whether the command like parsing logic is needed),
547and to put the help entries in the right order in toys/help.c.</p>
Rob Landley4e68de12007-12-13 07:00:27 -0600548
549<h3>toys/help.h</h3>
550
551<p>#defines two help text strings for each command: a single line
552command_help and an additinal command_help_long. This is used by help_main()
553in toys/help.c to display help for commands.</p>
554
555<p>Although this file is generated from Config.in help entries by
556scripts/config2help.py, it's shipped in release tarballs so you don't need
557python on the build system. (If you check code out of source control, or
558modify Config.in, then you'll need python installed to rebuild it.)</p>
559
560<p>This file contains help for all commands, regardless of current
561configuration, but only the currently enabled ones are entered into help_data[]
562in toys/help.c.</p>
563
Rob Landley137bf342012-03-09 08:33:57 -0600564<a name="lib">
Rob Landley81b899d2007-12-18 02:02:47 -0600565<h2>Directory lib/</h2>
Rob Landley4e68de12007-12-13 07:00:27 -0600566
Rob Landley137bf342012-03-09 08:33:57 -0600567<p>TODO: document lots more here.</p>
568
Rob Landleyc8d0da52012-07-15 17:47:08 -0500569<p>lib: getmountlist(), error_msg/error_exit, xmalloc(),
Rob Landley7c04f012008-01-20 19:00:16 -0600570strlcpy(), xexec(), xopen()/xread(), xgetcwd(), xabspath(), find_in_path(),
571itoa().</p>
572
Rob Landley137bf342012-03-09 08:33:57 -0600573<h3>lib/portability.h</h3>
574
575<p>This file is automatically included from the top of toys.h, and smooths
576over differences between platforms (hardware targets, compilers, C libraries,
577operating systems, etc).</p>
578
579<p>This file provides SWAP macros (SWAP_BE16(x) and SWAP_LE32(x) and so on).</p>
580
581<p>A macro like SWAP_LE32(x) means "The value in x is stored as a little
582endian 32 bit value, so perform the translation to/from whatever the native
58332-bit format is". You do the swap once on the way in, and once on the way
584out. If your target is already little endian, the macro is a NOP.</p>
585
586<p>The SWAP macros come in BE and LE each with 16, 32, and 64 bit versions.
587In each case, the name of the macro refers to the _external_ representation,
588and converts to/from whatever your native representation happens to be (which
589can vary depending on what you're currently compiling for).</p>
590
Rob Landleyc8d0da52012-07-15 17:47:08 -0500591<a name="lib_llist"><h3>lib/llist.c</h3>
592
593<p>Some generic single and doubly linked list functions, which take
594advantage of a couple properties of C:</p>
595
596<ul>
597<li><p>Structure elements are laid out in memory in the order listed, and
598the first element has no padding. This means you can always treat (typecast)
599a pointer to a structure as a pointer to the first element of the structure,
600even if you don't know anything about the data following it.</p></li>
601
602<li><p>An array of length zero at the end of a structure adds no space
603to the sizeof() the structure, but if you calculate how much extra space
604you want when you malloc() the structure it will be available at the end.
605Since C has no bounds checking, this means each struct can have one variable
606length array.</p></li>
607</ul>
608
609<p>Toybox's list structures always have their <b>next</b> pointer as
610the first entry of each struct, and singly linked lists end with a NULL pointer.
611This allows generic code to traverse such lists without knowing anything
612else about the specific structs composing them: if your pointer isn't NULL
613typecast it to void ** and dereference once to get the next entry.</p>
614
615<p><b>lib/lib.h</b> defines three structure types:</p>
616<ul>
617<li><p><b>struct string_list</b> - stores a single string (<b>char str[0]</b>),
618memory for which is allocated as part of the node. (I.E. llist_traverse(list,
619free); can clean up after this type of list.)</p></li>
620
621<li><p><b>struct arg_list</b> - stores a pointer to a single string
622(<b>char *arg</b>) which is stored in a separate chunk of memory.</p></li>
623
624<li><p><b>struct double_list</b> - has a second pointer (<b>struct double_list
625*prev</b> along with a <b>char *data</b> for payload.</p></li>
626</ul>
627
628<b>List Functions</b>
629
630<ul>
631<li><p>void *<b>llist_pop</b>(void **list) - advances through a list ala
632<b>node = llist_pop(&list);</b> This doesn't modify the list contents,
633but does advance the pointer you feed it (which is why you pass the _address_
634of that pointer, not the pointer itself).</p></li>
635
636<li><p>void <b>llist_traverse</b>(void *list, void (*using)(void *data)) -
637iterate through a list calling a function on each node.</p></li>
638
639<li><p>struct double_list *<b>dlist_add</b>(struct double_list **llist, char *data)
640- append an entry to a circular linked list.
641This function allocates a new struct double_list wrapper and returns the
642pointer to the new entry (which you can usually ignore since it's llist->prev,
643but if llist was NULL you need it). The argument is the ->data field for the
644new node.</p></li>
645<ul><li><p>void <b>dlist_add_nomalloc</b>(struct double_list **llist,
646struct double_list *new) - append existing struct double_list to
647list, does not allocate anything.</p></li></ul>
648</ul>
649
650<b>Trivia questions:</b>
651
652<ul>
653<li><p><b>Why do arg_list and double_list contain a char * payload instead of
654a void *?</b> - Because you always have to typecast a void * to use it, and
655typecasting a char * does no harm. Thus having it default to the most common
656pointer type saves a few typecasts (strings are the most common payload),
657and doesn't hurt anything otherwise.</p>
658</li>
659
660<li><p><b>Why do the names ->str, ->arg, and ->data differ?</b> - To force
661you to keep track of which one you're using, calling free(node->str) would
662be bad, and _failing_ to free(node->arg) leaks memory.</p></li>
663
664<li><p><b>Why does llist_pop() take a void * instead of void **?</b> -
665because the stupid compiler complains about "type punned pointers" when
666you typecast and dereference ont he same line,
667due to insane FSF developers hardwiring limitations of their optimizer
668into gcc's warning system. Since C automatically typecasts any other
669pointer _down_ to a void *, the current code works fine. It's sad that it
670won't warn you if you forget the &, but the code crashes pretty quickly in
671that case.</p></li>
672
673<li><p><b>How do I assemble a singly-linked-list in order?</b> - use
674a double_list, dlist_add() your entries, and then break the circle with
675<b>list->prev->next = NULL;</b> when done.</li>
676</ul>
677
Rob Landley66a69d92012-01-16 01:44:17 -0600678<a name="lib_args"><h3>lib/args.c</h3>
Rob Landley7c04f012008-01-20 19:00:16 -0600679
Rob Landley66a69d92012-01-16 01:44:17 -0600680<p>Toybox's main.c automatically parses command line options before calling the
681command's main function. Option parsing starts in get_optflags(), which stores
682results in the global structures "toys" (optflags and optargs) and "this".</p>
683
684<p>The option parsing infrastructure stores a bitfield in toys.optflags to
685indicate which options the current command line contained. Arguments
686attached to those options are saved into the command's global structure
687("this"). Any remaining command line arguments are collected together into
688the null-terminated array toys.optargs, with the length in toys.optc. (Note
689that toys.optargs does not contain the current command name at position zero,
690use "toys.which->name" for that.) The raw command line arguments get_optflags()
691parsed are retained unmodified in toys.argv[].</p>
692
693<p>Toybox's option parsing logic is controlled by an "optflags" string, using
694a format reminiscent of getopt's optargs but has several important differences.
695Toybox does not use the getopt()
696function out of the C library, get_optflags() is an independent implementation
697which doesn't permute the original arguments (and thus doesn't change how the
698command is displayed in ps and top), and has many features not present in
699libc optargs() (such as the ability to describe long options in the same string
700as normal options).</p>
701
702<p>Each command's NEWTOY() macro has an optflags string as its middle argument,
703which sets toy_list.options for that command to tell get_optflags() what
704command line arguments to look for, and what to do with them.
705If a command has no option
706definition string (I.E. the argument is NULL), option parsing is skipped
707for that command, which must look at the raw data in toys.argv to parse its
708own arguments. (If no currently enabled command uses option parsing,
709get_optflags() is optimized out of the resulting binary by the compiler's
710--gc-sections option.)</p>
711
712<p>You don't have to free the option strings, which point into the environment
713space (I.E. the string data is not copied). A TOYFLAG_NOFORK command
714that uses the linked list type "*" should free the list objects but not
715the data they point to, via "llist_free(TT.mylist, NULL);". (If it's not
716NOFORK, exit() will free all the malloced data anyway unless you want
717to implement a CONFIG_TOYBOX_FREE cleanup for it.)</p>
718
719<h4>Optflags format string</h4>
720
721<p>Note: the optflags option description string format is much more
722concisely described by a large comment at the top of lib/args.c.</p>
723
724<p>The general theory is that letters set optflags, and punctuation describes
725other actions the option parsing logic should take.</p>
726
727<p>For example, suppose the command line <b>command -b fruit -d walrus -a 42</b>
728is parsed using the optflags string "<b>a#b:c:d</b>". (I.E.
729toys.which->options="a#b:c:d" and argv = ["command", "-b", "fruit", "-d",
730"walrus", "-a", "42"]). When get_optflags() returns, the following data is
731available to command_main():
732
733<ul>
734<li><p>In <b>struct toys</b>:
735<ul>
736<li>toys.optflags = 13; // -a = 8 | -b = 4 | -d = 1</li>
737<li>toys.optargs[0] = "walrus"; // leftover argument</li>
738<li>toys.optargs[1] = NULL; // end of list</li>
Rob Landleyb911d4d2013-09-21 14:27:26 -0500739<li>toys.optc = 1; // there was 1 leftover argument</li>
Rob Landley66a69d92012-01-16 01:44:17 -0600740<li>toys.argv[] = {"-b", "fruit", "-d", "walrus", "-a", "42"}; // The original command line arguments
741</ul>
742<p></li>
743
744<li><p>In <b>union this</b> (treated as <b>long this[]</b>):
745<ul>
746<li>this[0] = NULL; // -c didn't get an argument this time, so get_optflags() didn't change it and toys_init() zeroed "this" during setup.)</li>
747<li>this[1] = (long)"fruit"; // argument to -b</li>
748<li>this[2] = 42; // argument to -a</li>
749</ul>
750</p></li>
751</ul>
752
753<p>If the command's globals are:</p>
754
755<blockquote><pre>
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500756GLOBALS(
Rob Landley66a69d92012-01-16 01:44:17 -0600757 char *c;
758 char *b;
759 long a;
760)
Rob Landley66a69d92012-01-16 01:44:17 -0600761</pre></blockquote>
762<p>That would mean TT.c == NULL, TT.b == "fruit", and TT.a == 42. (Remember,
763each entry that receives an argument must be a long or pointer, to line up
764with the array position. Right to left in the optflags string corresponds to
Rob Landleyc0e56ed2012-10-08 00:02:30 -0500765top to bottom in GLOBALS().</p>
Rob Landley66a69d92012-01-16 01:44:17 -0600766
Rob Landleyb911d4d2013-09-21 14:27:26 -0500767<p>Put globals not filled out by the option parsing logic at the end of the
768GLOBALS block. Common practice is to list the options one per line (to
769make the ordering explicit, first to last in globals corresponds to right
770to left in the option string), then leave a blank line before any non-option
771globals.</p>
772
Rob Landley66a69d92012-01-16 01:44:17 -0600773<p><b>long toys.optflags</b></p>
774
775<p>Each option in the optflags string corresponds to a bit position in
776toys.optflags, with the same value as a corresponding binary digit. The
777rightmost argument is (1<<0), the next to last is (1<<1) and so on. If
Rob Landleyb4a0efa2012-02-06 21:15:19 -0600778the option isn't encountered while parsing argv[], its bit remains 0.</p>
779
780<p>For example,
Rob Landley66a69d92012-01-16 01:44:17 -0600781the optflags string "abcd" would parse the command line argument "-c" to set
782optflags to 2, "-a" would set optflags to 8, "-bd" would set optflags to
7836 (I.E. 4|2), and "-a -c" would set optflags to 10 (2|8).</p>
784
785<p>Only letters are relevant to optflags, punctuation is skipped: in the
786string "a*b:c#d", d=1, c=2, b=4, a=8. The punctuation after a letter
787usually indicate that the option takes an argument.</p>
788
Rob Landleyb4a0efa2012-02-06 21:15:19 -0600789<p>Since toys.optflags is an unsigned int, it only stores 32 bits. (Which is
790the amount a long would have on 32-bit platforms anyway; 64 bit code on
79132 bit platforms is too expensive to require in common code used by almost
792all commands.) Bit positions beyond the 1<<31 aren't recorded, but
793parsing higher options can still set global variables.</p>
794
Rob Landley66a69d92012-01-16 01:44:17 -0600795<p><b>Automatically setting global variables from arguments (union this)</b></p>
796
797<p>The following punctuation characters may be appended to an optflags
798argument letter, indicating the option takes an additional argument:</p>
799
800<ul>
801<li><b>:</b> - plus a string argument, keep most recent if more than one.</li>
802<li><b>*</b> - plus a string argument, appended to a linked list.</li>
Rob Landley66a69d92012-01-16 01:44:17 -0600803<li><b>@</b> - plus an occurrence counter (stored in a long)</li>
Rob Landleyb6063de2012-01-29 13:54:13 -0600804<li><b>#</b> - plus a signed long argument.
Rob Landleyb911d4d2013-09-21 14:27:26 -0500805<li><b>-</b> - plus a signed long argument defaulting to negative (start argument with + to force a positive value).</li>
Rob Landleyb6063de2012-01-29 13:54:13 -0600806<li><b>.</b> - plus a floating point argument (if CFG_TOYBOX_FLOAT).</li>
807<ul>The following can be appended to a float or double:
808<li><b>&lt;123</b> - error if argument is less than this</li>
809<li><b>&gt;123</b> - error if argument is greater than this</li>
810<li><b>=123</b> - default value if argument not supplied</li>
811</ul>
Rob Landley66a69d92012-01-16 01:44:17 -0600812</ul>
813
Rob Landleyb911d4d2013-09-21 14:27:26 -0500814<p>A note about "." and CFG_TOYBOX_FLOAT: option parsing only understands <>=
815after . when CFG_TOYBOX_FLOAT
816is enabled. (Otherwise the code to determine where floating point constants
817end drops out; it requires floating point). When disabled, it can reserve a
818global data slot for the argument (so offsets won't change in your
819GLOBALS[] block), but will never fill it out. You can handle
820this by using the USE_BLAH() macros with C string concatenation, ala:
821"abc." USE_TOYBOX_FLOAT("<1.23>4.56=7.89") "def"</p>
822
823<p><b>GLOBALS</b></p>
Rob Landley66a69d92012-01-16 01:44:17 -0600824
825<p>Options which have an argument fill in the corresponding slot in the global
826union "this" (see generated/globals.h), treating it as an array of longs
Rob Landleyb911d4d2013-09-21 14:27:26 -0500827with the rightmost saved in this[0]. As described above, using "a*b:c#d",
828"-c 42" would set this[0] = 42; and "-b 42" would set this[1] = "42"; each
829slot is left NULL if the corresponding argument is not encountered.</p>
Rob Landley66a69d92012-01-16 01:44:17 -0600830
831<p>This behavior is useful because the LP64 standard ensures long and pointer
Rob Landleyb4a0efa2012-02-06 21:15:19 -0600832are the same size. C99 guarantees structure members will occur in memory
833in the same order they're declared, and that padding won't be inserted between
Rob Landley66a69d92012-01-16 01:44:17 -0600834consecutive variables of register size. Thus the first few entries can
835be longs or pointers corresponding to the saved arguments.</p>
836
Rob Landleyb911d4d2013-09-21 14:27:26 -0500837<p>See toys/other/hello.c for a longer example of parsing options into the
838GLOBALS block.</p>
839
Rob Landley66a69d92012-01-16 01:44:17 -0600840<p><b>char *toys.optargs[]</b></p>
841
842<p>Command line arguments in argv[] which are not consumed by option parsing
843(I.E. not recognized either as -flags or arguments to -flags) will be copied
844to toys.optargs[], with the length of that array in toys.optc.
845(When toys.optc is 0, no unrecognized command line arguments remain.)
846The order of entries is preserved, and as with argv[] this new array is also
847terminated by a NULL entry.</p>
848
849<p>Option parsing can require a minimum or maximum number of optargs left
850over, by adding "<1" (read "at least one") or ">9" ("at most nine") to the
851start of the optflags string.</p>
852
853<p>The special argument "--" terminates option parsing, storing all remaining
854arguments in optargs. The "--" itself is consumed.</p>
855
856<p><b>Other optflags control characters</b></p>
857
858<p>The following characters may occur at the start of each command's
859optflags string, before any options that would set a bit in toys.optflags:</p>
860
861<ul>
862<li><b>^</b> - stop at first nonoption argument (for nice, xargs...)</li>
863<li><b>?</b> - allow unknown arguments (pass non-option arguments starting
864with - through to optargs instead of erroring out).</li>
865<li><b>&amp;</b> - the first argument has imaginary dash (ala tar/ps. If given twice, all arguments have imaginary dash.)</li>
866<li><b>&lt;</b> - must be followed by a decimal digit indicating at least this many leftover arguments are needed in optargs (default 0)</li>
867<li><b>&gt;</b> - must be followed by a decimal digit indicating at most this many leftover arguments allowed (default MAX_INT)</li>
868</ul>
869
870<p>The following characters may be appended to an option character, but do
871not by themselves indicate an extra argument should be saved in this[].
872(Technically any character not recognized as a control character sets an
873optflag, but letters are never control characters.)</p>
874
875<ul>
876<li><b>^</b> - stop parsing options after encountering this option, everything else goes into optargs.</li>
877<li><b>|</b> - this option is required. If more than one marked, only one is required.</li>
Rob Landley66a69d92012-01-16 01:44:17 -0600878</ul>
879
Rob Landleyb6063de2012-01-29 13:54:13 -0600880<p>The following may be appended to a float or double:</p>
881
882<ul>
883<li><b>&lt;123</b> - error if argument is less than this</li>
884<li><b>&gt;123</b> - error if argument is greater than this</li>
885<li><b>=123</b> - default value if argument not supplied</li>
886</ul>
887
888<p>Option parsing only understands <>= after . when CFG_TOYBOX_FLOAT
889is enabled. (Otherwise the code to determine where floating point constants
890end drops out. When disabled, it can reserve a global data slot for the
891argument so offsets won't change, but will never fill it out.). You can handle
892this by using the USE_BLAH() macros with C string concatenation, ala:</p>
893
894<blockquote>"abc." USE_TOYBOX_FLOAT("<1.23>4.56=7.89") "def"</blockquote>
895
Rob Landley66a69d92012-01-16 01:44:17 -0600896<p><b>--longopts</b></p>
897
898<p>The optflags string can contain long options, which are enclosed in
899parentheses. They may be appended to an existing option character, in
900which case the --longopt is a synonym for that option, ala "a:(--fred)"
901which understands "-a blah" or "--fred blah" as synonyms.</p>
902
903<p>Longopts may also appear before any other options in the optflags string,
904in which case they have no corresponding short argument, but instead set
905their own bit based on position. So for "(walrus)#(blah)xy:z" "command
906--walrus 42" would set toys.optflags = 16 (-z = 1, -y = 2, -x = 4, --blah = 8)
907and would assign this[1] = 42;</p>
908
909<p>A short option may have multiple longopt synonyms, "a(one)(two)", but
910each "bare longopt" (ala "(one)(two)abc" before any option characters)
911always sets its own bit (although you can group them with +X).</p>
Rob Landley7c04f012008-01-20 19:00:16 -0600912
Rob Landleyb911d4d2013-09-21 14:27:26 -0500913<p><b>[groups]</b></p>
914
915<p>At the end of the option string, square bracket groups can define
916relationships between existing options. (This only applies to short
917options, bare --longopts can't participate.)</p>
918
919<p>The first character of the group defines the type, the remaining
920characters are options it applies to:</p>
921
922<ul>
923<li><b>-</b> - Exclusive, switch off all others in this group.</li>
924<li><b>+</b> - Inclusive, switch on all others in this group.</li>
925<li><b>!</b> - Error, fail if more than one defined.</li>
926</ul>
927
928<p>So "abc[-abc]" means -ab = -b, -ba = -a, -abc = -c. "abc[+abc]"
929means -ab=-abc, -c=-abc, and "abc[!abc] means -ab calls error_exit("no -b
930with -a"). Note that [-] groups clear the GLOBALS option slot of
931options they're switching back off, but [+] won't set options it didn't see
932(just the optflags).</p>
933
934<p><b>whitespace</b></p>
935
936<p>Arguments may occur with or without a space (I.E. "-a 42" or "-a42").
937The command line argument "-abc" may be interepreted many different ways:
938the optflags string "cba" sets toys.optflags = 7, "c:ba" sets toys.optflags=4
939and saves "ba" as the argument to -c, and "cb:a" sets optflags to 6 and saves
940"c" as the argument to -b.</p>
941
942<p>Note that &amp; changes whitespace handling, so that the command line
943"tar cvfCj outfile.tar.bz2 topdir filename" is parsed the same as
944"tar filename -c -v -j -f outfile.tar.bz2 -C topdir". Note that "tar -cvfCj
945one two three" would equal "tar -c -v -f Cj one two three". (This matches
946historical usage.)</p>
947
948<p>Appending a space to the option in the option string ("a: b") makes it
949require a space, I.E. "-ab" is interpreted as "-a" "-b". That way "kill -stop"
950differs from "kill -s top".</p>
951
952<p>Appending ; to a longopt in the option string makes its argument optional,
953and only settable with =, so in ls "(color):;" can accept "ls --color" and
954"ls --color=auto" without complaining that the first has no argument.</p>
955
Rob Landleyc8d0da52012-07-15 17:47:08 -0500956<a name="lib_dirtree"><h3>lib/dirtree.c</h3>
957
958<p>The directory tree traversal code should be sufficiently generic
959that commands never need to use readdir(), scandir(), or the fts.h family
960of functions.</p>
961
962<p>These functions do not call chdir() or rely on PATH_MAX. Instead they
963use openat() and friends, using one filehandle per directory level to
964recurseinto subdirectories. (I.E. they can descend 1000 directories deep
965if setrlimit(RLIMIT_NOFILE) allows enough open filehandles, and the default
966in /proc/self/limits is generally 1024.)</p>
967
968<p>The basic dirtree functions are:</p>
969
970<ul>
971<li><p><b>dirtree_read(char *path, int (*callback)(struct dirtree node))</b> -
972recursively read directories, either applying callback() or returning
973a tree of struct dirtree if callback is NULL.</p></li>
974
975<li><p><b>dirtree_path(struct dirtree *node, int *plen)</b> - malloc() a
976string containing the path from the root of this tree to this node. If
977plen isn't NULL then *plen is how many extra bytes to malloc at the end
978of string.</p></li>
979
980<li><p><b>dirtree_parentfd(struct dirtree *node)</b> - return fd of
981containing directory, for use with openat() and such.</p></li>
982</ul>
983
984<p>The <b>dirtree_read()</b> function takes two arguments, a starting path for
985the root of the tree, and a callback function. The callback takes a
986<b>struct dirtree *</b> (from lib/lib.h) as its argument. If the callback is
987NULL, the traversal uses a default callback (dirtree_notdotdot()) which
988recursively assembles a tree of struct dirtree nodes for all files under
989this directory and subdirectories (filtering out "." and ".." entries),
990after which dirtree_read() returns the pointer to the root node of this
991snapshot tree.</p>
992
993<p>Otherwise the callback() is called on each entry in the directory,
994with struct dirtree * as its argument. This includes the initial
995node created by dirtree_read() at the top of the tree.</p>
996
997<p><b>struct dirtree</b></p>
998
999<p>Each struct dirtree node contains <b>char name[]</b> and <b>struct stat
1000st</b> entries describing a file, plus a <b>char *symlink</b>
1001which is NULL for non-symlinks.</p>
1002
1003<p>During a callback function, the <b>int data</b> field of directory nodes
1004contains a dirfd (for use with the openat() family of functions). This is
1005generally used by calling dirtree_parentfd() on the callback's node argument.
1006For symlinks, data contains the length of the symlink string. On the second
1007callback from DIRTREE_COMEAGAIN (depth-first traversal) data = -1 for
1008all nodes (that's how you can tell it's the second callback).</p>
1009
1010<p>Users of this code may put anything they like into the <b>long extra</b>
1011field. For example, "cp" and "mv" use this to store a dirfd for the destination
1012directory (and use DIRTREE_COMEAGAIN to get the second callback so they can
1013close(node->extra) to avoid running out of filehandles).
1014This field is not directly used by the dirtree code, and
1015thanks to LP64 it's large enough to store a typecast pointer to an
1016arbitrary struct.</p>
1017
1018<p>The return value of the callback combines flags (with boolean or) to tell
1019the traversal infrastructure how to behave:</p>
1020
1021<ul>
1022<li><p><b>DIRTREE_SAVE</b> - Save this node, assembling a tree. (Without
1023this the struct dirtree is freed after the callback returns. Filtering out
1024siblings is fine, but discarding a parent while keeping its child leaks
1025memory.)</p></li>
1026<li><p><b>DIRTREE_ABORT</b> - Do not examine any more entries in this
1027directory. (Does not propagate up tree: to abort entire traversal,
1028return DIRTREE_ABORT from parent callbacks too.)</p></li>
1029<li><p><b>DIRTREE_RECURSE</b> - Examine directory contents. Ignored for
1030non-directory entries. The remaining flags only take effect when
1031recursing into the children of a directory.</p></li>
1032<li><p><b>DIRTREE_COMEAGAIN</b> - Call the callback a second time after
1033examining all directory contents, allowing depth-first traversal.
1034On the second call, dirtree->data = -1.</p></li>
1035<li><p><b>DIRTREE_SYMFOLLOW</b> - follow symlinks when populating children's
1036<b>struct stat st</b> (by feeding a nonzero value to the symfollow argument of
1037dirtree_add_node()), which means DIRTREE_RECURSE treats symlinks to
1038directories as directories. (Avoiding infinite recursion is the callback's
1039problem: the non-NULL dirtree->symlink can still distinguish between
1040them.)</p></li>
1041</ul>
1042
1043<p>Each struct dirtree contains three pointers (next, parent, and child)
1044to other struct dirtree.</p>
1045
1046<p>The <b>parent</b> pointer indicates the directory
1047containing this entry; even when not assembling a persistent tree of
1048nodes the parent entries remain live up to the root of the tree while
1049child nodes are active. At the top of the tree the parent pointer is
1050NULL, meaning the node's name[] is either an absolute path or relative
1051to cwd. The function dirtree_parentfd() gets the directory file descriptor
1052for use with openat() and friends, returning AT_FDCWD at the top of tree.</p>
1053
1054<p>The <b>child</b> pointer points to the first node of the list of contents of
1055this directory. If the directory contains no files, or the entry isn't
1056a directory, child is NULL.</p>
1057
1058<p>The <b>next</b> pointer indicates sibling nodes in the same directory as this
1059node, and since it's the first entry in the struct the llist.c traversal
1060mechanisms work to iterate over sibling nodes. Each dirtree node is a
1061single malloc() (even char *symlink points to memory at the end of the node),
1062so llist_free() works but its callback must descend into child nodes (freeing
1063a tree, not just a linked list), plus whatever the user stored in extra.</p>
1064
1065<p>The <b>dirtree_read</b>() function is a simple wrapper, calling <b>dirtree_add_node</b>()
1066to create a root node relative to the current directory, then calling
1067<b>handle_callback</b>() on that node (which recurses as instructed by the callback
1068return flags). Some commands (such as chgrp) bypass this wrapper, for example
1069to control whether or not to follow symlinks to the root node; symlinks
1070listed on the command line are often treated differently than symlinks
1071encountered during recursive directory traversal).
1072
1073<p>The ls command not only bypasses the wrapper, but never returns
1074<b>DIRTREE_RECURSE</b> from the callback, instead calling <b>dirtree_recurse</b>() manually
1075from elsewhere in the program. This gives ls -lR manual control
1076of traversal order, which is neither depth first nor breadth first but
1077instead a sort of FIFO order requried by the ls standard.</p>
1078
Rob Landleyc0e56ed2012-10-08 00:02:30 -05001079<a name="#toys">
1080<h2>Directory toys/</h2>
1081
1082<p>This directory contains command implementations. Each command is a single
1083self-contained file. Adding a new command involves adding a single
1084file, and removing a command involves removing that file. Commands use
1085shared infrastructure from the lib/ and generated/ directories.</p>
1086
1087<p>Currently there are three subdirectories under "toys/" containing commands
1088described in POSIX-2008, the Linux Standard Base 4.1, or "other". The only
1089difference this makes is which menu the command shows up in during "make
1090menuconfig", the directories are otherwise identical. Note that they commands
1091exist within a single namespace at runtime, so you can't have the same
1092command in multiple subdirectories.</p>
1093
1094<p>(There are actually four sub-menus in "make menuconfig", the fourth
1095contains global configuration options for toybox, and lives in Config.in at
1096the top level.)</p>
1097
1098<p>See <a href="#adding">adding a new command</a> for details on the
1099layout of a command file.</p>
1100
Rob Landley81b899d2007-12-18 02:02:47 -06001101<h2>Directory scripts/</h2>
Rob Landley4e68de12007-12-13 07:00:27 -06001102
Rob Landley1f4f41a2012-10-08 21:31:07 -05001103<p>Build infrastructure. The makefile calls scripts/make.sh for "make"
1104and scripts/install.sh for "make install".</p>
1105
1106<p>There's also a test suite, "make test" calls make/test.sh, which runs all
1107the tests in make/test/*. You can run individual tests via
1108"scripts/test.sh command", or "TEST_HOST=1 scripts/test.sh command" to run
1109that test against the host implementation instead of the toybox one.</p>
1110
Rob Landley4e68de12007-12-13 07:00:27 -06001111<h3>scripts/cfg2files.sh</h3>
1112
1113<p>Run .config through this filter to get a list of enabled commands, which
1114is turned into a list of files in toys via a sed invocation in the top level
1115Makefile.
1116</p>
1117
Rob Landley81b899d2007-12-18 02:02:47 -06001118<h2>Directory kconfig/</h2>
Rob Landley4e68de12007-12-13 07:00:27 -06001119
1120<p>Menuconfig infrastructure copied from the Linux kernel. See the
1121Linux kernel's Documentation/kbuild/kconfig-language.txt</p>
1122
Rob Landley66a69d92012-01-16 01:44:17 -06001123<a name="generated">
1124<h2>Directory generated/</h2>
1125
1126<p>All the files in this directory except the README are generated by the
1127build. (See scripts/make.sh)</p>
1128
1129<ul>
1130<li><p><b>config.h</b> - CFG_COMMAND and USE_COMMAND() macros set by menuconfig via .config.</p></li>
1131
1132<li><p><b>Config.in</b> - Kconfig entries for each command. Included by top level Config.in. The help text in here is used to generated help.h</p></li>
1133
1134<li><p><b>help.h</b> - Help text strings for use by "help" command. Building
1135this file requires python on the host system, so the prebuilt file is shipped
1136in the build tarball to avoid requiring python to build toybox.</p></li>
1137
1138<li><p><b>newtoys.h</b> - List of NEWTOY() or OLDTOY() macros for all available
1139commands. Associates command_main() functions with command names, provides
1140option string for command line parsing (<a href="#lib_args">see lib/args.c</a>),
1141specifies where to install each command and whether toysh should fork before
1142calling it.</p></li>
1143</ul>
1144
1145<p>Everything in this directory is a derivative file produced from something
1146else. The entire directory is deleted by "make distclean".</p>
Rob Landley4e68de12007-12-13 07:00:27 -06001147<!--#include file="footer.html" -->