Blame - www/code.html - fp2-dev/platform/external/toybox

blob: 7603c841304e4db343503abb899d55d6309d3a53 [file] [log] [blame]

Rob Landley	349ff52	2014-01-04 13:09:42 -0600	[diff] [blame]	1	<html><head><title>toybox source code walkthrough</title></head>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	2	<!--#include file="header.html" -->
				3
Rob Landley	2704834	2013-08-18 14:24:59 -0500	[diff] [blame]	4	<p><h1><a name="style" /><a href="#style">Code style</a></h1></p>
Rob Landley	e7c9a6d	2012-02-28 06:34:09 -0600	[diff] [blame]	5
				6	<p>The primary goal of toybox is _simple_ code. Keeping the code small is
Rob Landley	ed6ed62	2012-03-06 20:49:03 -0600	[diff] [blame]	7	second, with speed and lots of features coming in somewhere after that.
				8	(For more on that, see the <a href=design.html>design</a> page.)</p>
Rob Landley	e7c9a6d	2012-02-28 06:34:09 -0600	[diff] [blame]	9
				10	<p>A simple implementation usually takes up fewer lines of source code,
				11	meaning more code can fit on the screen at once, meaning the programmer can
				12	see more of it on the screen and thus keep more if in their head at once.
Rob Landley	ed6ed62	2012-03-06 20:49:03 -0600	[diff] [blame]	13	This helps code auditing and thus reduces bugs. That said, sometimes being
				14	more explicit is preferable to being clever enough to outsmart yourself:
				15	don't be so terse your code is unreadable.</p>
Rob Landley	5a0660f	2007-12-27 21:36:44 -0600	[diff] [blame]	16
Rob Landley	ca73392	2014-05-19 18:24:35 -0500	[diff] [blame]	17	<p>Toybox has an actual coding style guide over on
				18	<a href=design.html#codestyle>the design page</a>, but in general we just
				19	want the code to be consistent.</p>
Rob Landley	5a0660f	2007-12-27 21:36:44 -0600	[diff] [blame]	20
Rob Landley	2704834	2013-08-18 14:24:59 -0500	[diff] [blame]	21	<p><h1><a name="building" /><a href="#building">Building Toybox</a></h1></p>
Rob Landley	e7c9a6d	2012-02-28 06:34:09 -0600	[diff] [blame]	22
				23	<p>Toybox is configured using the Kconfig language pioneered by the Linux
				24	kernel, and adopted by many other projects (uClibc, OpenEmbedded, etc).
				25	This generates a ".config" file containing the selected options, which
Rob Landley	7aa651a	2012-11-13 17:14:08 -0600	[diff] [blame]	26	controls which features are included when compiling toybox.</p>
Rob Landley	e7c9a6d	2012-02-28 06:34:09 -0600	[diff] [blame]	27
				28	<p>Each configuration option has a default value. The defaults indicate the
				29	"maximum sane configuration", I.E. if the feature defaults to "n" then it
				30	either isn't complete or is a special-purpose option (such as debugging
				31	code) that isn't intended for general purpose use.</p>
				32
				33	<p>The standard build invocation is:</p>
				34
				35	<ul>
				36	<li>make defconfig #(or menuconfig)</li>
				37	<li>make</li>
				38	<li>make install</li>
				39	</ul>
				40
				41	<p>Type "make help" to see all available build options.</p>
				42
				43	<p>The file "configure" contains a number of environment variable definitions
				44	which influence the build, such as specifying which compiler to use or where
				45	to install the resulting binaries. This file is included by the build, but
				46	accepts existing definitions of the environment variables, so it may be sourced
				47	or modified by the developer before building and the definitions exported
				48	to the environment will take precedence.</p>
				49
				50	<p>(To clarify: "configure" describes the build and installation environment,
				51	".config" lists the features selected by defconfig/menuconfig.)</p>
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	52
Rob Landley	2704834	2013-08-18 14:24:59 -0500	[diff] [blame]	53	<p><h1><a name="running"><a href="#running">Running a command</a></h1></p>
				54
				55	<h2>main</h2>
				56
				57	<p>The toybox main() function is at the end of main.c at the top level. It has
				58	two possible codepaths, only one of which is configured into any given build
				59	of toybox.</p>
				60
				61	<p>If CONFIG_SINGLE is selected, toybox is configured to contain only a single
				62	command, so most of the normal setup can be skipped. In this case the
				63	multiplexer isn't used, instead main() calls toy_singleinit() (also in main.c)
				64	to set up global state and parse command line arguments, calls the command's
				65	main function out of toy_list (in the CONFIG_SINGLE case the array has a single entry, no need to search), and if the function returns instead of exiting
				66	it flushes stdout (detecting error) and returns toys.exitval.</p>
				67
				68	<p>When CONFIG_SINGLE is not selected, main() uses basename() to find the
				69	name it was run as, shifts its argument list one to the right so it lines up
				70	with where the multiplexer function expects it, and calls toybox_main(). This
				71	leverages the multiplexer command's infrastructure to find and run the
				72	appropriate command. (A command name starting with "toybox" will
				73	recursively call toybox_main(); you can go "./toybox toybox toybox toybox ls"
				74	if you want to...)</p>
				75
				76	<h2>toybox_main</h2>
				77
				78	<p>The toybox_main() function is also in main,c. It handles a possible
				79	--help option ("toybox --help ls"), prints the list of available commands if no
				80	arguments were provided to the multiplexer (or with full path names if any
				81	other option is provided before a command name, ala "toybox --list").
				82	Otherwise it calls toy_exec() on its argument list.</p>
				83
				84	<p>Note that the multiplexer is the first entry in toy_list (the rest of the
				85	list is sorted alphabetically to allow binary search), so toybox_main can
				86	cheat and just grab the first entry to quickly set up its context without
				87	searching. Since all command names go through the multiplexer at least once
				88	in the non-TOYBOX_SINGLE case, this avoids a redundant search of
				89	the list.</p>
				90
				91	<p>The toy_exec() function is also in main.c. It performs toy_find() to
				92	perform a binary search on the toy_list array to look up the command's
				93	entry by name and saves it in the global variable which, calls toy_init()
				94	to parse command line arguments and set up global state (using which->options),
				95	and calls the appropriate command's main() function (which->toy_main). On
				96	return it flushes all pending ansi FILE * I/O, detects if stdout had an
				97	error, and then calls xexit() (which uses toys.exitval).</p>
				98
				99	<p><h1><a name="infrastructure" /><a href="#infrastructure">Infrastructure</a></h1></p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	100
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	101	<p>The toybox source code is in following directories:</p>
				102	<ul>
				103	<li>The <a href="#top">top level directory</a> contains the file main.c (were
				104	execution starts), the header file toys.h (included by every command), and
				105	other global infrastructure.</li>
				106	<li>The <a href="#lib">lib directory</a> contains common functions shared by
Rob Landley	c8d0da5	2012-07-15 17:47:08 -0500	[diff] [blame]	107	multiple commands:</li>
				108	<ul>
				109	<li><a href="#lib_lib">lib/lib.c</a></li>
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	110	<li><a href="#lib_xwrap">lib/xwrap.c</a></li>
Rob Landley	c8d0da5	2012-07-15 17:47:08 -0500	[diff] [blame]	111	<li><a href="#lib_llist">lib/llist.c</a></li>
				112	<li><a href="#lib_args">lib/args.c</a></li>
				113	<li><a href="#lib_dirtree">lib/dirtree.c</a></li>
				114	</ul>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	115	<li>The <a href="#toys">toys directory</a> contains the C files implementating
Rob Landley	002a11e	2014-05-22 08:16:55 -0500	[diff] [blame]	116	each command. Currently it contains five subdirectories categorizing the
				117	commands: posix, lsb, other, example, and pending.</li>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	118	<li>The <a href="#scripts">scripts directory</a> contains the build and
				119	test infrastructure.</li>
				120	<li>The <a href="#kconfig">kconfig directory</a> contains the configuration
				121	infrastructure implementing menuconfig (copied from the Linux kernel).</li>
				122	<li>The <a href="#generated">generated directory</a> contains intermediate
				123	files generated from other parts of the source code.</li>
				124	</ul>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	125
Rob Landley	bbe500e	2012-02-26 21:53:15 -0600	[diff] [blame]	126	<a name="adding" />
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	127	<p><h1><a href="#adding">Adding a new command</a></h1></p>
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	128	<p>To add a new command to toybox, add a C file implementing that command to
				129	one of the subdirectories under the toys directory. No other files need to
				130	be modified; the build extracts all the information it needs (such as command
				131	line arguments) from specially formatted comments and macros in the C file.
				132	(See the description of the <a href="#generated">"generated" directory</a>
				133	for details.)</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	134
Rob Landley	002a11e	2014-05-22 08:16:55 -0500	[diff] [blame]	135	<p>Currently there are five subdirectories under "toys", one for commands
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	136	defined by the POSIX standard, one for commands defined by the Linux Standard
Rob Landley	002a11e	2014-05-22 08:16:55 -0500	[diff] [blame]	137	Base, an "other" directory for commands not covered by an obvious standard,
				138	a directory of example commands (templates to use when starting new commands),
				139	and a "pending" directory of commands that need further review/cleanup
				140	before moving to one of the other directories (run these at your own risk,
				141	cleanup patches welcome).
				142	These directories are just for developer convenience sorting the commands,
				143	the directories are otherwise functionally identical. To add a new category,
				144	create the appropriate directory with a README file in it whose first line
				145	is the description menuconfig should use for the directory.)</p>
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	146
Rob Landley	002a11e	2014-05-22 08:16:55 -0500	[diff] [blame]	147	<p>An easy way to start a new command is copy the file "toys/example/hello.c"
				148	to the name of the new command, and modify this copy to implement the new
				149	command (more or less by turning every instance of "hello" into the
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	150	name of your command, updating the command line arguments, globals, and
Rob Landley	002a11e	2014-05-22 08:16:55 -0500	[diff] [blame]	151	help data, and then filling out its "main" function with code that does
				152	something interesting).</p>
				153
				154	<p>You could also start with "toys/example/skeleton.c", which provides a lot
				155	more example code (showing several variants of command line option
				156	parsing, how to implement multiple commands in the same file, and so on).
				157	But usually it's just more stuff to delete.</p>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	158
				159	<p>Here's a checklist of steps to turn hello.c into another command:</p>
				160
				161	<ul>
Rob Landley	002a11e	2014-05-22 08:16:55 -0500	[diff] [blame]	162	<li><p>First "cp toys/example/hello.c toys/other/yourcommand.c" and open
				163	the new file in your preferred text editor.</p>
				164	<ul><li><p>Note that the
				165	name of the new file is significant: it's the name of the new command you're
				166	adding to toybox. The build includes all .c files under toys// whose
				167	names are a case insensitive match for an enabled config symbol. So
				168	toys/posix/cat.c only gets included if you have "CAT=y" in ".config".</p></li>
				169	</ul></p></li>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	170
				171	<li><p>Change the one line comment at the top of the file (currently
				172	"hello.c - A hello world program") to describe your new file.</p></li>
				173
				174	<li><p>Change the copyright notice to your name, email, and the current
				175	year.</p></li>
				176
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	177	<li><p>Give a URL to the relevant standards document, where applicable.
				178	(Sample links to SUSv4 and LSB are provided, feel free to link to other
				179	documentation or standards as appropriate.)</p></li>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	180
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	181	<li><p>Update the USE_YOURCOMMAND(NEWTOY(yourcommand,"blah",0)) line.
				182	The NEWTOY macro fills out this command's <a href="#toy_list">toy_list</a>
				183	structure. The arguments to the NEWTOY macro are:</p>
				184
				185	<ol>
				186	<li><p>the name used to run your command</p></li>
Rob Landley	002a11e	2014-05-22 08:16:55 -0500	[diff] [blame]	187	<li><p>the command line argument <a href="#lib_args">option parsing string</a> (0 if none)</p></li>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	188	<li><p>a bitfield of TOYFLAG values
				189	(defined in toys.h) providing additional information such as where your
				190	command should be installed on a running system, whether to blank umask
				191	before running, whether or not the command must run as root (and thus should
				192	retain root access if installed SUID), and so on.</p></li>
				193	</ol>
				194	</li>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	195
				196	<li><p>Change the kconfig data (from "config YOURCOMMAND" to the end of the
				197	comment block) to supply your command's configuration and help
				198	information. The uppper case config symbols are used by menuconfig, and are
				199	also what the CFG_ and USE_() macros are generated from (see [TODO]). The
				200	help information here is used by menuconfig, and also by the "help" command to
				201	describe your new command. (See [TODO] for details.) By convention,
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	202	unfinished commands default to "n" and finished commands default to "y",
				203	so "make defconfig" selects all finished commands. (Note, "finished" means
				204	"ready to be used", not that it'll never change again.)<p>
				205
				206	<p>Each help block should start with a "usage: yourcommand" line explaining
				207	any command line arguments added by this config option. The "help" command
				208	outputs this text, and scripts/config2help.c in the build infrastructure
				209	collates these usage lines for commands with multiple configuration
				210	options when producing generated/help.h.</p>
				211	</li>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	212
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	213	<li><p>Change the "#define FOR_hello" line to "#define FOR_yourcommand" right
				214	before the "#include <toys.h>". (This selects the appropriate FLAG_ macros and
				215	does a "#define TT this.yourcommand" so you can access the global variables
				216	out of the space-saving union of structures. If you aren't using any command
				217	flag bits and aren't defining a GLOBAL block, you can delete this line.)</p></li>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	218
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	219	<li><p>Update the GLOBALS() macro to contain your command's global
				220	variables. If your command has no global variables, delete this macro.</p>
				221
				222	<p>Variables in the GLOBALS() block are are stored in a space saving
				223	<a href="#toy_union">union of structures</a> format, which may be accessed
				224	using the TT macro as if TT were a global structure (so TT.membername).
				225	If you specified two-character command line arguments in
				226	NEWTOY(), the first few global variables will be initialized by the automatic
				227	argument parsing logic, and the type and order of these variables must
				228	correspond to the arguments specified in NEWTOY().
				229	(See <a href="#lib_args">lib/args.c</a> for details.)</p></li>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	230
				231	<li><p>Rename hello_main() to yourcommand_main(). This is the main() function
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	232	where execution of your command starts. Your command line options are
				233	already sorted into this.optflags, this.optargs, this.optc, and the GLOBALS()
				234	as appropriate by the time this function is called. (See
Rob Landley	002a11e	2014-05-22 08:16:55 -0500	[diff] [blame]	235	<a href="#lib_args">get_optflags()</a> for details.)</p></li>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	236	</ul>
				237
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	238	<a name="headers" /><h2><a href="#headers">Headers.</a></h2>
Rob Landley	85a3241	2013-12-27 06:53:15 -0600	[diff] [blame]	239
				240	<p>Commands generally don't have their own headers. If it's common code
				241	it can live in lib/, if it isn't put it in the command's .c file. (The line
				242	between implementing multiple commands in a C file via OLDTOY() to share
				243	infrastructure and moving that shared infrastructure to lib/ is a judgement
				244	call. Try to figure out which is simplest.)</p>
				245
				246	<p>The top level toys.h should #include all the standard (posix) headers
				247	that any command uses. (Partly this is friendly to ccache and partly this
				248	makes the command implementations shorter.) Individual commands should only
				249	need to include nonstandard headers that might prevent that command from
				250	building in some context we'd care about (and thus requiring that command to
				251	be disabled to avoid a build break).</p>
				252
				253	<p>Target-specific stuff (differences between compiler versions, libc versions,
				254	or operating systems) should be confined to lib/portability.h and
				255	lib/portability.c. (There's even some minimal compile-time environment probing
				256	that writes data to generated/portability.h, see scripts/genconfig.sh.)</p>
				257
				258	<p>Only include linux/*.h headers from individual commands (not from other
				259	headers), and only if you really need to. Data that varies per architecture
				260	is a good reason to include a header. If you just need a couple constants
				261	that haven't changed since the 1990's, it's ok to #define them yourself or
				262	just use the constant inline with a comment explaining what it is. (A
				263	#define that's only used once isn't really helping.)</p>
				264
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	265	<p><a name="top" /><h1><a href="#top">Top level directory.</a></h1></p>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	266
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	267	<p>This directory contains global infrastructure.</p>
				268
				269	<h3>toys.h</h3>
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	270	<p>Each command #includes "toys.h" as part of its standard prolog. It
				271	may "#define FOR_commandname" before doing so to get some extra entries
				272	specific to this command.</p>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	273
				274	<p>This file sucks in most of the commonly used standard #includes, so
				275	individual files can just #include "toys.h" and not have to worry about
				276	stdargs.h and so on. Individual commands still need to #include
				277	special-purpose headers that may not be present on all systems (and thus would
				278	prevent toybox from building that command on such a system with that command
				279	enabled). Examples include regex support, any "linux/" or "asm/" headers, mtab
				280	support (mntent.h and sys/mount.h), and so on.</p>
				281
				282	<p>The toys.h header also defines structures for most of the global variables
				283	provided to each command by toybox_main(). These are described in
				284	detail in the description for main.c, where they are initialized.</p>
				285
				286	<p>The global variables are grouped into structures (and a union) for space
				287	savings, to more easily track the amount of memory consumed by them,
				288	so that they may be automatically cleared/initialized as needed, and so
				289	that access to global variables is more easily distinguished from access to
				290	local variables.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	291
				292	<h3>main.c</h3>
				293	<p>Contains the main() function where execution starts, plus
				294	common infrastructure to initialize global variables and select which command
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	295	to run. The "toybox" multiplexer command also lives here. (This is the
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	296	only command defined outside of the toys directory.)</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	297
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	298	<p>Execution starts in main() which trims any path off of the first command
				299	name and calls toybox_main(), which calls toy_exec(), which calls toy_find()
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	300	and toy_init() before calling the appropriate command's function from
				301	toy_list[] (via toys.which->toy_main()).
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	302	If the command is "toybox", execution recurses into toybox_main(), otherwise
				303	the call goes to the appropriate commandname_main() from a C file in the toys
				304	directory.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	305
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	306	<p>The following global variables are defined in main.c:</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	307	<ul>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	308	<a name="toy_list" />
				309	<li><p><b>struct toy_list toy_list[]</b> - array describing all the
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	310	commands currently configured into toybox. The first entry (toy_list[0]) is
				311	for the "toybox" multiplexer command, which runs all the other built-in commands
				312	without symlinks by using its first argument as the name of the command to
				313	run and the rest as that command's argument list (ala "./toybox echo hello").
				314	The remaining entries are the commands in alphabetical order (for efficient
				315	binary search).</p>
				316
				317	<p>This is a read-only array initialized at compile time by
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	318	defining macros and #including generated/newtoys.h.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	319
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	320	<p>Members of struct toy_list (defined in "toys.h") include:</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	321	<ul>
				322	<li><p>char *<b>name</b> - the name of this command.</p></li>
				323	<li><p>void (*<b>toy_main</b>)(void) - function pointer to run this
				324	command.</p></li>
				325	<li><p>char *<b>options</b> - command line option string (used by
				326	get_optflags() in lib/args.c to intialize toys.optflags, toys.optargs, and
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	327	entries in the toy's GLOBALS struct). When this is NULL, no option
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	328	parsing is done before calling toy_main().</p></li>
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	329	<li><p>int <b>flags</b> - Behavior flags for this command. The following flags are currently understood:</p>
				330
				331	<ul>
				332	<li><b>TOYFLAG_USR</b> - Install this command under /usr</li>
				333	<li><b>TOYFLAG_BIN</b> - Install this command under /bin</li>
				334	<li><b>TOYFLAG_SBIN</b> - Install this command under /sbin</li>
				335	<li><b>TOYFLAG_NOFORK</b> - This command can be used as a shell builtin.</li>
				336	<li><b>TOYFLAG_UMASK</b> - Call umask(0) before running this command.</li>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	337	<li><b>TOYFLAG_STAYROOT</b> - Don't drop permissions for this command if toybox is installed SUID root.</li>
				338	<li><b>TOYFLAG_NEEDROOT</b> - This command cannot function unless run with root access.</li>
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	339	</ul>
				340	<br>
				341
				342	<p>These flags are combined with \| (or). For example, to install a command
				343	in /usr/bin, or together TOYFLAG_USR\|TOYFLAG_BIN.</p>
				344	</ul>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	345	</li>
				346
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	347	<li><p><b>struct toy_context toys</b> - global structure containing information
				348	common to all commands, initializd by toy_init() and defined in "toys.h".
				349	Members of this structure include:</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	350	<ul>
				351	<li><p>struct toy_list *<b>which</b> - a pointer to this command's toy_list
				352	structure. Mostly used to grab the name of the running command
				353	(toys->which.name).</p>
				354	</li>
				355	<li><p>int <b>exitval</b> - Exit value of this command. Defaults to zero. The
				356	error_exit() functions will return 1 if this is zero, otherwise they'll
				357	return this value.</p></li>
				358	<li><p>char **<b>argv</b> - "raw" command line options, I.E. the original
				359	unmodified string array passed in to main(). Note that modifying this changes
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	360	"ps" output, and is not recommended. This array is null terminated; a NULL
				361	entry indicates the end of the array.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	362	<p>Most commands don't use this field, instead the use optargs, optflags,
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	363	and the fields in the GLOBALS struct initialized by get_optflags().</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	364	</li>
				365	<li><p>unsigned <b>optflags</b> - Command line option flags, set by
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	366	<a href="#lib_args">get_optflags()</a>. Indicates which of the command line options listed in
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	367	toys->which.options occurred this time.</p>
				368
				369	<p>The rightmost command line argument listed in toys->which.options sets bit
				370	1, the next one sets bit 2, and so on. This means the bits are set in the same
				371	order the binary digits would be listed if typed out as a string. For example,
				372	the option string "abcd" would parse the command line "-c" to set optflags to 2,
				373	"-a" would set optflags to 8, and "-bd" would set optflags to 6 (4\|2).</p>
				374
				375	<p>Only letters are relevant to optflags. In the string "a*b:c#d", d=1, c=2,
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	376	b=4, a=8. Punctuation after a letter initializes global variables at the
				377	start of the GLOBALS() block (see <a href="#toy_union">union toy_union this</a>
				378	for details).</p>
				379
				380	<p>The build infrastructure creates FLAG_ macros for each option letter,
				381	corresponding to the bit position, so you can check (toys.optflags & FLAG_x)
				382	to see if a flag was specified. (The correct set of FLAG_ macros is selected
				383	by defining FOR_mycommand before #including toys.h. The macros live in
				384	toys/globals.h which is generated by scripts/make.sh.)</p>
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	385
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	386	<p>For more information on option parsing, see <a href="#lib_args">get_optflags()</a>.</p>
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	387
				388	</li>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	389	<li><p>char **<b>optargs</b> - Null terminated array of arguments left over
				390	after get_optflags() removed all the ones it understood. Note: optarg[0] is
				391	the first argument, not the command name. Use toys.which->name for the command
				392	name.</p></li>
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	393	<li><p>int <b>optc</b> - Optarg count, equivalent to argc but for
				394	optargs[].<p></li>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	395	<li><p>int <b>exithelp</b> - Whether error_exit() should print a usage message
				396	via help_main() before exiting. (True during option parsing, defaults to
				397	false afterwards.)</p></li>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	398	</ul>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	399
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	400	<a name="toy_union" />
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	401	<li><p><b>union toy_union this</b> - Union of structures containing each
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	402	command's global variables.</p>
				403
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	404	<p>Global variables are useful: they reduce the overhead of passing extra
				405	command line arguments between functions, they conveniently start prezeroed to
				406	save initialization costs, and the command line argument parsing infrastructure
				407	can also initialize global variables with its results.</p>
				408
				409	<p>But since each toybox process can only run one command at a time, allocating
				410	space for global variables belonging to other commands you aren't currently
				411	running would be wasteful.</p>
				412
				413	<p>Toybox handles this by encapsulating each command's global variables in
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	414	a structure, and declaring a union of those structures with a single global
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	415	instance (called "this"). The GLOBALS() macro contains the global
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	416	variables that should go in the current command's global structure. Each
				417	variable can then be accessed as "this.commandname.varname".
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	418	If you #defined FOR_commandname before including toys.h, the macro TT is
				419	#defined to this.commandname so the variable can then be accessed as
				420	"TT.variable". See toys/hello.c for an example.</p>
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	421
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	422	<p>A command that needs global variables should declare a structure to
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	423	contain them all, and add that structure to this union. A command should never
				424	declare global variables outside of this, because such global variables would
				425	allocate memory when running other commands that don't use those global
				426	variables.</p>
				427
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	428	<p>The first few fields of this structure can be intialized by <a href="#lib_args">get_optargs()</a>,
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	429	as specified by the options field off this command's toy_list entry. See
				430	the get_optargs() description in lib/args.c for details.</p>
				431	</li>
				432
Rob Landley	81b899d	2007-12-18 02:02:47 -0600	[diff] [blame]	433	<li><b>char toybuf[4096]</b> - a common scratch space buffer so
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	434	commands don't need to allocate their own. Any command is free to use this,
				435	and it should never be directly referenced by functions in lib/ (although
				436	commands are free to pass toybuf in to a library function as an argument).</li>
				437	</ul>
				438
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	439	<p>The following functions are defined in main.c:</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	440	<ul>
				441	<li><p>struct toy_list <b>toy_find</b>(char name) - Return the toy_list
				442	structure for this command name, or NULL if not found.</p></li>
Rob Landley	81b899d	2007-12-18 02:02:47 -0600	[diff] [blame]	443	<li><p>void <b>toy_init</b>(struct toy_list which, char argv[]) - fill out
				444	the global toys structure, calling get_optargs() if necessary.</p></li>
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	445	<li><p>void <b>toy_exec</b>(char *argv[]) - Run a built-in command with
				446	arguments.</p>
				447	<p>Calls toy_find() on argv[0] (which must be just a command name
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	448	without path). Returns if it can't find this command, otherwise calls
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	449	toy_init(), toys->which.toy_main(), and exit() instead of returning.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	450
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	451	<p>Use the library function xexec() to fall back to external executables
				452	in $PATH if toy_exec() can't find a built-in command. Note that toy_exec()
				453	does not strip paths before searching for a command, so "./command" will
				454	never match an internal command.</li>
				455
				456	<li><p>void <b>toybox_main</b>(void) - the main function for the multiplexer
				457	command (I.E. "toybox"). Given a command name as its first argument, calls
				458	toy_exec() on its arguments. With no arguments, it lists available commands.
				459	If the first argument starts with "-" it lists each command with its default
				460	install path prepended.</p></li>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	461
				462	</ul>
				463
				464	<h3>Config.in</h3>
				465
				466	<p>Top level configuration file in a stylized variant of
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	467	<a href=http://kernel.org/doc/Documentation/kbuild/kconfig-language.txt>kconfig</a> format. Includes generated/Config.in.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	468
				469	<p>These files are directly used by "make menuconfig" to select which commands
				470	to build into toybox (thus generating a .config file), and by
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	471	scripts/config2help.py to create generated/help.h.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	472
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	473	<a name="generated" />
				474	<h1><a href="#generated">Temporary files:</a></h1>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	475
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	476	<p>There is one temporary file in the top level source directory:</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	477	<ul>
				478	<li><p><b>.config</b> - Configuration file generated by kconfig, indicating
				479	which commands (and options to commands) are currently enabled. Used
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	480	to make generated/config.h and determine which toys//.c files to build.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	481
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	482	<p>You can create a human readable "miniconfig" version of this file using
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	483	<a href=http://landley.net/aboriginal/new_platform.html#miniconfig>these
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	484	instructions</a>.</p>
				485	</li>
				486	</ul>
				487
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	488	<p><h2>Directory generated/</h2></p>
				489
				490	<p>The remaining temporary files live in the "generated/" directory,
				491	which is for files generated at build time from other source files.</p>
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	492
				493	<ul>
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	494	<li><p><b>generated/Config.in</b> - Kconfig entries for each command, included
				495	from the top level Config.in. The help text here is used to generate
				496	help.h.</p>
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	497
				498	<p>Each command has a configuration entry with an upper case version of
				499	the command name. Options to commands start with the command
				500	name followed by an underscore and the option name. Global options are attached
				501	to the "toybox" command, and thus use the prefix "TOYBOX_". This organization
				502	is used by scripts/cfg2files to select which toys//.c files to compile for a
				503	given .config.</p>
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	504	</li>
				505
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	506	<li><p><b>generated/config.h</b> - list of CFG_SYMBOL and USE_SYMBOL() macros,
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	507	generated from .config by a sed invocation in scripts/make.sh.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	508
				509	<p>CFG_SYMBOL is a comple time constant set to 1 for enabled symbols and 0 for
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	510	disabled symbols. This allows the use of normal if() statements to remove
Rob Landley	6882ee8	2008-02-12 18:41:34 -0600	[diff] [blame]	511	code at compile time via the optimizer's dead code elimination (which removes
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	512	from the binary any code that cannot be reached). This saves space without
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	513	cluttering the code with #ifdefs or leading to configuration dependent build
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	514	breaks. (See the 1992 Usenix paper
Rob Landley	b6063de	2012-01-29 13:54:13 -0600	[diff] [blame]	515	<a href=http://doc.cat-v.org/henry_spencer/ifdef_considered_harmful.pdf>#ifdef
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	516	Considered Harmful</a> for more information.)</p>
				517
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	518	<p>When you can't entirely avoid an #ifdef, the USE_SYMBOL(code) macro
				519	provides a less intrusive alternative, evaluating to the code in parentheses
				520	when the symbol is enabled, and nothing when the symbol is disabled. This
				521	is most commonly used around NEWTOY() declarations (so only the enabled
				522	commands show up in toy_list), and in option strings. This can also be used
				523	for things like varargs or structure members which can't always be
				524	eliminated by a simple test on CFG_SYMBOL. Remember, unlike CFG_SYMBOL
				525	this is really just a variant of #ifdef, and can still result in configuration
				526	dependent build breaks. Use with caution.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	527	</li>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	528
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	529	<li><p><b>generated/flags.h</b> - FLAG_? macros indicating which command
				530	line options were seen. The option parsing in lib/args.c sets bits in
				531	toys.optflags, which can be tested by anding with the appropriate FLAG_
				532	macro. (Bare longopts, which have no corresponding short option, will
				533	have the longopt name after FLAG_. All others use the single letter short
				534	option.)</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	535
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	536	<p>To get the appropriate macros for your command, #define FOR_commandname
				537	before #including toys.h. To switch macro sets (because you have an OLDTOY()
				538	with different options in the same .c file), #define CLEANUP_oldcommand
				539	and also #define FOR_newcommand, then #include "generated/flags.h" to switch.
				540	</p>
				541	</li>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	542
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	543	<li><p><b>generated/globals.h</b> -
				544	Declares structures to hold the contents of each command's GLOBALS(),
				545	and combines them into "global_union this". (Yes, the name was
				546	chosen to piss off C++ developers who think that C
				547	is merely a subset of C++, not a language in its own right.)</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	548
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	549	<p>The union reuses the same memory for each command's global struct:
				550	since only one command's globals are in use at any given time, collapsing
				551	them together saves space. The headers #define TT to the appropriate
				552	"this.commandname", so you can refer to the current command's global
				553	variables out of "this" as TT.variablename.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	554
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	555	<p>The globals start zeroed, and the first few are filled out by the
				556	lib/args.c argument parsing code called from main.c.</p>
				557	</li>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	558
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	559	<li><p><b>toys/help.h</b> - Help strings for use by the "help" command and
				560	--help options. This file #defines a help_symbolname string for each
				561	symbolname, but only the symbolnames matching command names get used
				562	by show_help() in lib/help.c to display help for commands.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	563
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	564	<p>This file is created by scripts/make.sh, which compiles scripts/config2help.c
				565	into the binary generated/config2help, and then runs it against the top
				566	level .config and Config.in files to extract the help text from each config
				567	entry and collate together dependent options.</p>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	568
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	569	<p>This file contains help text for all commands, regardless of current
				570	configuration, but only the ones currently enabled in the .config file
				571	wind up in the help_data[] array, and only the enabled dependent options
				572	have their help text added to the command they depend on.</p>
				573	</li>
				574
				575	<li><p><b>generated/newtoys.h</b> -
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	576	All the NEWTOY() and OLDTOY() macros from toys//.c. The "toybox" multiplexer
				577	is the first entry, the rest are in alphabetical order. Each line should be
				578	inside an appropriate USE_ macro, so code that #includes this file only sees
				579	the currently enabled commands.</p>
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	580
				581	<p>By #definining NEWTOY() to various things before #including this file,
				582	it may be used to create function prototypes (in toys.h), initialize the
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	583	help_data array (in lib/help.c), initialize the toy_list array (in main.c,
				584	the alphabetical order lets toy_find() do a binary search, the exception to
				585	the alphabetical order lets it use the multiplexer without searching), and so
				586	on. (It's even used to initialize the NEED_OPTIONS macro, which produces a 1
				587	or 0 for each command using command line option parsing, which is ORed together
				588	to allow compile-time dead code elimination to remove the whole of
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	589	lib/args.c if nothing currently enabled is using it.)<p>
				590
				591	<p>Each NEWTOY and OLDTOY macro contains the command name, command line
				592	option string (telling lib/args.c how to parse command line options for
				593	this command), recommended install location, and miscelaneous data such
				594	as whether this command should retain root permissions if installed suid.</p>
				595	</li>
				596
				597	<li><p><b>toys/oldtoys.h</b> - Macros with the command line option parsing
				598	string for each NEWTOY. This allows an OLDTOY that's just an alias for an
				599	existing command to refer to the existing option string instead of
				600	having to repeat it.</p>
				601	</li>
				602	</ul>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	603
Rob Landley	137bf34	2012-03-09 08:33:57 -0600	[diff] [blame]	604	<a name="lib">
Rob Landley	81b899d	2007-12-18 02:02:47 -0600	[diff] [blame]	605	<h2>Directory lib/</h2>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	606
Rob Landley	137bf34	2012-03-09 08:33:57 -0600	[diff] [blame]	607	<p>TODO: document lots more here.</p>
				608
Rob Landley	c8d0da5	2012-07-15 17:47:08 -0500	[diff] [blame]	609	<p>lib: getmountlist(), error_msg/error_exit, xmalloc(),
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	610	strlcpy(), xexec(), xopen()/xread(), xgetcwd(), xabspath(), find_in_path(),
				611	itoa().</p>
				612
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	613
				614
				615	<a name="lib_xwrap"><h3>lib/xwrap.c</h3>
				616
				617	<p>Functions prefixed with the letter x call perror_exit() when they hit
				618	errors, to eliminate common error checking. This prints an error message
				619	and the strerror() string for the errno encountered.</p>
				620
				621	<p>You can intercept this exit by assigning a setjmp/longjmp buffer to
				622	toys.rebound (set it back to zero to restore the default behavior).
				623	If you do this, cleaning up resource leaks is your problem.</p>
				624
				625	<ul>
				626	<li><b>void xstrncpy(char dest, char src, size_t size)</b></li>
				627	<li><b>void xexit(void)</b></li>
				628	<li><b>void *xmalloc(size_t size)</b></li>
				629	<li><b>void *xzalloc(size_t size)</b></li>
				630	<li><b>void xrealloc(void ptr, size_t size)</b></li>
				631	<li><b>char xstrndup(char s, size_t n)</b></li>
				632	<li><b>char xstrdup(char s)</b></li>
				633	<li><b>char xmprintf(char format, ...)</b></li>
				634	<li><b>void xprintf(char *format, ...)</b></li>
				635	<li><b>void xputs(char *s)</b></li>
				636	<li><b>void xputc(char c)</b></li>
				637	<li><b>void xflush(void)</b></li>
				638	<li><b>pid_t xfork(void)</b></li>
				639	<li><b>void xexec_optargs(int skip)</b></li>
				640	<li><b>void xexec(char **argv)</b></li>
				641	<li><b>pid_t xpopen(char *argv, int pipes)</b></li>
				642	<li><b>int xpclose(pid_t pid, int *pipes)</b></li>
				643	<li><b>void xaccess(char *path, int flags)</b></li>
				644	<li><b>void xunlink(char *path)</b></li>
				645	<li><p><b>int xcreate(char *path, int flags, int mode)<br />
				646	int xopen(char *path, int flags)</b></p>
				647
				648	<p>The xopen() and xcreate() functions open an existing file (exiting if
				649	it's not there) and create a new file (exiting if it can't).</p>
				650
				651	<p>They default to O_CLOEXEC so the filehandles aren't passed on to child
				652	processes. Feed in O_CLOEXEC to disable this.</p>
				653	</li>
				654	<li><p><b>void xclose(int fd)</b></p>
				655
				656	<p>Because NFS is broken, and won't necessarily perform the requested
				657	operation (and report the error) until you close the file. Of course, this
				658	being NFS, it's not guaranteed to report the error there either, but it
				659	_can_.</p>
				660
				661	<p>Nothing else ever reports an error on close, everywhere else it's just a
				662	VFS operation freeing some resources. NFS is _special_, in a way that
				663	other network filesystems like smbfs and v9fs aren't..</p>
				664	</li>
				665	<li><b>int xdup(int fd)</b></li>
				666	<li><p><b>size_t xread(int fd, void *buf, size_t len)</b></p>
				667
				668	<p>Can return 0, but not -1.</p>
				669	</li>
				670	<li><p><b>void xreadall(int fd, void *buf, size_t len)</b></p>
				671
				672	<p>Reads the entire len-sized buffer, retrying to complete short
				673	reads. Exits if it can't get enough data.</p></li>
				674
				675	<li><p><b>void xwrite(int fd, void *buf, size_t len)</b></p>
				676
				677	<p>Retries short writes, exits if can't write the entire buffer.</p></li>
				678
				679	<li><b>off_t xlseek(int fd, off_t offset, int whence)</b></li>
				680	<li><b>char *xgetcwd(void)</b></li>
				681	<li><b>void xstat(char path, struct stat st)</b></li>
				682	<li><p><b>char xabspath(char path, int exact) </b></p>
				683
				684	<p>After several years of
				685	<a href=http://landley.net/notes-2007.html#18-06-2007>wrestling</a>
				686	<a href=http://landley.net/notes-2008.html#19-01-2008>with</a> realpath(),
				687	I broke down and <a href=http://landley.net/notes-2012.html#20-11-2012>wrote
				688	my own</a> implementation that doesn't use the one in libc. As I explained:
				689
				690	<blockquote><p>If the path ends with a broken link,
				691	readlink -f should show where the link points to, not where the broken link
				692	lives. (The point of readlink -f is "if I write here, where would it attempt
				693	to create a file".) The problem is, realpath() returns NULL for a path ending
				694	with a broken link, and I can't beat different behavior out of code locked
				695	away in libc.</p></blockquote>
				696
				697	<p>
				698	</li>
				699	<li><b>void xchdir(char *path)</b></li>
				700	<li><b>void xchroot(char *path)</b></li>
				701
				702	<li><p><b>struct passwd *xgetpwuid(uid_t uid)<br />
				703	struct group *xgetgrgid(gid_t gid)<br />
				704	struct passwd xgetpwnam(char name)</b></p>
				705
				706	<p></p>
				707	</li>
				708
				709
				710
				711	<li><b>void xsetuser(struct passwd *pwd)</b></li>
				712	<li><b>char xreadlink(char name)</b></li>
				713	<li><b>char xreadfile(char name, char *buf, off_t len)</b></li>
				714	<li><b>int xioctl(int fd, int request, void *data)</b></li>
				715	<li><b>void xpidfile(char *name)</b></li>
				716	<li><b>void xsendfile(int in, int out)</b></li>
				717	<li><b>long xparsetime(char arg, long units, long fraction)</b></li>
				718	<li><b>void xregcomp(regex_t preg, char regex, int cflags)</b></li>
				719	</ul>
				720
				721	<a name="lib_lib"><h3>lib/lib.c</h3>
				722	<p>Eight gazillion common functions:</p>
				723
				724	<ul>
				725	<li><b>void verror_msg(char *msg, int err, va_list va)</b></li>
				726	<li><b>void error_msg(char *msg, ...)</b></li>
				727	<li><b>void perror_msg(char *msg, ...)</b></li>
				728	<li><b>void error_exit(char *msg, ...)</b></li>
				729	<li><b>void perror_exit(char *msg, ...)</b></li>
				730	<li><b>ssize_t readall(int fd, void *buf, size_t len)</b></li>
				731	<li><b>ssize_t writeall(int fd, void *buf, size_t len)</b></li>
				732	<li><b>off_t lskip(int fd, off_t offset)</b></li>
				733	<li><b>int mkpathat(int atfd, char *dir, mode_t lastmode, int flags)</b></li>
				734	<li><b>struct string_list *splitpath(char path, struct string_list **list)</b></li>
				735	<li><b>struct string_list find_in_path(char path, char *filename)</b></li>
				736	<li><b>long atolx(char *numstr)</b></li>
				737	<li><b>long atolx_range(char *numstr, long low, long high)</b></li>
				738	<li><b>int numlen(long l)</b></li>
				739	<li><b>int stridx(char *haystack, char needle)</b></li>
				740	<li><b>int strstart(char *a, char b)</b></li>
				741	<li><b>off_t fdlength(int fd)</b></li>
				742	<li><b>char readfile(char name, char *ibuf, off_t len)</b></li>
				743	<li><b>void msleep(long miliseconds)</b></li>
				744	<li><b>int64_t peek_le(void *ptr, unsigned size)</b></li>
				745	<li><b>int64_t peek_be(void *ptr, unsigned size)</b></li>
				746	<li><b>int64_t peek(void *ptr, unsigned size)</b></li>
				747	<li><b>void poke(void *ptr, uint64_t val, int size)</b></li>
				748	<li><b>void loopfiles_rw(char **argv, int flags, int permissions, int failok,</b></li>
				749	<li><b>void loopfiles(char *argv, void (function)(int fd, char *name))</b></li>
				750	<li><b>char get_rawline(int fd, long plen, char end)</b></li>
				751	<li><b>char *get_line(int fd)</b></li>
				752	<li><b>int wfchmodat(int fd, char *name, mode_t mode)</b></li>
				753	<li><b>static void tempfile_handler(int i)</b></li>
				754	<li><b>int copy_tempfile(int fdin, char name, char *tempname)</b></li>
				755	<li><b>void delete_tempfile(int fdin, int fdout, char **tempname)</b></li>
				756	<li><b>void replace_tempfile(int fdin, int fdout, char **tempname)</b></li>
				757	<li><b>void crc_init(unsigned int *crc_table, int little_endian)</b></li>
				758	<li><b>int terminal_size(unsigned xx, unsigned yy)</b></li>
				759	<li><b>int yesno(char *prompt, int def)</b></li>
				760	<li><b>void generic_signal(int sig)</b></li>
				761	<li><b>void sigatexit(void *handler)</b></li>
				762	<li><b>int sig_to_num(char *pidstr)</b></li>
				763	<li><b>char *num_to_sig(int sig)</b></li>
				764	<li><b>mode_t string_to_mode(char *modestr, mode_t mode)</b></li>
				765	<li><b>void mode_to_string(mode_t mode, char *buf)</b></li>
				766	<li><b>void names_to_pid(char *names, int (callback)(pid_t pid, char *name))</b></li>
				767	<li><b>int human_readable(char *buf, unsigned long long num)</b></li>
				768	</ul>
				769
Rob Landley	137bf34	2012-03-09 08:33:57 -0600	[diff] [blame]	770	<h3>lib/portability.h</h3>
				771
				772	<p>This file is automatically included from the top of toys.h, and smooths
				773	over differences between platforms (hardware targets, compilers, C libraries,
				774	operating systems, etc).</p>
				775
				776	<p>This file provides SWAP macros (SWAP_BE16(x) and SWAP_LE32(x) and so on).</p>
				777
				778	<p>A macro like SWAP_LE32(x) means "The value in x is stored as a little
				779	endian 32 bit value, so perform the translation to/from whatever the native
				780	32-bit format is". You do the swap once on the way in, and once on the way
				781	out. If your target is already little endian, the macro is a NOP.</p>
				782
				783	<p>The SWAP macros come in BE and LE each with 16, 32, and 64 bit versions.
				784	In each case, the name of the macro refers to the _external_ representation,
				785	and converts to/from whatever your native representation happens to be (which
				786	can vary depending on what you're currently compiling for).</p>
				787
Rob Landley	c8d0da5	2012-07-15 17:47:08 -0500	[diff] [blame]	788	<a name="lib_llist"><h3>lib/llist.c</h3>
				789
				790	<p>Some generic single and doubly linked list functions, which take
				791	advantage of a couple properties of C:</p>
				792
				793	<ul>
				794	<li><p>Structure elements are laid out in memory in the order listed, and
				795	the first element has no padding. This means you can always treat (typecast)
				796	a pointer to a structure as a pointer to the first element of the structure,
				797	even if you don't know anything about the data following it.</p></li>
				798
				799	<li><p>An array of length zero at the end of a structure adds no space
				800	to the sizeof() the structure, but if you calculate how much extra space
				801	you want when you malloc() the structure it will be available at the end.
				802	Since C has no bounds checking, this means each struct can have one variable
				803	length array.</p></li>
				804	</ul>
				805
				806	<p>Toybox's list structures always have their <b>next</b> pointer as
				807	the first entry of each struct, and singly linked lists end with a NULL pointer.
				808	This allows generic code to traverse such lists without knowing anything
				809	else about the specific structs composing them: if your pointer isn't NULL
				810	typecast it to void ** and dereference once to get the next entry.</p>
				811
				812	<p><b>lib/lib.h</b> defines three structure types:</p>
				813	<ul>
				814	<li><p><b>struct string_list</b> - stores a single string (<b>char str[0]</b>),
				815	memory for which is allocated as part of the node. (I.E. llist_traverse(list,
				816	free); can clean up after this type of list.)</p></li>
				817
				818	<li><p><b>struct arg_list</b> - stores a pointer to a single string
				819	(<b>char *arg</b>) which is stored in a separate chunk of memory.</p></li>
				820
				821	<li><p><b>struct double_list</b> - has a second pointer (<b>struct double_list
				822	prev</b> along with a <b>char data</b> for payload.</p></li>
				823	</ul>
				824
				825	<b>List Functions</b>
				826
				827	<ul>
				828	<li><p>void <b>llist_pop</b>(void *list) - advances through a list ala
				829	<b>node = llist_pop(&list);</b> This doesn't modify the list contents,
				830	but does advance the pointer you feed it (which is why you pass the _address_
				831	of that pointer, not the pointer itself).</p></li>
				832
				833	<li><p>void <b>llist_traverse</b>(void list, void (using)(void *data)) -
				834	iterate through a list calling a function on each node.</p></li>
				835
				836	<li><p>struct double_list <b>dlist_add</b>(struct double_list llist, char data)
				837	- append an entry to a circular linked list.
				838	This function allocates a new struct double_list wrapper and returns the
				839	pointer to the new entry (which you can usually ignore since it's llist->prev,
				840	but if llist was NULL you need it). The argument is the ->data field for the
				841	new node.</p></li>
				842	<ul><li><p>void <b>dlist_add_nomalloc</b>(struct double_list **llist,
				843	struct double_list *new) - append existing struct double_list to
				844	list, does not allocate anything.</p></li></ul>
				845	</ul>
				846
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	847	<b>List code trivia questions:</b>
Rob Landley	c8d0da5	2012-07-15 17:47:08 -0500	[diff] [blame]	848
				849	<ul>
				850	<li><p><b>Why do arg_list and double_list contain a char * payload instead of
				851	a void ?</b> - Because you always have to typecast a void to use it, and
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	852	typecasting a char * does no harm. Since strings are the most common
				853	payload, and doing math on the pointer ala
				854	"(type )(ptr+sizeof(thing)+sizeof(otherthing))" requires ptr to be char
				855	anyway (at least according to the C standard), defaulting to char * saves
				856	a typecast.</p>
Rob Landley	c8d0da5	2012-07-15 17:47:08 -0500	[diff] [blame]	857	</li>
				858
				859	<li><p><b>Why do the names ->str, ->arg, and ->data differ?</b> - To force
				860	you to keep track of which one you're using, calling free(node->str) would
				861	be bad, and _failing_ to free(node->arg) leaks memory.</p></li>
				862
				863	<li><p><b>Why does llist_pop() take a void * instead of void **?</b> -
				864	because the stupid compiler complains about "type punned pointers" when
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	865	you typecast and dereference on the same line,
Rob Landley	c8d0da5	2012-07-15 17:47:08 -0500	[diff] [blame]	866	due to insane FSF developers hardwiring limitations of their optimizer
				867	into gcc's warning system. Since C automatically typecasts any other
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	868	pointer type to and from void *, the current code works fine. It's sad that it
Rob Landley	c8d0da5	2012-07-15 17:47:08 -0500	[diff] [blame]	869	won't warn you if you forget the &, but the code crashes pretty quickly in
				870	that case.</p></li>
				871
				872	<li><p><b>How do I assemble a singly-linked-list in order?</b> - use
				873	a double_list, dlist_add() your entries, and then break the circle with
				874	<b>list->prev->next = NULL;</b> when done.</li>
				875	</ul>
				876
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	877	<a name="lib_args"><h3>lib/args.c</h3>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	878
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	879	<p>Toybox's main.c automatically parses command line options before calling the
				880	command's main function. Option parsing starts in get_optflags(), which stores
				881	results in the global structures "toys" (optflags and optargs) and "this".</p>
				882
				883	<p>The option parsing infrastructure stores a bitfield in toys.optflags to
				884	indicate which options the current command line contained. Arguments
				885	attached to those options are saved into the command's global structure
				886	("this"). Any remaining command line arguments are collected together into
				887	the null-terminated array toys.optargs, with the length in toys.optc. (Note
				888	that toys.optargs does not contain the current command name at position zero,
				889	use "toys.which->name" for that.) The raw command line arguments get_optflags()
				890	parsed are retained unmodified in toys.argv[].</p>
				891
				892	<p>Toybox's option parsing logic is controlled by an "optflags" string, using
				893	a format reminiscent of getopt's optargs but has several important differences.
				894	Toybox does not use the getopt()
				895	function out of the C library, get_optflags() is an independent implementation
				896	which doesn't permute the original arguments (and thus doesn't change how the
				897	command is displayed in ps and top), and has many features not present in
				898	libc optargs() (such as the ability to describe long options in the same string
				899	as normal options).</p>
				900
				901	<p>Each command's NEWTOY() macro has an optflags string as its middle argument,
				902	which sets toy_list.options for that command to tell get_optflags() what
				903	command line arguments to look for, and what to do with them.
				904	If a command has no option
				905	definition string (I.E. the argument is NULL), option parsing is skipped
				906	for that command, which must look at the raw data in toys.argv to parse its
				907	own arguments. (If no currently enabled command uses option parsing,
				908	get_optflags() is optimized out of the resulting binary by the compiler's
				909	--gc-sections option.)</p>
				910
				911	<p>You don't have to free the option strings, which point into the environment
				912	space (I.E. the string data is not copied). A TOYFLAG_NOFORK command
				913	that uses the linked list type "*" should free the list objects but not
				914	the data they point to, via "llist_free(TT.mylist, NULL);". (If it's not
				915	NOFORK, exit() will free all the malloced data anyway unless you want
				916	to implement a CONFIG_TOYBOX_FREE cleanup for it.)</p>
				917
				918	<h4>Optflags format string</h4>
				919
				920	<p>Note: the optflags option description string format is much more
				921	concisely described by a large comment at the top of lib/args.c.</p>
				922
				923	<p>The general theory is that letters set optflags, and punctuation describes
				924	other actions the option parsing logic should take.</p>
				925
				926	<p>For example, suppose the command line <b>command -b fruit -d walrus -a 42</b>
				927	is parsed using the optflags string "<b>a#b:c:d</b>". (I.E.
				928	toys.which->options="a#b:c:d" and argv = ["command", "-b", "fruit", "-d",
				929	"walrus", "-a", "42"]). When get_optflags() returns, the following data is
				930	available to command_main():
				931
				932	<ul>
				933	<li><p>In <b>struct toys</b>:
				934	<ul>
				935	<li>toys.optflags = 13; // -a = 8 \| -b = 4 \| -d = 1</li>
				936	<li>toys.optargs[0] = "walrus"; // leftover argument</li>
				937	<li>toys.optargs[1] = NULL; // end of list</li>
Rob Landley	b911d4d	2013-09-21 14:27:26 -0500	[diff] [blame]	938	<li>toys.optc = 1; // there was 1 leftover argument</li>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	939	<li>toys.argv[] = {"-b", "fruit", "-d", "walrus", "-a", "42"}; // The original command line arguments
				940	</ul>
				941	<p></li>
				942
				943	<li><p>In <b>union this</b> (treated as <b>long this[]</b>):
				944	<ul>
				945	<li>this[0] = NULL; // -c didn't get an argument this time, so get_optflags() didn't change it and toys_init() zeroed "this" during setup.)</li>
				946	<li>this[1] = (long)"fruit"; // argument to -b</li>
				947	<li>this[2] = 42; // argument to -a</li>
				948	</ul>
				949	</p></li>
				950	</ul>
				951
				952	<p>If the command's globals are:</p>
				953
				954	<blockquote><pre>
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	955	GLOBALS(
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	956	char *c;
				957	char *b;
				958	long a;
				959	)
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	960	</pre></blockquote>
				961	<p>That would mean TT.c == NULL, TT.b == "fruit", and TT.a == 42. (Remember,
				962	each entry that receives an argument must be a long or pointer, to line up
				963	with the array position. Right to left in the optflags string corresponds to
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	964	top to bottom in GLOBALS().</p>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	965
Rob Landley	b911d4d	2013-09-21 14:27:26 -0500	[diff] [blame]	966	<p>Put globals not filled out by the option parsing logic at the end of the
				967	GLOBALS block. Common practice is to list the options one per line (to
				968	make the ordering explicit, first to last in globals corresponds to right
				969	to left in the option string), then leave a blank line before any non-option
				970	globals.</p>
				971
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	972	<p><b>long toys.optflags</b></p>
				973
				974	<p>Each option in the optflags string corresponds to a bit position in
				975	toys.optflags, with the same value as a corresponding binary digit. The
				976	rightmost argument is (1<<0), the next to last is (1<<1) and so on. If
Rob Landley	b4a0efa	2012-02-06 21:15:19 -0600	[diff] [blame]	977	the option isn't encountered while parsing argv[], its bit remains 0.</p>
				978
				979	<p>For example,
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	980	the optflags string "abcd" would parse the command line argument "-c" to set
				981	optflags to 2, "-a" would set optflags to 8, "-bd" would set optflags to
				982	6 (I.E. 4\|2), and "-a -c" would set optflags to 10 (2\|8).</p>
				983
				984	<p>Only letters are relevant to optflags, punctuation is skipped: in the
				985	string "a*b:c#d", d=1, c=2, b=4, a=8. The punctuation after a letter
				986	usually indicate that the option takes an argument.</p>
				987
Rob Landley	b4a0efa	2012-02-06 21:15:19 -0600	[diff] [blame]	988	<p>Since toys.optflags is an unsigned int, it only stores 32 bits. (Which is
				989	the amount a long would have on 32-bit platforms anyway; 64 bit code on
				990	32 bit platforms is too expensive to require in common code used by almost
				991	all commands.) Bit positions beyond the 1<<31 aren't recorded, but
				992	parsing higher options can still set global variables.</p>
				993
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	994	<p><b>Automatically setting global variables from arguments (union this)</b></p>
				995
				996	<p>The following punctuation characters may be appended to an optflags
				997	argument letter, indicating the option takes an additional argument:</p>
				998
				999	<ul>
				1000	<li><b>:</b> - plus a string argument, keep most recent if more than one.</li>
				1001	<li><b>*</b> - plus a string argument, appended to a linked list.</li>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1002	<li><b>@</b> - plus an occurrence counter (stored in a long)</li>
Rob Landley	b6063de	2012-01-29 13:54:13 -0600	[diff] [blame]	1003	<li><b>#</b> - plus a signed long argument.
Rob Landley	b911d4d	2013-09-21 14:27:26 -0500	[diff] [blame]	1004	<li><b>-</b> - plus a signed long argument defaulting to negative (start argument with + to force a positive value).</li>
Rob Landley	b6063de	2012-01-29 13:54:13 -0600	[diff] [blame]	1005	<li><b>.</b> - plus a floating point argument (if CFG_TOYBOX_FLOAT).</li>
				1006	<ul>The following can be appended to a float or double:
				1007	<li><b><123</b> - error if argument is less than this</li>
				1008	<li><b>>123</b> - error if argument is greater than this</li>
				1009	<li><b>=123</b> - default value if argument not supplied</li>
				1010	</ul>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1011	</ul>
				1012
Rob Landley	b911d4d	2013-09-21 14:27:26 -0500	[diff] [blame]	1013	<p>A note about "." and CFG_TOYBOX_FLOAT: option parsing only understands <>=
				1014	after . when CFG_TOYBOX_FLOAT
				1015	is enabled. (Otherwise the code to determine where floating point constants
				1016	end drops out; it requires floating point). When disabled, it can reserve a
				1017	global data slot for the argument (so offsets won't change in your
				1018	GLOBALS[] block), but will never fill it out. You can handle
				1019	this by using the USE_BLAH() macros with C string concatenation, ala:
				1020	"abc." USE_TOYBOX_FLOAT("<1.23>4.56=7.89") "def"</p>
				1021
				1022	<p><b>GLOBALS</b></p>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1023
				1024	<p>Options which have an argument fill in the corresponding slot in the global
				1025	union "this" (see generated/globals.h), treating it as an array of longs
Rob Landley	b911d4d	2013-09-21 14:27:26 -0500	[diff] [blame]	1026	with the rightmost saved in this[0]. As described above, using "a*b:c#d",
				1027	"-c 42" would set this[0] = 42; and "-b 42" would set this[1] = "42"; each
				1028	slot is left NULL if the corresponding argument is not encountered.</p>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1029
				1030	<p>This behavior is useful because the LP64 standard ensures long and pointer
Rob Landley	b4a0efa	2012-02-06 21:15:19 -0600	[diff] [blame]	1031	are the same size. C99 guarantees structure members will occur in memory
				1032	in the same order they're declared, and that padding won't be inserted between
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1033	consecutive variables of register size. Thus the first few entries can
				1034	be longs or pointers corresponding to the saved arguments.</p>
				1035
Rob Landley	b911d4d	2013-09-21 14:27:26 -0500	[diff] [blame]	1036	<p>See toys/other/hello.c for a longer example of parsing options into the
				1037	GLOBALS block.</p>
				1038
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1039	<p><b>char *toys.optargs[]</b></p>
				1040
				1041	<p>Command line arguments in argv[] which are not consumed by option parsing
				1042	(I.E. not recognized either as -flags or arguments to -flags) will be copied
				1043	to toys.optargs[], with the length of that array in toys.optc.
				1044	(When toys.optc is 0, no unrecognized command line arguments remain.)
				1045	The order of entries is preserved, and as with argv[] this new array is also
				1046	terminated by a NULL entry.</p>
				1047
				1048	<p>Option parsing can require a minimum or maximum number of optargs left
				1049	over, by adding "<1" (read "at least one") or ">9" ("at most nine") to the
				1050	start of the optflags string.</p>
				1051
				1052	<p>The special argument "--" terminates option parsing, storing all remaining
				1053	arguments in optargs. The "--" itself is consumed.</p>
				1054
				1055	<p><b>Other optflags control characters</b></p>
				1056
				1057	<p>The following characters may occur at the start of each command's
				1058	optflags string, before any options that would set a bit in toys.optflags:</p>
				1059
				1060	<ul>
				1061	<li><b>^</b> - stop at first nonoption argument (for nice, xargs...)</li>
				1062	<li><b>?</b> - allow unknown arguments (pass non-option arguments starting
				1063	with - through to optargs instead of erroring out).</li>
				1064	<li><b>&</b> - the first argument has imaginary dash (ala tar/ps. If given twice, all arguments have imaginary dash.)</li>
				1065	<li><b><</b> - must be followed by a decimal digit indicating at least this many leftover arguments are needed in optargs (default 0)</li>
				1066	<li><b>></b> - must be followed by a decimal digit indicating at most this many leftover arguments allowed (default MAX_INT)</li>
				1067	</ul>
				1068
				1069	<p>The following characters may be appended to an option character, but do
				1070	not by themselves indicate an extra argument should be saved in this[].
				1071	(Technically any character not recognized as a control character sets an
				1072	optflag, but letters are never control characters.)</p>
				1073
				1074	<ul>
				1075	<li><b>^</b> - stop parsing options after encountering this option, everything else goes into optargs.</li>
				1076	<li><b>\|</b> - this option is required. If more than one marked, only one is required.</li>
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1077	</ul>
				1078
Rob Landley	b6063de	2012-01-29 13:54:13 -0600	[diff] [blame]	1079	<p>The following may be appended to a float or double:</p>
				1080
				1081	<ul>
				1082	<li><b><123</b> - error if argument is less than this</li>
				1083	<li><b>>123</b> - error if argument is greater than this</li>
				1084	<li><b>=123</b> - default value if argument not supplied</li>
				1085	</ul>
				1086
				1087	<p>Option parsing only understands <>= after . when CFG_TOYBOX_FLOAT
				1088	is enabled. (Otherwise the code to determine where floating point constants
				1089	end drops out. When disabled, it can reserve a global data slot for the
				1090	argument so offsets won't change, but will never fill it out.). You can handle
				1091	this by using the USE_BLAH() macros with C string concatenation, ala:</p>
				1092
				1093	<blockquote>"abc." USE_TOYBOX_FLOAT("<1.23>4.56=7.89") "def"</blockquote>
				1094
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1095	<p><b>--longopts</b></p>
				1096
				1097	<p>The optflags string can contain long options, which are enclosed in
				1098	parentheses. They may be appended to an existing option character, in
				1099	which case the --longopt is a synonym for that option, ala "a:(--fred)"
				1100	which understands "-a blah" or "--fred blah" as synonyms.</p>
				1101
				1102	<p>Longopts may also appear before any other options in the optflags string,
				1103	in which case they have no corresponding short argument, but instead set
				1104	their own bit based on position. So for "(walrus)#(blah)xy:z" "command
				1105	--walrus 42" would set toys.optflags = 16 (-z = 1, -y = 2, -x = 4, --blah = 8)
				1106	and would assign this[1] = 42;</p>
				1107
				1108	<p>A short option may have multiple longopt synonyms, "a(one)(two)", but
				1109	each "bare longopt" (ala "(one)(two)abc" before any option characters)
				1110	always sets its own bit (although you can group them with +X).</p>
Rob Landley	7c04f01	2008-01-20 19:00:16 -0600	[diff] [blame]	1111
Rob Landley	b911d4d	2013-09-21 14:27:26 -0500	[diff] [blame]	1112	<p><b>[groups]</b></p>
				1113
				1114	<p>At the end of the option string, square bracket groups can define
				1115	relationships between existing options. (This only applies to short
				1116	options, bare --longopts can't participate.)</p>
				1117
				1118	<p>The first character of the group defines the type, the remaining
				1119	characters are options it applies to:</p>
				1120
				1121	<ul>
				1122	<li><b>-</b> - Exclusive, switch off all others in this group.</li>
				1123	<li><b>+</b> - Inclusive, switch on all others in this group.</li>
				1124	<li><b>!</b> - Error, fail if more than one defined.</li>
				1125	</ul>
				1126
				1127	<p>So "abc[-abc]" means -ab = -b, -ba = -a, -abc = -c. "abc[+abc]"
				1128	means -ab=-abc, -c=-abc, and "abc[!abc] means -ab calls error_exit("no -b
				1129	with -a"). Note that [-] groups clear the GLOBALS option slot of
				1130	options they're switching back off, but [+] won't set options it didn't see
				1131	(just the optflags).</p>
				1132
				1133	<p><b>whitespace</b></p>
				1134
				1135	<p>Arguments may occur with or without a space (I.E. "-a 42" or "-a42").
				1136	The command line argument "-abc" may be interepreted many different ways:
				1137	the optflags string "cba" sets toys.optflags = 7, "c:ba" sets toys.optflags=4
				1138	and saves "ba" as the argument to -c, and "cb:a" sets optflags to 6 and saves
				1139	"c" as the argument to -b.</p>
				1140
				1141	<p>Note that & changes whitespace handling, so that the command line
				1142	"tar cvfCj outfile.tar.bz2 topdir filename" is parsed the same as
				1143	"tar filename -c -v -j -f outfile.tar.bz2 -C topdir". Note that "tar -cvfCj
				1144	one two three" would equal "tar -c -v -f Cj one two three". (This matches
				1145	historical usage.)</p>
				1146
				1147	<p>Appending a space to the option in the option string ("a: b") makes it
				1148	require a space, I.E. "-ab" is interpreted as "-a" "-b". That way "kill -stop"
				1149	differs from "kill -s top".</p>
				1150
				1151	<p>Appending ; to a longopt in the option string makes its argument optional,
				1152	and only settable with =, so in ls "(color):;" can accept "ls --color" and
				1153	"ls --color=auto" without complaining that the first has no argument.</p>
				1154
Rob Landley	c8d0da5	2012-07-15 17:47:08 -0500	[diff] [blame]	1155	<a name="lib_dirtree"><h3>lib/dirtree.c</h3>
				1156
				1157	<p>The directory tree traversal code should be sufficiently generic
				1158	that commands never need to use readdir(), scandir(), or the fts.h family
				1159	of functions.</p>
				1160
				1161	<p>These functions do not call chdir() or rely on PATH_MAX. Instead they
				1162	use openat() and friends, using one filehandle per directory level to
				1163	recurseinto subdirectories. (I.E. they can descend 1000 directories deep
				1164	if setrlimit(RLIMIT_NOFILE) allows enough open filehandles, and the default
				1165	in /proc/self/limits is generally 1024.)</p>
				1166
				1167	<p>The basic dirtree functions are:</p>
				1168
				1169	<ul>
				1170	<li><p><b>dirtree_read(char path, int (callback)(struct dirtree node))</b> -
				1171	recursively read directories, either applying callback() or returning
				1172	a tree of struct dirtree if callback is NULL.</p></li>
				1173
				1174	<li><p><b>dirtree_path(struct dirtree node, int plen)</b> - malloc() a
				1175	string containing the path from the root of this tree to this node. If
				1176	plen isn't NULL then *plen is how many extra bytes to malloc at the end
				1177	of string.</p></li>
				1178
				1179	<li><p><b>dirtree_parentfd(struct dirtree *node)</b> - return fd of
				1180	containing directory, for use with openat() and such.</p></li>
				1181	</ul>
				1182
				1183	<p>The <b>dirtree_read()</b> function takes two arguments, a starting path for
				1184	the root of the tree, and a callback function. The callback takes a
				1185	<b>struct dirtree *</b> (from lib/lib.h) as its argument. If the callback is
				1186	NULL, the traversal uses a default callback (dirtree_notdotdot()) which
				1187	recursively assembles a tree of struct dirtree nodes for all files under
				1188	this directory and subdirectories (filtering out "." and ".." entries),
				1189	after which dirtree_read() returns the pointer to the root node of this
				1190	snapshot tree.</p>
				1191
				1192	<p>Otherwise the callback() is called on each entry in the directory,
				1193	with struct dirtree * as its argument. This includes the initial
				1194	node created by dirtree_read() at the top of the tree.</p>
				1195
				1196	<p><b>struct dirtree</b></p>
				1197
				1198	<p>Each struct dirtree node contains <b>char name[]</b> and <b>struct stat
				1199	st</b> entries describing a file, plus a <b>char *symlink</b>
				1200	which is NULL for non-symlinks.</p>
				1201
				1202	<p>During a callback function, the <b>int data</b> field of directory nodes
				1203	contains a dirfd (for use with the openat() family of functions). This is
				1204	generally used by calling dirtree_parentfd() on the callback's node argument.
				1205	For symlinks, data contains the length of the symlink string. On the second
				1206	callback from DIRTREE_COMEAGAIN (depth-first traversal) data = -1 for
				1207	all nodes (that's how you can tell it's the second callback).</p>
				1208
				1209	<p>Users of this code may put anything they like into the <b>long extra</b>
				1210	field. For example, "cp" and "mv" use this to store a dirfd for the destination
				1211	directory (and use DIRTREE_COMEAGAIN to get the second callback so they can
				1212	close(node->extra) to avoid running out of filehandles).
				1213	This field is not directly used by the dirtree code, and
				1214	thanks to LP64 it's large enough to store a typecast pointer to an
				1215	arbitrary struct.</p>
				1216
				1217	<p>The return value of the callback combines flags (with boolean or) to tell
				1218	the traversal infrastructure how to behave:</p>
				1219
				1220	<ul>
				1221	<li><p><b>DIRTREE_SAVE</b> - Save this node, assembling a tree. (Without
				1222	this the struct dirtree is freed after the callback returns. Filtering out
				1223	siblings is fine, but discarding a parent while keeping its child leaks
				1224	memory.)</p></li>
				1225	<li><p><b>DIRTREE_ABORT</b> - Do not examine any more entries in this
				1226	directory. (Does not propagate up tree: to abort entire traversal,
				1227	return DIRTREE_ABORT from parent callbacks too.)</p></li>
				1228	<li><p><b>DIRTREE_RECURSE</b> - Examine directory contents. Ignored for
				1229	non-directory entries. The remaining flags only take effect when
				1230	recursing into the children of a directory.</p></li>
				1231	<li><p><b>DIRTREE_COMEAGAIN</b> - Call the callback a second time after
				1232	examining all directory contents, allowing depth-first traversal.
				1233	On the second call, dirtree->data = -1.</p></li>
				1234	<li><p><b>DIRTREE_SYMFOLLOW</b> - follow symlinks when populating children's
				1235	<b>struct stat st</b> (by feeding a nonzero value to the symfollow argument of
				1236	dirtree_add_node()), which means DIRTREE_RECURSE treats symlinks to
				1237	directories as directories. (Avoiding infinite recursion is the callback's
				1238	problem: the non-NULL dirtree->symlink can still distinguish between
				1239	them.)</p></li>
				1240	</ul>
				1241
				1242	<p>Each struct dirtree contains three pointers (next, parent, and child)
				1243	to other struct dirtree.</p>
				1244
				1245	<p>The <b>parent</b> pointer indicates the directory
				1246	containing this entry; even when not assembling a persistent tree of
				1247	nodes the parent entries remain live up to the root of the tree while
				1248	child nodes are active. At the top of the tree the parent pointer is
				1249	NULL, meaning the node's name[] is either an absolute path or relative
				1250	to cwd. The function dirtree_parentfd() gets the directory file descriptor
				1251	for use with openat() and friends, returning AT_FDCWD at the top of tree.</p>
				1252
				1253	<p>The <b>child</b> pointer points to the first node of the list of contents of
				1254	this directory. If the directory contains no files, or the entry isn't
				1255	a directory, child is NULL.</p>
				1256
				1257	<p>The <b>next</b> pointer indicates sibling nodes in the same directory as this
				1258	node, and since it's the first entry in the struct the llist.c traversal
				1259	mechanisms work to iterate over sibling nodes. Each dirtree node is a
				1260	single malloc() (even char *symlink points to memory at the end of the node),
				1261	so llist_free() works but its callback must descend into child nodes (freeing
				1262	a tree, not just a linked list), plus whatever the user stored in extra.</p>
				1263
				1264	<p>The <b>dirtree_read</b>() function is a simple wrapper, calling <b>dirtree_add_node</b>()
				1265	to create a root node relative to the current directory, then calling
				1266	<b>handle_callback</b>() on that node (which recurses as instructed by the callback
				1267	return flags). Some commands (such as chgrp) bypass this wrapper, for example
				1268	to control whether or not to follow symlinks to the root node; symlinks
				1269	listed on the command line are often treated differently than symlinks
				1270	encountered during recursive directory traversal).
				1271
				1272	<p>The ls command not only bypasses the wrapper, but never returns
				1273	<b>DIRTREE_RECURSE</b> from the callback, instead calling <b>dirtree_recurse</b>() manually
				1274	from elsewhere in the program. This gives ls -lR manual control
				1275	of traversal order, which is neither depth first nor breadth first but
				1276	instead a sort of FIFO order requried by the ls standard.</p>
				1277
Rob Landley	7eaf4f5	2014-04-09 08:30:09 -0500	[diff] [blame]	1278	<a name="toys">
				1279	<h1><a href="#toys">Directory toys/</a></h1>
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	1280
				1281	<p>This directory contains command implementations. Each command is a single
				1282	self-contained file. Adding a new command involves adding a single
				1283	file, and removing a command involves removing that file. Commands use
				1284	shared infrastructure from the lib/ and generated/ directories.</p>
				1285
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	1286	<p>Currently there are five subdirectories under "toys/" containing "posix"
				1287	commands described in POSIX-2008, "lsb" commands described in the Linux
				1288	Standard Base 4.1, "other" commands not described by either standard,
				1289	"pending" commands awaiting cleanup (which default to "n" in menuconfig
				1290	because they don't necessarily work right yet), and "example" code showing
				1291	how toybox infrastructure works and providing template/skeleton files to
				1292	start new commands.</p>
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	1293
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	1294	<p>The only difference directory location makes is which menu the command
				1295	shows up in during "make menuconfig", the directories are otherwise identical.
				1296	Note that the commands exist within a single namespace at runtime, so you can't
				1297	have the same command in multiple subdirectories. (The build tries to fail
				1298	informatively when you do that.)</p>
				1299
				1300	<p>There is one more sub-menus in "make menuconfig" containing global
				1301	configuration options for toybox. This menu is defined in the top level
				1302	Config.in.</p>
Rob Landley	c0e56ed	2012-10-08 00:02:30 -0500	[diff] [blame]	1303
				1304	<p>See <a href="#adding">adding a new command</a> for details on the
				1305	layout of a command file.</p>
				1306
Rob Landley	81b899d	2007-12-18 02:02:47 -0600	[diff] [blame]	1307	<h2>Directory scripts/</h2>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	1308
Rob Landley	1f4f41a	2012-10-08 21:31:07 -0500	[diff] [blame]	1309	<p>Build infrastructure. The makefile calls scripts/make.sh for "make"
				1310	and scripts/install.sh for "make install".</p>
				1311
				1312	<p>There's also a test suite, "make test" calls make/test.sh, which runs all
				1313	the tests in make/test/*. You can run individual tests via
				1314	"scripts/test.sh command", or "TEST_HOST=1 scripts/test.sh command" to run
				1315	that test against the host implementation instead of the toybox one.</p>
				1316
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	1317	<h3>scripts/cfg2files.sh</h3>
				1318
				1319	<p>Run .config through this filter to get a list of enabled commands, which
				1320	is turned into a list of files in toys via a sed invocation in the top level
				1321	Makefile.
				1322	</p>
				1323
Rob Landley	81b899d	2007-12-18 02:02:47 -0600	[diff] [blame]	1324	<h2>Directory kconfig/</h2>
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	1325
				1326	<p>Menuconfig infrastructure copied from the Linux kernel. See the
				1327	Linux kernel's Documentation/kbuild/kconfig-language.txt</p>
				1328
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	1329	<!-- todo
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1330
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	1331	Better OLDTOY and multiple command explanation. From Config.in:
Rob Landley	66a69d9	2012-01-16 01:44:17 -0600	[diff] [blame]	1332
Rob Landley	5237015	2014-09-20 14:11:59 -0500	[diff] [blame^]	1333	<p>A command with multiple names (or multiple similar commands implemented in
				1334	the same .c file) should have config symbols prefixed with the name of their
				1335	C file. I.E. config symbol prefixes are NEWTOY() names. If OLDTOY() names
				1336	have config symbols they must be options (symbols with an underscore and
				1337	suffix) to the NEWTOY() name. (See generated/toylist.h)</p>
				1338	-->
Rob Landley	ca73392	2014-05-19 18:24:35 -0500	[diff] [blame]	1339
Rob Landley	4e68de1	2007-12-13 07:00:27 -0600	[diff] [blame]	1340	<!--#include file="footer.html" -->