Blame - www/roadmap.html - fp2-dev/platform/external/toybox

blob: ba128f211e05af57a1d93f3a54bb7424eef270d5 [file] [log] [blame]

Rob Landley	349ff52	2014-01-04 13:09:42 -0600	[diff] [blame]	1	<html><head><title>toybox roadmap</title>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	2	<!--#include file="header.html" -->
				3	<title>Toybox Roadmap</title>
				4
				5	<h2>Goals and use cases</h2>
				6
				7	<p>We have several potential use cases for a new set of command line
				8	utilities, and are using those to determine which commands to implement
				9	for Toybox's 1.0 release.</p>
				10
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	11	<p>The most interesting standards are POSIX-2008 (also known as the Single
				12	Unix Specification version 4) and the Linux Standard Base (version 4.1).
				13	The main test harness including toybox in Aboriginal Linux and if that can
				14	build itself using the result to build Linux From Scratch (version 6.8).
				15	We also aim to replace Android's Toolbox.</p>
				16
				17	<p>At a secondary level we'd like to meet other use cases. We've analyzed
				18	the commands provided by similar projects (klibc, sash, sbase, s6, embutils,
				19	nash, and beastiebox), along with various vendor configurations of busybox,
				20	and some end user requests.</p>
				21
				22	<p>Finally, we'd like to provide a good replacement for the Bash shell,
				23	which was the first program Linux ever ran and remains the standard shell
				24	of Linux no matter what Ubuntu says. This doesn't mean including the full
				25	set of Bash 4.x functionality, but does involve {various,features} beyond
				26	posix.</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	27
				28	<p>See the <a href=status.html>status page</a> for the combined list
				29	and progress towards implementing it.</p>
				30
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	31	<ul>
				32	<li><a href=#susv4>POSIX-2008/SUSv4</a></li>
				33	<li><a href=#sigh>Linux "Standard" Base</a></li>
				34	<li><a href=#dev_env>Development Environment</a></li>
				35	<li><a href=#android>Android Toolbox</a></li>
Rob Landley	348a800	2014-04-09 07:57:08 -0500	[diff] [blame]	36	<li>Miscelaneous: <a href=#klibc>klibc</a>, <a href=#glibc>glibc</a>,
				37	<a href=#sash>sash</a>, <a href=#sbase>sbase</a>, <a href=#s6>s6</a>...</li>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	38	</ul>
				39
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	40	<hr />
				41	<a name="standards">
				42	<h2>Use case: standards compliance.</h2>
				43
Rob Landley	571b070	2012-11-13 16:13:45 -0600	[diff] [blame]	44	<h3><a name=susv4 /><a href="#susv4">POSIX-2008/SUSv4</a></h3>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	45	<p>The best standards are the kind that describe reality, rather than
				46	attempting to impose a new one. (I.E. a good standard should document, not
				47	legislate.)</p>
				48
				49	<p>The kind of standards which describe existing reality tend to be approved by
				50	more than one standards body, such ANSI and ISO both approving C. That's why
				51	the IEEE POSIX committee's 2008 standard, the Single Unix Specification version
				52	4, and the Open Group Base Specification edition 7 are all the same standard
				53	from three sources.</p>
				54
Rob Landley	62f0021	2012-12-06 15:15:30 -0600	[diff] [blame]	55	<p>The <a href="http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html">"utilities"
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	56	section</a>
				57	of these standards is devoted to the unix command line, and are the best such
				58	standard for our purposes. (My earlier work on BusyBox was implemented with
				59	regard to SUSv3, an earlier version of this standard.)</p>
				60
				61	<h3>Problems with the standard</h3>
				62
				63	<p>Unfortunately, these standards describe a subset of reality, lacking any
				64	mention of commands such as init, login, or mount required to actually boot a
				65	system. It provides ipcrm and ipcs, but not ipcmk, so you can use System V IPC
				66	resources but not create them.</p>
				67
				68	<p>These standards also contain a large number of commands that are
				69	inappropriate for toybox to implement in its 1.0 release. (Perhaps some of
				70	these could be reintroduced in later releases, but not now.)</p>
				71
				72	<p>Starting with the full "utilities" list, we first remove generally obsolete
				73	commands (compess ed ex pr uncompress uccp uustat uux), commands for the
				74	pre-CVS "SCCS" source control system (admin delta get prs rmdel sact sccs unget
				75	val what), fortran support (asa fort77), and batch processing support (batch
				76	qalter qdel qhold qmove qmsg qrerun qrls qselect qsig qstat qsub).</p>
				77
				78	<p>Some commands are for a compiler toolchain (ar c99 cflow ctags cxref gencat
				79	iconv lex m4 make nm strings strip tsort yacc), which is outside of toybox's
				80	mandate and should be supplied externally. (Again, some of these may be
				81	revisited later, but not for toybox 1.0.)</p>
				82
				83	<p>Some commands are part of a command shell, and cannot be implemented as
				84	separate executables (alias bg cd command fc fg getopts hash jobs kill read
				85	type ulimit umask unalias wait). These may be revisited as part of a built-in
				86	toybox shell, but are not exported into $PATH via symlinks. (If you fork a
				87	child process and have it "cd" then exit, you've accomplished nothing.)</p>
				88
				89	<p>A few other commands are judgement calls, providing command-line
				90	internationalization support (iconv locale localedef), System V inter-process
				91	communication (ipcrm ipcs), and cross-tty communication from the minicomputer
				92	days (talk mesg write). The "pax" utility was supplanted by tar, "mailx" is
				93	a command line email client, and "lp" submits files for printing to... what
				94	exactly? (cups?) The standard defines crontab but not crond.</p>
				95
				96	<p>Removing all of that leaves the following commands, which toybox should
				97	implement:</p>
				98
				99	<blockquote><b>
				100	<span id=posix>
				101	at awk basename bc cal cat chgrp chmod chown cksum cmp comm cp
				102	csplit cut date dd df diff dirname du echo env expand expr false file find
				103	fold fuser getconf grep head id join kill link ln logger logname ls man
				104	mkdir mkfifo more mv newgrp nice nl nohup od paste patch pathchk printf ps
				105	pwd renice rm rmdir sed sh sleep sort split stty tabs tail tee test time
				106	touch tput tr true tty uname unexpand uniq unlink uudecode uuencode vi wc
				107	who xargs zcat
				108	</span>
				109	</b></blockquote>
				110
Rob Landley	571b070	2012-11-13 16:13:45 -0600	[diff] [blame]	111	<h3><a name=sigh /><a href="#sigh">Linux Standard Base</a></h3>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	112
				113	<p>One attempt to supplement POSIX towards an actual usable system was the
				114	Linux Standard Base. Unfortunately, the quality of this "standard" is
				115	fairly low.</p>
				116
				117	<p>POSIX allowed its standards process to be compromised
				118	by leaving things out, thus allowing IBM mainframes and Windows NT to drive
				119	a truck through the holes and declare themselves compilant. But it means what
				120	they DID standardize tends to be respected.</p>
				121
				122	<p>The Linux Standard Base's failure mode is different, they respond to
				123	pressure by including special-case crap, such as allowing Red Hat to shoehorn
Rob Landley	62f0021	2012-12-06 15:15:30 -0600	[diff] [blame]	124	RPM on the standard even though all sorts of distros (Debian, Slackware, Arch,
				125	Gentoo) don't use it and probably never will. This means anything in the LSB is
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	126	at best a suggestion: arbitrary portions of this standard are widely
				127	ignored.</p>
				128
				129	<p>The LSB does specify a <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/cmdbehav.html>list of command line
				130	utilities</a>:</p>
				131
				132	<blockquote><b>
				133	ar at awk batch bc chfn chsh col cpio crontab df dmesg du echo egrep
				134	fgrep file fuser gettext grep groupadd groupdel groupmod groups
				135	gunzip gzip hostname install install_initd ipcrm ipcs killall lpr ls
				136	lsb_release m4 md5sum mknod mktemp more mount msgfmt newgrp od passwd
				137	patch pidof remove_initd renice sed sendmail seq sh shutdown su sync
				138	tar umount useradd userdel usermod xargs zcat
				139	</b></blockquote>
				140
				141	<p>Where posix specifies one of those commands, LSB's deltas tend to be
				142	accomodations for broken tool versions which aren't up to date with the
				143	standard yet. (See <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/more.html>more</a> and <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/xargs.html>xargs</a>
				144	for examples.)</p>
				145
				146	<p>Since we've already committed to using our own judgement to skip bits of
				147	POSIX, and LSB's "judgement" in this regard is purely bug workarounds to declare
				148	various legacy tool implementations "compliant", this means we're mostly
				149	interested in the set of tools that aren't specified in posix at all.</p>
				150
				151	<p>Of these, gettext and msgfmt are internationalization, install_initd and
				152	remove_initd aren't present on ubuntu 10.04, lpr is out of scope, and
				153	lsb_release is a distro issue (it's a nice command, but the output of
				154	lsb_release -a is the name and version number of the linux distro you're
				155	running, which toybox doesn't know).</p>
				156
				157	<p>This leaves:</p>
				158
				159	<blockquote><b>
				160	<span id=lsb>
				161	chfn chsh dmesg egrep fgrep groupadd groupdel groupmod groups
				162	gunzip gzip hostname install killall md5sum
				163	mknod mktemp mount passwd pidof sendmail seq shutdown
				164	su sync tar umount useradd userdel usermod zcat
				165	</span>
				166	</b></blockquote>
				167
				168	<hr />
				169	<a name="dev_env">
Rob Landley	571b070	2012-11-13 16:13:45 -0600	[diff] [blame]	170	<h2><a href="#dev_env">Use case: provide a self-hosting development environment</a></h2>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	171
				172	<p>The following commands are enough to build the Aboriginal Linux development
				173	environment, boot it to a shell prompt, and build Linux From Scratch 6.8 under
				174	it. (Aboriginal Linux currently uses BusyBox for this, thus provides a
				175	drop-in test environment for toybox. We install both implementations side
				176	by side, redirecting the symlinks a command at a time until the older
				177	package is no longer used, and can be removed.)</p>
				178
				179	<p>This use case includes running init scripts and other shell scripts, running
				180	configure, make, and install in each package, and providing basic command line
				181	facilities such as a text editor. (It does not include a compiler toolchain or
				182	C library, those are outside the scope of this project.)</p>
				183
				184	<blockquote><b>
				185	<span id=development>
				186	bzcat cat cp dirname echo env patch rmdir sha1sum sleep sort sync
				187	true uname wc which yes zcat
				188	awk basename bzip2 chmod chown cmp cut date dd diff
				189	egrep expr find grep gzip head hostname id install ln ls
				190	mkdir mktemp mv od readlink rm sed sh tail tar touch tr uniq
				191	wget whoami xargs chgrp comm gunzip less logname man split
				192	tee test time bunzip2 chgrp chroot comm cpio dmesg
				193	dnsdomainname ftpd ftpget ftpput gunzip ifconfig init less
				194	logname losetup man mdev mount mountpoint nc pgrep pkill
				195	pwd route split stat switch_root tac umount vi
				196	</span>
				197	</b></blockquote>
				198
				199	<p>Note: Aboriginal Linux installs bash 2.05b as #!/bin/sh and its scripts
				200	require bash extensions not present in shells such as busybox ash.
				201	This means that toysh needs to supply several bash extensions _and_ work
				202	when called under the name "bash".</p>
				203
Rob Landley	8fe1814	2014-10-07 14:11:54 -0500	[diff] [blame]	204	<p>The <a href=http://landley.net/aboriginal>Aboriginal Linux</a>
				205	self-bootstrapping build still uses the following busybox commands,
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	206	not yet supplied by toybox:</p>
Rob Landley	8fe1814	2014-10-07 14:11:54 -0500	[diff] [blame]	207
				208	<blockquote><p>
				209	ash awk bunzip2 bzip2 dd diff expr fdisk ftpd ftpget ftpput gunzip
				210	gzip less man pgrep ping pkill ps route sed sh sha512sum tar test tr unxz vi
				211	wget xzcat zcat</p></blockquote>
				212
				213	<p>Many of those are in "pending". Most of the archive commands are needed
				214	because busybox tar doesn't call external versions. The remaining "difficult"
				215	commands are vi, awk, and ash.</p>
				216
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	217	<hr />
Rob Landley	571b070	2012-11-13 16:13:45 -0600	[diff] [blame]	218	<h2><a name=android /><a href="#android">Use case: Replacing Android Toolbox</a></h2>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	219
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	220	<p>Android has a policy against GPL in userspace, so even though BusyBox
				221	predates Android by many years, they couldn't use it. Instead they grabbed
				222	an old version of ash and implemented their own command line utility set
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	223	called "toolbox". ash was later replaced by
				224	<a href="https://www.mirbsd.org/mksh.htm">mksh</a>; toolbox is being
				225	replaced by toybox.</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	226
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	227	<p>Toolbox doesn't have its own repository, instead it's part of Android's
				228	<a href=https://android.googlesource.com/platform/system/core>system/core
				229	git repository</a> (this analysis looked at commit 51ccef27cab58).</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	230
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	231	<h3>Toolbox commands:</h3>
				232
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	233	<p>According to system/core/toolbox/Android.mk the toolbox directory builds
				234	the following commands:</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	235
				236	<blockquote><b>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	237	ls mount cat ps kill ln insmod rmmod lsmod ifconfig setconsole
				238	rm mkdir rmdir reboot getevent sendevent date wipe sync umount
				239	start stop notify cmp dmesg route hd dd df getprop setprop watchprops
				240	log sleep renice printenv smd chmod chown newfs_msdos netstat ioctl
				241	mv schedtop top iftop id uptime vmstat nandread ionice touch lsof md5 r
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	242	cp du grep watchdogd
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	243	</b></blockquote>
				244
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	245	<p>If selinux is enabled, you also get:</p>
				246	<blockquote><b>
				247	getenforce setenforce chcon restorecon runcon getsebool setsebool load_policy
				248	</b></blockquote>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	249
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	250	<h3>Other Android core commands</h3>
				251
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	252	<p>Other than the toolbox directory, the currently interesting
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	253	subdirectories in the core repository are fs_mgr, gpttool, init,
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	254	logcat, logwrapper, mkbootimg, netcfg, reboot, and run-as.</p>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	255
				256	<ul>
				257	<li><b>fs_mgr</b> - subset of mount</li>
				258	<li><b>gpttool</b> - subset of fdisk</li>
				259	<li><b>init</b> - Android's PID 1</li>
				260	<li><b>logcat</b> - read android log format</li>
				261	<li><b>logwrapper</b> - redirect stdio to android log</li>
				262	<li><b>mkbootimg</b> - create signed boot image</li>
				263	<li><b>netcfg</b> - network configuration (sucks in libnetutils)</li>
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	264	<li><b>reboot</b> - Android's reboot(1)</li>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	265	<li><b>run-as</b> - subset of sudo</li>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	266	</ul>
				267
				268	<p>Almost all of these reinvent an existing wheel with less functionality and a
				269	different user interface. We may want to provide that interface, but
				270	implementing the full commands (mount, fdisk, init, ifconfig with dhcp,
				271	and sudo) come first.</p>
				272
				273	<p>Although logcat/logwrapper also reinvent a wheel, Android did so in the
				274	kernel and these provide an interface to that.</p>
				275
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	276	<p>Also, gpttool and mkbootimg are install tools.
				277	These aren't a priority if android wants to use its own
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	278	bespoke code to install itself.</p>
				279
				280	<h3>Analysis</h3>
				281
				282	<p>For reference, combining everything listed above, we get:</p>
				283
				284	<blockquote><b>
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	285	cat chcon chmod chown cmp cp date dd df dmesg du fs_mgr
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	286	getenforce
				287	getevent getprop getsebool gpttool grep hd id ifconfig iftop init insmod ioctl
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	288	ionice kill ln load_policy log logcat logwrapper ls lsmod lsof md5
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	289	mkbootimg mkdir mount mv nandread netcfg netstat newfs_msdos notify printenv
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	290	ps r reboot renice restorecon rm rmdir rmmod route run-as
				291	runcon schedtop sendevent setconsole setenforce setprop setsebool
				292	sleep smd start stop sync top touch umount uptime vmstat watchdogd
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	293	watchprops wipe
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	294	</b></blockquote>
				295
				296	<p>We may eventually implement all of that, but for toybox 1.0 we need to
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame^]	297	focus a bit. For our first pass, let's ignore selinux,
				298	and grab just logcat and logwrapper from the "core"
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	299	commands (since the rest have some full/standard version providing that
				300	functionality, which we can implement a shim interface for later).</p>
				301
				302	<p>This means toybox should implement:</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	303	<blockquote><b>
				304	<span id=toolbox>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	305	cat chmod chown cmp cp date dd df dmesg du getevent getprop grep hd id ifconfig
				306	iftop insmod ioctl ionice kill ln log logcat logwrapper ls lsmod lsof lsusb md5
				307	mkdir mount mv nandread
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	308	netstat newfs_msdos notify printenv ps r reboot renice rm rmdir rmmod route
				309	schedtop sendevent setconsole setprop sleep smd start stop sync top touch
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	310	umount uptime vmstat watchprops watchdogd wipe
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	311	</span>
Rob Landley	c26ca6e	2013-01-31 04:05:56 -0600	[diff] [blame]	312	</b></blockquote>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	313
				314	<p>The following Toolbox commands are already covered in previous
				315	sections of this analysis:</p>
				316
				317	<blockquote><b>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	318	cat chmod chown cmp cp date dd df dmesg du grep id ifconfig insmod kill ln ls
				319	lsmod mkdir mount mv ps renice rm rmdir rmmod route sleep sync top touch umount
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	320	</b></blockquote>
				321
				322	<p>Which leaves the following commands as new from Toolbox:</p>
				323
				324	<blockquote><b>
				325	getevent getprop hd iftop ioctl ionice log lsof nandread netstat
				326	newfs_msdos notify printenv r reboot schedtop sendevent setconsole
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	327	setprop smd start stop top uptime vmstat watchprops watchdogd wipe
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	328	</b></blockquote>
				329
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	330	<hr /><a name=klibc />
Rob Landley	934b2d3	2013-05-10 18:54:14 -0500	[diff] [blame]	331	<h2>klibc:</h2>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	332
				333	<p>Long ago some kernel developers came up with a project called
				334	<a href=http://en.wikipedia.org/wiki/Klibc>klibc</a>.
				335	After a decade of development it still has no web page or HOWTO,
				336	and nobody's quite sure if the license is BSD or GPL. It inexplicably
				337	<a href=http://www.infoworld.com/d/data-center/perl-isnt-going-anywhere-better-or-worse-211580>requires perl to build</a>, and seems like an ideal candidate for
				338	replacement.</p>
				339
				340	<p>In addition to a C library even less capable than bionic (obsoleted by
				341	musl), klibc builds a random assortment of executables to run init scripts
				342	with. There's no multiplexer command, these are individual executables:</p>
				343
				344	<blockquote><p>
				345	cat chroot cpio dd dmesg false fixdep fstype gunzip gzip halt ipconfig kill
				346	kinit ln losetup ls minips mkdir mkfifo mknodes
				347	mksyntax mount mv nfsmount nuke pivot_root poweroff readlink reboot resume
				348	run-init sh sha1hash sleep sync true umount uname zcat
				349	</p></blockquote>
				350
				351	<p>To get that list, build klibc according to the instructions (I
				352	<a href=http://landley.net/notes-2013.html#23-01-2013>looked at</a> version
				353	2.0.2 and did cd klibc-*; ln -s /output/of/kernel/make/headers_install
				354	linux; make) then <b>echo $(for i in $(find . -type f); do file $i \| grep -q
				355	executable && basename $i; done \| grep -v '[.]g$' \| sort -u)</b> to find
				356	executables, then eliminated the .so files and .shared duplicates.</p>
				357
				358	<p>Some of those binaries are build-time tools that don't get installed,
				359	which removes mknodes, mksyntax, sha1hash, and fixdep from the list.
				360	(And sha1hash is just an unpolished sha1sum anyway.)</p>
				361
				362	<p>The run-init command is more commonly called switch_root, nuke is just
				363	"rm -rf -- $@", and minips is more commonly called "ps". I'm not doing aliases
				364	for the oddball names.</p>
				365
				366	<p>Yet more stale forks of dash and gzip sucked in here (see "dubious
				367	license terms" above), adding nothing to the other projects we've looked at.
				368	But we still need sh, gunzip, gzip, and zcat to replace this package.</p>
				369
				370	<p>By the time I did the analysis toybox already had cat, chroot, dmesg, false,
				371	kill, ln, losetup, ls, mkdir, mkfifo, readlink, rm, switch_root, sleep, sync,
				372	true, and uname.</p>
				373
				374	<p>The low hanging fruit is cpio, dd, ps, mv, and pivot_root.</p>
				375
				376	<p>The "kinit" command is another gratuitous rename, it's init running as PID 1.
				377	The halt, poweroff, and reboot commands work with it.</p>
				378
				379	<p>I've got mount and umount queued up already, fstype and nfsmount go with
				380	those. (And probably smbmount and p9mount, but this hasn't got one. Those
				381	are all about querying for login credentials, probably workable into the
				382	base mount command.)</p>
				383
				384	<p>The ipconfig command here has a built in dhcp client, so it's ifconfig
				385	and dhcpcd and maybe some other stuff.</p>
				386
				387	<p>The resume command is... weird. It finds a swap partition and reads data
				388	from it into a /proc file, something the kernel is capable of doing itself.
				389	(Even though the klibc author
				390	<a href=http://www.zytor.com/pipermail/klibc/2006-June/001748.html>attempted
				391	to remove</a> that capability from the kernel, current kernel/power/hibernate.c
				392	still parses "resume=" on the command line). And yet various distros seem to
				393	make use of klibc for this>
				394	Given the history of swsusp/hibernate (and
				395	<a href=http://lwn.net/Articles/333007>TuxOnIce</a>
				396	and <a href=http://lwn.net/Articles/242107>kexec jump</a>) I've lost track
				397	of the current state of the art here. Ah, Documentation/power/userland-swsusp.txt
				398	has the API docs, and <a href=http://suspend.sf.net>here's a better
				399	tool</a>...</p>
				400
				401	<p>So the list of things actually in klibc are:</p>
				402
				403	<blockquote><b>
Rob Landley	c166faf	2013-09-01 07:25:37 -0500	[diff] [blame]	404	<span id=klibc_cmd>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	405	cat chroot dmesg false kill ln losetup ls mkdir mkfifo readlink rm switch_root
				406	sleep sync true uname
				407
				408	cpio dd ps mv pivot_root
				409	mount nfsmount fstype umount
				410	sh gunzip gzip zcat
				411	kinit halt poweroff reboot
				412	ipconfig
				413	resume
				414	</span>
				415	</b></blockquote>
				416
				417	<hr />
Rob Landley	348a800	2014-04-09 07:57:08 -0500	[diff] [blame]	418	<a name=glibc />
				419	<h2>glibc</h2>
				420
				421	<p>Rather a lot of command line utilities come bundled with glibc:</p>
				422
				423	<blockquote><b>
				424	catchsegv getconf getent iconv iconvconfig ldconfig ldd locale localedef
				425	mtrace nscd rpcent rpcinfo tzselect zdump zic
				426	</b></blockquote>
				427
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	428	<p>Of those, musl libc only implements ldd.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	429
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	430	<p>catchsegv is a rudimentary debugger, probably out of scope for toybox.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	431
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	432	<p>iconv has been <a href="#susv4">previously discussed</a>.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	433
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	434	<p>iconvconfig is only relevant if iconv is user-configurable; musl uses a
				435	non-configurable iconv.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	436
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	437	<p>getconf is a posix utility which displays several variables from
				438	unistd.h; it probably belongs in the development toolchain.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	439
				440	<p>getent handles retrieving entries from passwd-style databases
				441	(in a rather lame way) and is trivially replacable by grep.</p>
				442
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	443	<p>locale was discussed under <a href=#susv4>posix</a>.
				444	localedef compiles locale definitions, which musl currently does not use.</p>
				445
				446	<p>mtrace is a perl script to use the malloc debugging that glibc has built-in;
				447	this is not relevant for musl, and would necessarily vary with libc. </p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	448
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	449	<p>nscd is a name service caching daemon, which is not yet relevant for musl.
				450	rpcinfo and rpcent are related to rpc, which musl does not include.</p>
				451
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	452	<p>The remaining commands involve glibc's bundled timezone database,
				453	which seems to be derived from the <a href=http://www.iana.org/time-zones>IANA
				454	timezone database</a>. Unless we want to maintain our own fork of the
				455	standards body's database like glibc does, these are of no interest,
				456	but for completeness:</p>
				457
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	458	<p>tzselect outputs a TZ variable correponding to user input.
				459	The documentation does not indicate how to use it in a script, but it seems
				460	that Debian may have done so.
				461	zdump prints current time in each of several timezones, optionally
				462	outputting a great deal of extra information about each timezone.
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	463	zic converts a description of a timezone to a file in tz format.</p>
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	464
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	465	<p>None of glibc's bundled commands are currently of interest to toybox.</p>
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	466
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	467	</b></blockquote>
Rob Landley	348a800	2014-04-09 07:57:08 -0500	[diff] [blame]	468
				469	<hr />
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	470	<a name=sash />
				471	<h2>Stand-Alone Shell</h2>
				472
				473	<p>Wikipedia has <a href=http://en.wikipedia.org/wiki/Stand-alone_shell>a good
				474	summary of sash</a>, with links. The original Stand-Alone Shell project reached
				475	a stopping point, and then <a href=http://www.baiti.net/sash>"sash plus
				476	patches"</a> extended it a bit further. The result is a megabyte executable
				477	that provides 40 commands.</p>
				478
				479	<p>Sash is a shell with built-in commands. It doesn't have a multiplexer
				480	command, meaning "sash ls -l" doesn't work (you have to go "sash -c 'ls -l'").
				481	</p>
				482
				483	<p>The list of commands can be obtained via building it and doing
				484	"echo help \| ./sash \| awk '{print $1}' \| sed 's/^-//' \| xargs echo", which
				485	gives us:</p>
				486
				487	<blockquote><b>
				488	alias aliasall ar cd chattr chgrp chmod chown cmp cp chroot dd echo ed exec
				489	exit file find grep gunzip gzip help kill losetup losetup ln ls lsattr mkdir
				490	mknod more mount mv pivot_root printenv prompt pwd quit rm rmdir setenv source
				491	sum sync tar touch umask umount unalias where
				492	</b></blockquote>
				493
				494	<p>Plus sh because it's a shell. A dozen or so commands can only sanely be
				495	implemented as shell builtins (alias aliasall cd exec exit prompt quit setenv
				496	source umask unalias), where is an alias for which, and at triage time toybox
				497	already has chgrp, chmod, chown, cmp, cp, chroot, echo, help, kill, losetup,
				498	ln, ls, mkdir, mknod, printenv, pwd, rm, rmdir, sync, and touch.</p>
				499
				500	<p>This leaves:</p>
				501
				502	<blockquote><b>
				503	<span id=sash_cmd>
				504	ar chattr dd ed file find grep gunzip gzip lsattr more mount mv pivot_root
				505	sh sum tar umount
				506	</span>
				507	</b></blockquote>
				508
				509	<p>(For once, this project doesn't include a fork of gzip, instead
				510	it sucks in -lz from the host.)</p>
				511
				512	<hr />
				513	<a name=sbase />
				514	<h2>sbase:</h2>
				515
				516	<p>It's <a href=http://git.suckless.org/sbase>on suckless</a>. So far it's
				517	implemented:</p>
				518
				519	<blockquote><p>
Rob Landley	c166faf	2013-09-01 07:25:37 -0500	[diff] [blame]	520	<span id=sbase_cmd>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	521	basename cat chmod chown cksum cmp cp date dirname echo false fold grep head
				522	kill ln ls mc mkdir mkfifo mv nl nohup pwd rm seq sleep sort tail tee test
				523	touch true tty uname uniq wc yes
				524	</span>
				525	</p></blockquote>
				526
				527	<p>And has a TODO list:</p>
				528
				529	<blockquote><p>
Rob Landley	c166faf	2013-09-01 07:25:37 -0500	[diff] [blame]	530	<span id=sbase_cmd>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	531	cal chgrp chvt comm cut df diff du env expand expr id md5sum nice paste
				532	printenv printf readlink rmdir seq sha1sum split sync test tr unexpand unlink
				533	who
				534	</span>
				535	</p></blockquote>
				536
				537	<p>At triage time, of the first list I still need to do: fold grep mc mv nl. Of
				538	the second list: diff expr paste printf split test tr unexpand who.</p>
				539
				540	<hr />
				541	<a name=s6 />
				542	<h2>s6</h2>
				543
				544	<p>The website <a href=http://skarnet.org/software/>skarnet</a> has a bunch
				545	of small utilities as part of something called "s6". This includes the
				546	<a href=http://skarnet.org/software/s6-portable-utils>s6-portabile-utils</a>
				547	and the <a href=http://skarnet.org/software/s6-linux-utils>s6-linux-utils</a>.
				548	</p>
				549
				550	<p>Both packages rely on multiple bespoke external libraries without which
				551	they can't compile. The source is completely uncommented and doesn't wrap at
				552	80 characters. Doing a find for *.c files brings up the following commands:</p>
				553
				554	<blockquote><b>
				555	<span id=s6>
				556	basename cat chmod chown chroot clock cut devd dirname echo env expr false
				557	format-filter freeramdisk grep halt head hiercopy hostname linkname ln
				558	logwatch ls maximumtime memoryhog mkdir mkfifo mount nice nuke pause
				559	pivotchroot poweroff printenv quote quote-filter reboot rename rmrf sleep
				560	sort swapoff swapon sync tail test touch true umount uniquename unquote
				561	unquote-filter update-symlinks
				562	</span>
				563	</b></blockquote>
				564
				565	<p>Triage: memoryhog isn't even listed on the website nor does it have
				566	a documentation file, clock seems like a subset
				567	of date, devd is some sort of netlink wrapper that spawns its command line
				568	every time it gets a message (maybe this is meant to implement part of
				569	udev/mdev?), format-filter is sort of awk's '{print $2}' function split out
				570	into its own command, hiercopy a subset of "cp -r", maximumtime is something
				571	I implemented as a shell script (more/timeout.sh in Aboriginal Linux),
				572	nuke isn't the same as klibc (this one's "kill SIG -1" only with hardwared
				573	SIG options), pause is a program that literally waits to be killed (I
				574	generally sleep 999999999 which is a little over 30 years),
				575	pivotchroot is a subset of switch_root, rmrf is rm -rf...</p>
				576
				577	<p>I see "nuke" resurface, and if "rmrf" wasn't also here I might think
				578	klibc had a point.</b>
				579
				580	<blockquote>
				581	basename cat chmod chown chroot cut dirname echo env expr false
				582	freeramdisk grep halt head hostname linkname ln
				583	logwatch ls mkdir mkfifo mount nice
				584	pivotchroot poweroff printenv quote quote-filter reboot rename sleep
				585	sort swapoff swapon sync tail test touch true umount uniquename unquote
				586	unquote-filter update-symlinks
				587	</blockquote>
				588
				589
				590	<hr />
				591	<a name=nash />
				592	<h2>nash:</h2>
				593
				594	<p>Red Hat's nash was part of its "mkinitrd" package, replacement for a shell
				595	and utilities on the boot floppy back in the 1990's (the same general idea
				596	as BusyBox, developed independently). Red Hat discontinued nash development
				597	in 2010, replacing it with dracut (which collects together existing packages,
				598	including busybox).</p>
				599
				600	<p>I couldn't figure out how to beat source code out of
				601	<a href=http://pkgs.fedoraproject.org/git/mkinitrd>Fedora's current git</a>
				602	repository. The last release version that used it was Fedora Core 12
				603	which has <a href=http://archive.fedoraproject.org/pub/archive/fedora/linux/releases/12/Fedora/source/SRPMS/mkinitrd-6.0.93-1.fc12.src.rpm>a source rpm</a>
				604	that can be unwound with "rpm2cpio mkinitrd.src.rpm \| cpio -i -d -H newc
				605	--no-absolute-filenames" and in there is a mkinitrd-6.0.93.tar.bz2 which
				606	has the source.</p>
				607
				608	<p>In addition to being a bit like a command shell, the nash man page lists the
				609	following commands:</p>
				610
				611	<blockquote><p>
				612	access echo find losetup mkdevices mkdir mknod mkdmnod mkrootdev mount
				613	pivot_root readlink raidautorun setquiet showlabels sleep switchroot umount
				614	</p></blockquote>
				615
				616	<p>Oddly, the only occurrence of the string pivot_root in the nash source code
				617	is in the man page, the command isn't there. (It seems to have been removed
				618	when the underscoreless switchroot went in.)</p>
				619
				620	<p>A more complete list seems to be the handlers[] array in nash.c:</p>
				621
				622	<blockquote><p>
				623	access buildEnv cat cond cp daemonize dm echo exec exit find kernelopt
				624	loadDrivers loadpolicy mkchardevs mkblktab mkblkdevs mkdir mkdmnod mknod
				625	mkrootdev mount netname network null plymouth hotplug killplug losetup
				626	ln ls raidautorun readlink resume resolveDevice rmparts setDeviceEnv
				627	setquiet setuproot showelfinterp showlabels sleep stabilized status switchroot
				628	umount waitdev
				629	</p></blockquote>
				630
				631	<p>This list is nuts: "plymouth" is an alias for "null" which is basically
				632	"true" (which thie above list doesn't have). Things like buildEnv and
				633	loadDrivers are bespoke Red Hat behavior that might as well be hardwired in
				634	to nash's main() without being called.</p>
				635
				636	<p>Instead of eliminating items
				637	from the list with an explanation for each, I'm just going to cherry pick
				638	a few: the device mapper (dm, raidautorun) is probably interesting,
				639	hotplug (may be obsolete due to kernel changes that now load firmware
				640	directly), and another "resume" ala klibc.</p>
				641
				642	<p>But mostly: I don't care about this one. And neither does Red Hat anymore.</p>
				643
Rob Landley	ca04c7f	2013-03-27 22:34:28 -0500	[diff] [blame]	644	<p>Verdict: ignore</p>
				645
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	646	<hr />
				647	<a name=beastiebox />
				648	<h2>Beastiebox</h2>
				649
				650	<p>Back in 2008, the BSD guys vented some busybox-envy
				651	<a href=http://beastiebox.sourceforge.net>on sourceforge</a>. Then stopped.
				652	Their repository is still in CVS, hasn't been touched in years, it's a giant
				653	hairball of existing code sucked together. (The web page says the author
				654	is aware of crunchgen, but decided to do this by hand anyway. This is not
				655	a collection of new code, it's a katamari of existing code rolled up in a
				656	ball.)</p>
				657
				658	<p>Combining the set of commands listed on the web page with the set of
				659	man pages in the source gives us:</P>
				660
				661	<blockquote><p>
				662	[ cat chmod cp csh date df disklabel dmesg echo ex fdisk fsck fsck_ffs getty
				663	halt hostname ifconfig init kill less lesskey ln login ls lv mksh more mount
				664	mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh stty sysctl tar test
				665	traceroute umount vi wiconfig
				666	</p></blockquote>
				667
				668	<p>Apparently lv is the missing link ed and vi, copyright 1982-1997 (do not
				669	want), ex is another obsolete vi mode, lesskey is "used to
				670	specify a set of key bindings to be used with less", and csh is a shell they
				671	sucked in, [ is an alias for test. Several more bsd-isms that don't have Linux
				672	equivalents (even in the ubuntu "install this package" search) are
				673	disklabel, fsck_ffs, mount_ffs, and pfctl. And wiconfig is a wavelan interface
				674	network card driver utility. Subtracting all that and the commands toybox
				675	already implements at triage time, we get:</p>
				676
				677	<blockquote><p>
				678	<span id=beastiebox_cmd>
				679	fdisk fsck getty halt ifconfig init kill less mksh more mount mv ping poweroff
				680	ps reboot route sed sh stty sysctl tar test traceroute umount vi
				681	</span>
				682	</p></blockquote>
				683
				684	<p>Not a hugely interesting list, but eh.</p>
				685
Rob Landley	ca04c7f	2013-03-27 22:34:28 -0500	[diff] [blame]	686	<p>Verdict: ignore</p>
				687
				688	<hr />
				689	<a name=BsdBox />
				690	<h2>BsdBox</h2>
				691
				692	<p>Somebody decided to do a <a href=https://wiki.freebsd.org/AdrianChadd/BsdBox>multicall binary for freebsd</a>.</p>
				693
				694	<p>They based it on crunchgen, a tool that glues existing programs together
				695	into an archive and uses the name to execute the right one. It has no
				696	simplification or code sharing benefits whatsoever, it's basically an
				697	archiver that produces executables.</p>
				698
				699	<p>That's about where I stopped reading.</p>
				700
				701	<p>Verdict: ignore.</p>
				702
				703	<hr />
				704	<a name=slowaris />
				705	<h2>OpenSolaris Busybox</h2>
				706
				707	<p>Somebody <a href=http://hub.opensolaris.org/bin/view/Project+busybox/>wrote
				708	a wiki page</a> saying that Busybox for OpenSolaris would be a good idea.</p>
				709
				710	<p>The corresponding "files" tab is an auto-generated stub. The project never
				711	even got as far as suggesting commands to include before Oracle discontinued
				712	OpenSolaris.</p>
				713
				714	<p>Verdict: ignore.</p>
				715
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	716	<hr />
				717	<h2>Requests:</h2>
				718
Rob Landley	8fe1814	2014-10-07 14:11:54 -0500	[diff] [blame]	719	<p>The following additional commands have been requested (and often submitted)
				720	by various users:</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	721	<blockquote><b>
				722	<span id=request>
Rob Landley	64b6319	2013-12-31 09:01:32 -0600	[diff] [blame]	723	dig freeramdisk getty halt hexdump hwclock klogd modprobe ping ping6 pivot_root
Rob Landley	ca04c7f	2013-03-27 22:34:28 -0500	[diff] [blame]	724	poweroff readahead rev sfdisk sudo syslogd taskset telnet telnetd tracepath
				725	traceroute unzip usleep vconfig zip free login modinfo unshare netcat help w
Rob Landley	64b6319	2013-12-31 09:01:32 -0600	[diff] [blame]	726	ntpd iwconfig iwlist rdate
Rob Landley	7dbb982	2014-02-21 22:24:02 -0600	[diff] [blame]	727	dos2unix unix2dos catv clear
				728	pmap realpath setsid timeout truncate
				729	mkswap swapon swapoff
				730	count oneit fstype
				731	acpi blkid eject pwdx
Rob Landley	8fe1814	2014-10-07 14:11:54 -0500	[diff] [blame]	732	sulogin rfkill bootchartd
				733	arp makedevs sysctl killall5 crond crontab deluser last mkpasswd watch
				734	ipaddr iplink iproute blockdev rpm2cpio arping brctl dumpleases fsck
				735	tcpsvd tftpd
				736	factor fallocate fsfreeze inotifyd lspci nbd-client partprobe strings
				737
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	738	</span>
				739	</b></blockquote>
				740
				741	<!-- #include "footer.html" -->
				742