Blame - www/roadmap.html - fp2-dev/platform/external/toybox

blob: bba08b1b3311e77a301043a18e423a30a6196f64 [file] [log] [blame]

Rob Landley	349ff52	2014-01-04 13:09:42 -0600	[diff] [blame]	1	<html><head><title>toybox roadmap</title>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	2	<!--#include file="header.html" -->
				3	<title>Toybox Roadmap</title>
				4
				5	<h2>Goals and use cases</h2>
				6
				7	<p>We have several potential use cases for a new set of command line
				8	utilities, and are using those to determine which commands to implement
				9	for Toybox's 1.0 release.</p>
				10
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	11	<p>The most interesting standards are POSIX-2008 (also known as the Single
				12	Unix Specification version 4) and the Linux Standard Base (version 4.1).
				13	The main test harness including toybox in Aboriginal Linux and if that can
				14	build itself using the result to build Linux From Scratch (version 6.8).
				15	We also aim to replace Android's Toolbox.</p>
				16
				17	<p>At a secondary level we'd like to meet other use cases. We've analyzed
				18	the commands provided by similar projects (klibc, sash, sbase, s6, embutils,
				19	nash, and beastiebox), along with various vendor configurations of busybox,
				20	and some end user requests.</p>
				21
				22	<p>Finally, we'd like to provide a good replacement for the Bash shell,
				23	which was the first program Linux ever ran and remains the standard shell
				24	of Linux no matter what Ubuntu says. This doesn't mean including the full
				25	set of Bash 4.x functionality, but does involve {various,features} beyond
				26	posix.</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	27
				28	<p>See the <a href=status.html>status page</a> for the combined list
				29	and progress towards implementing it.</p>
				30
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	31	<ul>
				32	<li><a href=#susv4>POSIX-2008/SUSv4</a></li>
				33	<li><a href=#sigh>Linux "Standard" Base</a></li>
				34	<li><a href=#dev_env>Development Environment</a></li>
				35	<li><a href=#android>Android Toolbox</a></li>
Rob Landley	a136fa5	2014-12-20 14:58:03 -0600	[diff] [blame]	36	<li><a href=#tizen>Tizen Core</a></li>
Rob Landley	348a800	2014-04-09 07:57:08 -0500	[diff] [blame]	37	<li>Miscelaneous: <a href=#klibc>klibc</a>, <a href=#glibc>glibc</a>,
				38	<a href=#sash>sash</a>, <a href=#sbase>sbase</a>, <a href=#s6>s6</a>...</li>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	39	</ul>
				40
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	41	<hr />
				42	<a name="standards">
				43	<h2>Use case: standards compliance.</h2>
				44
Rob Landley	571b070	2012-11-13 16:13:45 -0600	[diff] [blame]	45	<h3><a name=susv4 /><a href="#susv4">POSIX-2008/SUSv4</a></h3>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	46	<p>The best standards are the kind that describe reality, rather than
				47	attempting to impose a new one. (I.E. a good standard should document, not
				48	legislate.)</p>
				49
				50	<p>The kind of standards which describe existing reality tend to be approved by
				51	more than one standards body, such ANSI and ISO both approving C. That's why
				52	the IEEE POSIX committee's 2008 standard, the Single Unix Specification version
				53	4, and the Open Group Base Specification edition 7 are all the same standard
				54	from three sources.</p>
				55
Rob Landley	62f0021	2012-12-06 15:15:30 -0600	[diff] [blame]	56	<p>The <a href="http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html">"utilities"
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	57	section</a>
				58	of these standards is devoted to the unix command line, and are the best such
				59	standard for our purposes. (My earlier work on BusyBox was implemented with
				60	regard to SUSv3, an earlier version of this standard.)</p>
				61
				62	<h3>Problems with the standard</h3>
				63
				64	<p>Unfortunately, these standards describe a subset of reality, lacking any
				65	mention of commands such as init, login, or mount required to actually boot a
				66	system. It provides ipcrm and ipcs, but not ipcmk, so you can use System V IPC
				67	resources but not create them.</p>
				68
				69	<p>These standards also contain a large number of commands that are
				70	inappropriate for toybox to implement in its 1.0 release. (Perhaps some of
				71	these could be reintroduced in later releases, but not now.)</p>
				72
				73	<p>Starting with the full "utilities" list, we first remove generally obsolete
				74	commands (compess ed ex pr uncompress uccp uustat uux), commands for the
				75	pre-CVS "SCCS" source control system (admin delta get prs rmdel sact sccs unget
				76	val what), fortran support (asa fort77), and batch processing support (batch
				77	qalter qdel qhold qmove qmsg qrerun qrls qselect qsig qstat qsub).</p>
				78
				79	<p>Some commands are for a compiler toolchain (ar c99 cflow ctags cxref gencat
				80	iconv lex m4 make nm strings strip tsort yacc), which is outside of toybox's
				81	mandate and should be supplied externally. (Again, some of these may be
				82	revisited later, but not for toybox 1.0.)</p>
				83
				84	<p>Some commands are part of a command shell, and cannot be implemented as
				85	separate executables (alias bg cd command fc fg getopts hash jobs kill read
				86	type ulimit umask unalias wait). These may be revisited as part of a built-in
				87	toybox shell, but are not exported into $PATH via symlinks. (If you fork a
				88	child process and have it "cd" then exit, you've accomplished nothing.)</p>
				89
				90	<p>A few other commands are judgement calls, providing command-line
				91	internationalization support (iconv locale localedef), System V inter-process
				92	communication (ipcrm ipcs), and cross-tty communication from the minicomputer
				93	days (talk mesg write). The "pax" utility was supplanted by tar, "mailx" is
				94	a command line email client, and "lp" submits files for printing to... what
				95	exactly? (cups?) The standard defines crontab but not crond.</p>
				96
				97	<p>Removing all of that leaves the following commands, which toybox should
				98	implement:</p>
				99
				100	<blockquote><b>
				101	<span id=posix>
				102	at awk basename bc cal cat chgrp chmod chown cksum cmp comm cp
				103	csplit cut date dd df diff dirname du echo env expand expr false file find
				104	fold fuser getconf grep head id join kill link ln logger logname ls man
				105	mkdir mkfifo more mv newgrp nice nl nohup od paste patch pathchk printf ps
				106	pwd renice rm rmdir sed sh sleep sort split stty tabs tail tee test time
				107	touch tput tr true tty uname unexpand uniq unlink uudecode uuencode vi wc
				108	who xargs zcat
				109	</span>
				110	</b></blockquote>
				111
Rob Landley	571b070	2012-11-13 16:13:45 -0600	[diff] [blame]	112	<h3><a name=sigh /><a href="#sigh">Linux Standard Base</a></h3>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	113
				114	<p>One attempt to supplement POSIX towards an actual usable system was the
				115	Linux Standard Base. Unfortunately, the quality of this "standard" is
				116	fairly low.</p>
				117
				118	<p>POSIX allowed its standards process to be compromised
				119	by leaving things out, thus allowing IBM mainframes and Windows NT to drive
				120	a truck through the holes and declare themselves compilant. But it means what
				121	they DID standardize tends to be respected.</p>
				122
				123	<p>The Linux Standard Base's failure mode is different, they respond to
				124	pressure by including special-case crap, such as allowing Red Hat to shoehorn
Rob Landley	62f0021	2012-12-06 15:15:30 -0600	[diff] [blame]	125	RPM on the standard even though all sorts of distros (Debian, Slackware, Arch,
				126	Gentoo) don't use it and probably never will. This means anything in the LSB is
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	127	at best a suggestion: arbitrary portions of this standard are widely
				128	ignored.</p>
				129
				130	<p>The LSB does specify a <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/cmdbehav.html>list of command line
				131	utilities</a>:</p>
				132
				133	<blockquote><b>
				134	ar at awk batch bc chfn chsh col cpio crontab df dmesg du echo egrep
				135	fgrep file fuser gettext grep groupadd groupdel groupmod groups
				136	gunzip gzip hostname install install_initd ipcrm ipcs killall lpr ls
				137	lsb_release m4 md5sum mknod mktemp more mount msgfmt newgrp od passwd
				138	patch pidof remove_initd renice sed sendmail seq sh shutdown su sync
				139	tar umount useradd userdel usermod xargs zcat
				140	</b></blockquote>
				141
				142	<p>Where posix specifies one of those commands, LSB's deltas tend to be
				143	accomodations for broken tool versions which aren't up to date with the
				144	standard yet. (See <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/more.html>more</a> and <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/xargs.html>xargs</a>
				145	for examples.)</p>
				146
				147	<p>Since we've already committed to using our own judgement to skip bits of
				148	POSIX, and LSB's "judgement" in this regard is purely bug workarounds to declare
				149	various legacy tool implementations "compliant", this means we're mostly
				150	interested in the set of tools that aren't specified in posix at all.</p>
				151
				152	<p>Of these, gettext and msgfmt are internationalization, install_initd and
				153	remove_initd aren't present on ubuntu 10.04, lpr is out of scope, and
				154	lsb_release is a distro issue (it's a nice command, but the output of
				155	lsb_release -a is the name and version number of the linux distro you're
				156	running, which toybox doesn't know).</p>
				157
				158	<p>This leaves:</p>
				159
				160	<blockquote><b>
				161	<span id=lsb>
				162	chfn chsh dmesg egrep fgrep groupadd groupdel groupmod groups
				163	gunzip gzip hostname install killall md5sum
				164	mknod mktemp mount passwd pidof sendmail seq shutdown
				165	su sync tar umount useradd userdel usermod zcat
				166	</span>
				167	</b></blockquote>
				168
				169	<hr />
				170	<a name="dev_env">
Rob Landley	571b070	2012-11-13 16:13:45 -0600	[diff] [blame]	171	<h2><a href="#dev_env">Use case: provide a self-hosting development environment</a></h2>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	172
				173	<p>The following commands are enough to build the Aboriginal Linux development
				174	environment, boot it to a shell prompt, and build Linux From Scratch 6.8 under
				175	it. (Aboriginal Linux currently uses BusyBox for this, thus provides a
				176	drop-in test environment for toybox. We install both implementations side
				177	by side, redirecting the symlinks a command at a time until the older
				178	package is no longer used, and can be removed.)</p>
				179
				180	<p>This use case includes running init scripts and other shell scripts, running
				181	configure, make, and install in each package, and providing basic command line
				182	facilities such as a text editor. (It does not include a compiler toolchain or
				183	C library, those are outside the scope of this project.)</p>
				184
				185	<blockquote><b>
				186	<span id=development>
				187	bzcat cat cp dirname echo env patch rmdir sha1sum sleep sort sync
				188	true uname wc which yes zcat
				189	awk basename bzip2 chmod chown cmp cut date dd diff
				190	egrep expr find grep gzip head hostname id install ln ls
				191	mkdir mktemp mv od readlink rm sed sh tail tar touch tr uniq
				192	wget whoami xargs chgrp comm gunzip less logname man split
				193	tee test time bunzip2 chgrp chroot comm cpio dmesg
				194	dnsdomainname ftpd ftpget ftpput gunzip ifconfig init less
				195	logname losetup man mdev mount mountpoint nc pgrep pkill
				196	pwd route split stat switch_root tac umount vi
				197	</span>
				198	</b></blockquote>
				199
				200	<p>Note: Aboriginal Linux installs bash 2.05b as #!/bin/sh and its scripts
				201	require bash extensions not present in shells such as busybox ash.
				202	This means that toysh needs to supply several bash extensions _and_ work
				203	when called under the name "bash".</p>
				204
Rob Landley	8fe1814	2014-10-07 14:11:54 -0500	[diff] [blame]	205	<p>The <a href=http://landley.net/aboriginal>Aboriginal Linux</a>
				206	self-bootstrapping build still uses the following busybox commands,
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame]	207	not yet supplied by toybox:</p>
Rob Landley	8fe1814	2014-10-07 14:11:54 -0500	[diff] [blame]	208
				209	<blockquote><p>
				210	ash awk bunzip2 bzip2 dd diff expr fdisk ftpd ftpget ftpput gunzip
				211	gzip less man pgrep ping pkill ps route sed sh sha512sum tar test tr unxz vi
				212	wget xzcat zcat</p></blockquote>
				213
				214	<p>Many of those are in "pending". Most of the archive commands are needed
				215	because busybox tar doesn't call external versions. The remaining "difficult"
				216	commands are vi, awk, and ash.</p>
				217
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	218	<hr />
Rob Landley	571b070	2012-11-13 16:13:45 -0600	[diff] [blame]	219	<h2><a name=android /><a href="#android">Use case: Replacing Android Toolbox</a></h2>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	220
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	221	<p>Android has a policy against GPL in userspace, so even though BusyBox
				222	predates Android by many years, they couldn't use it. Instead they grabbed
				223	an old version of ash and implemented their own command line utility set
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame]	224	called "toolbox". ash was later replaced by
				225	<a href="https://www.mirbsd.org/mksh.htm">mksh</a>; toolbox is being
				226	replaced by toybox.</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	227
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	228	<p>Toolbox doesn't have its own repository, instead it's part of Android's
				229	<a href=https://android.googlesource.com/platform/system/core>system/core
Elliott Hughes	e00c786	2015-03-28 02:53:08 -0500	[diff] [blame]	230	git repository</a>.</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	231
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	232	<h3>Toolbox commands:</h3>
				233
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	234	<p>According to <a href=https://android.googlesource.com/platform/system/core/+/master/toolbox/Android.mk>
				235	system/core/toolbox/Android.mk</a> the toolbox directory builds the
				236	following commands:</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	237
				238	<blockquote><b>
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	239	dd du df getevent iftop ioctl ionice log ls
				240	lsof mount nandread newfs_msdos ps prlimit renice
				241	sendevent start stop top uptime watchprops
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	242	</b></blockquote>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	243
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	244	<h3>Other Android core commands</h3>
				245
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame]	246	<p>Other than the toolbox directory, the currently interesting
Elliott Hughes	2c7028a	2015-03-23 11:44:56 -0500	[diff] [blame]	247	subdirectories in the core repository are gpttool, init,
Elliott Hughes	8c4fdd8	2015-02-07 19:51:27 -0600	[diff] [blame]	248	logcat, logwrapper, mkbootimg, reboot, and run-as.</p>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	249
				250	<ul>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	251	<li><b>gpttool</b> - subset of fdisk</li>
				252	<li><b>init</b> - Android's PID 1</li>
				253	<li><b>logcat</b> - read android log format</li>
				254	<li><b>logwrapper</b> - redirect stdio to android log</li>
				255	<li><b>mkbootimg</b> - create signed boot image</li>
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame]	256	<li><b>reboot</b> - Android's reboot(1)</li>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	257	<li><b>run-as</b> - subset of sudo</li>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	258	</ul>
				259
				260	<p>Almost all of these reinvent an existing wheel with less functionality and a
				261	different user interface. We may want to provide that interface, but
Elliott Hughes	2c7028a	2015-03-23 11:44:56 -0500	[diff] [blame]	262	implementing the full commands (fdisk, init, and sudo) come first.</p>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	263
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame]	264	<p>Also, gpttool and mkbootimg are install tools.
				265	These aren't a priority if android wants to use its own
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	266	bespoke code to install itself.</p>
				267
				268	<h3>Analysis</h3>
				269
				270	<p>For reference, combining everything listed above, we get:</p>
				271
				272	<blockquote><b>
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	273	dd du df getevent gpttool iftop init ioctl ionice
Elliott Hughes	7072214	2015-04-10 16:57:54 -0500	[diff] [blame]	274	log logcat logwrapper ls lsof mkbootimg mount nandread
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	275	newfs_msdos ps prlimit reboot renice run-as
				276	sendevent start stop top uptime watchprops
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	277	</b></blockquote>
				278
				279	<p>We may eventually implement all of that, but for toybox 1.0 we need to
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	280	focus a bit. For our first pass, let's ignore selinux [note: the android
				281	guys submitted selinux code to us and we merged it],
enh	1fb2080	2014-11-24 17:26:09 -0600	[diff] [blame]	282	and grab just logcat and logwrapper from the "core"
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	283	commands (since the rest have some full/standard version providing that
				284	functionality, which we can implement a shim interface for later).</p>
				285
Elliott Hughes	2c7028a	2015-03-23 11:44:56 -0500	[diff] [blame]	286	<p>This means toybox should implement (or finish implementing):</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	287	<blockquote><b>
				288	<span id=toolbox>
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	289	dd du df getevent iftop ioctl ionice log logcat logwrapper ls lsof
Elliott Hughes	7072214	2015-04-10 16:57:54 -0500	[diff] [blame]	290	mount nandread newfs_msdos ps prlimit renice schedtop sendevent
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	291	smd start stop top uptime watchprops
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	292	</span>
Rob Landley	c26ca6e	2013-01-31 04:05:56 -0600	[diff] [blame]	293	</b></blockquote>
Rob Landley	fdc10c9	2012-10-16 17:09:30 -0500	[diff] [blame]	294
Rob Landley	a136fa5	2014-12-20 14:58:03 -0600	[diff] [blame]	295	<hr />
				296	<h2><a name=tizen /><a href="#tizen">Use case: Tizen Core</a></h2>
				297
				298	<p>The Tizen project has expressed a desire to eliminate GPLv3 software
				299	from its core system, and is installing toybox as
				300	<a href=https://wiki.tizen.org/wiki/Toybox>part of this process</a>.</p>
				301
				302	<p>They have a fairly long list of new commands they'd like to see in toybox:</p>
				303
				304	<blockquote><b>
				305	<span id=tizen>
				306	arch base64 users dir vdir unexpand shred join csplit
				307	hostid nproc runcon sha224 sha256 sha384 sha512 sha3 mkfs.vfat fsck.vfat
				308	dosfslabel uname stdbuf pinky diff3 sdiff zcmp zdiff zegrep zfgrep zless zmore
				309	</span>
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	310	</b></blockquote>
Rob Landley	a136fa5	2014-12-20 14:58:03 -0600	[diff] [blame]	311
				312	<p>In addition, they'd like to use several commands currently in pending:</p>
				313
				314	<blockquote><b>
				315	<span id=tizen>
				316	tar diff printf wget rsync fdisk vi less tr test stty fold expr dd
				317	</span>
				318	</b></blockquote>
				319
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	320	<p>Also, tizen uses a different Linux Security Module called SMACK, so
				321	many of the SELinux options ala ls -Z need smack alternatives in an
				322	if/else setup.</p>
				323
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	324	<hr /><a name=klibc />
Rob Landley	934b2d3	2013-05-10 18:54:14 -0500	[diff] [blame]	325	<h2>klibc:</h2>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	326
				327	<p>Long ago some kernel developers came up with a project called
				328	<a href=http://en.wikipedia.org/wiki/Klibc>klibc</a>.
				329	After a decade of development it still has no web page or HOWTO,
				330	and nobody's quite sure if the license is BSD or GPL. It inexplicably
				331	<a href=http://www.infoworld.com/d/data-center/perl-isnt-going-anywhere-better-or-worse-211580>requires perl to build</a>, and seems like an ideal candidate for
				332	replacement.</p>
				333
				334	<p>In addition to a C library even less capable than bionic (obsoleted by
				335	musl), klibc builds a random assortment of executables to run init scripts
				336	with. There's no multiplexer command, these are individual executables:</p>
				337
				338	<blockquote><p>
				339	cat chroot cpio dd dmesg false fixdep fstype gunzip gzip halt ipconfig kill
				340	kinit ln losetup ls minips mkdir mkfifo mknodes
				341	mksyntax mount mv nfsmount nuke pivot_root poweroff readlink reboot resume
				342	run-init sh sha1hash sleep sync true umount uname zcat
				343	</p></blockquote>
				344
				345	<p>To get that list, build klibc according to the instructions (I
				346	<a href=http://landley.net/notes-2013.html#23-01-2013>looked at</a> version
				347	2.0.2 and did cd klibc-*; ln -s /output/of/kernel/make/headers_install
				348	linux; make) then <b>echo $(for i in $(find . -type f); do file $i \| grep -q
				349	executable && basename $i; done \| grep -v '[.]g$' \| sort -u)</b> to find
				350	executables, then eliminated the .so files and .shared duplicates.</p>
				351
				352	<p>Some of those binaries are build-time tools that don't get installed,
				353	which removes mknodes, mksyntax, sha1hash, and fixdep from the list.
				354	(And sha1hash is just an unpolished sha1sum anyway.)</p>
				355
				356	<p>The run-init command is more commonly called switch_root, nuke is just
				357	"rm -rf -- $@", and minips is more commonly called "ps". I'm not doing aliases
				358	for the oddball names.</p>
				359
				360	<p>Yet more stale forks of dash and gzip sucked in here (see "dubious
				361	license terms" above), adding nothing to the other projects we've looked at.
				362	But we still need sh, gunzip, gzip, and zcat to replace this package.</p>
				363
				364	<p>By the time I did the analysis toybox already had cat, chroot, dmesg, false,
				365	kill, ln, losetup, ls, mkdir, mkfifo, readlink, rm, switch_root, sleep, sync,
				366	true, and uname.</p>
				367
				368	<p>The low hanging fruit is cpio, dd, ps, mv, and pivot_root.</p>
				369
				370	<p>The "kinit" command is another gratuitous rename, it's init running as PID 1.
				371	The halt, poweroff, and reboot commands work with it.</p>
				372
				373	<p>I've got mount and umount queued up already, fstype and nfsmount go with
				374	those. (And probably smbmount and p9mount, but this hasn't got one. Those
				375	are all about querying for login credentials, probably workable into the
				376	base mount command.)</p>
				377
				378	<p>The ipconfig command here has a built in dhcp client, so it's ifconfig
				379	and dhcpcd and maybe some other stuff.</p>
				380
				381	<p>The resume command is... weird. It finds a swap partition and reads data
				382	from it into a /proc file, something the kernel is capable of doing itself.
				383	(Even though the klibc author
				384	<a href=http://www.zytor.com/pipermail/klibc/2006-June/001748.html>attempted
				385	to remove</a> that capability from the kernel, current kernel/power/hibernate.c
				386	still parses "resume=" on the command line). And yet various distros seem to
Rob Landley	957c3f7	2015-04-19 00:15:46 -0500	[diff] [blame]	387	make use of klibc for this.
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	388	Given the history of swsusp/hibernate (and
				389	<a href=http://lwn.net/Articles/333007>TuxOnIce</a>
				390	and <a href=http://lwn.net/Articles/242107>kexec jump</a>) I've lost track
				391	of the current state of the art here. Ah, Documentation/power/userland-swsusp.txt
				392	has the API docs, and <a href=http://suspend.sf.net>here's a better
				393	tool</a>...</p>
				394
				395	<p>So the list of things actually in klibc are:</p>
				396
				397	<blockquote><b>
Rob Landley	c166faf	2013-09-01 07:25:37 -0500	[diff] [blame]	398	<span id=klibc_cmd>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	399	cat chroot dmesg false kill ln losetup ls mkdir mkfifo readlink rm switch_root
				400	sleep sync true uname
				401
				402	cpio dd ps mv pivot_root
				403	mount nfsmount fstype umount
				404	sh gunzip gzip zcat
				405	kinit halt poweroff reboot
				406	ipconfig
				407	resume
				408	</span>
				409	</b></blockquote>
				410
				411	<hr />
Rob Landley	348a800	2014-04-09 07:57:08 -0500	[diff] [blame]	412	<a name=glibc />
				413	<h2>glibc</h2>
				414
				415	<p>Rather a lot of command line utilities come bundled with glibc:</p>
				416
				417	<blockquote><b>
				418	catchsegv getconf getent iconv iconvconfig ldconfig ldd locale localedef
				419	mtrace nscd rpcent rpcinfo tzselect zdump zic
				420	</b></blockquote>
				421
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	422	<p>Of those, musl libc only implements ldd.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	423
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	424	<p>catchsegv is a rudimentary debugger, probably out of scope for toybox.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	425
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	426	<p>iconv has been <a href="#susv4">previously discussed</a>.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	427
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	428	<p>iconvconfig is only relevant if iconv is user-configurable; musl uses a
				429	non-configurable iconv.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	430
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	431	<p>getconf is a posix utility which displays several variables from
				432	unistd.h; it probably belongs in the development toolchain.</p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	433
				434	<p>getent handles retrieving entries from passwd-style databases
				435	(in a rather lame way) and is trivially replacable by grep.</p>
				436
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	437	<p>locale was discussed under <a href=#susv4>posix</a>.
				438	localedef compiles locale definitions, which musl currently does not use.</p>
				439
				440	<p>mtrace is a perl script to use the malloc debugging that glibc has built-in;
				441	this is not relevant for musl, and would necessarily vary with libc. </p>
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	442
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	443	<p>nscd is a name service caching daemon, which is not yet relevant for musl.
				444	rpcinfo and rpcent are related to rpc, which musl does not include.</p>
				445
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	446	<p>The remaining commands involve glibc's bundled timezone database,
				447	which seems to be derived from the <a href=http://www.iana.org/time-zones>IANA
				448	timezone database</a>. Unless we want to maintain our own fork of the
				449	standards body's database like glibc does, these are of no interest,
				450	but for completeness:</p>
				451
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	452	<p>tzselect outputs a TZ variable correponding to user input.
				453	The documentation does not indicate how to use it in a script, but it seems
				454	that Debian may have done so.
				455	zdump prints current time in each of several timezones, optionally
				456	outputting a great deal of extra information about each timezone.
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	457	zic converts a description of a timezone to a file in tz format.</p>
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	458
Rob Landley	cbd7752	2014-04-12 20:39:33 -0500	[diff] [blame]	459	<p>None of glibc's bundled commands are currently of interest to toybox.</p>
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	460
Isaac Dunham	931425c	2014-04-12 17:26:44 -0500	[diff] [blame]	461	</b></blockquote>
Rob Landley	348a800	2014-04-09 07:57:08 -0500	[diff] [blame]	462
				463	<hr />
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	464	<a name=sash />
				465	<h2>Stand-Alone Shell</h2>
				466
				467	<p>Wikipedia has <a href=http://en.wikipedia.org/wiki/Stand-alone_shell>a good
				468	summary of sash</a>, with links. The original Stand-Alone Shell project reached
				469	a stopping point, and then <a href=http://www.baiti.net/sash>"sash plus
				470	patches"</a> extended it a bit further. The result is a megabyte executable
				471	that provides 40 commands.</p>
				472
				473	<p>Sash is a shell with built-in commands. It doesn't have a multiplexer
				474	command, meaning "sash ls -l" doesn't work (you have to go "sash -c 'ls -l'").
				475	</p>
				476
				477	<p>The list of commands can be obtained via building it and doing
				478	"echo help \| ./sash \| awk '{print $1}' \| sed 's/^-//' \| xargs echo", which
				479	gives us:</p>
				480
				481	<blockquote><b>
				482	alias aliasall ar cd chattr chgrp chmod chown cmp cp chroot dd echo ed exec
				483	exit file find grep gunzip gzip help kill losetup losetup ln ls lsattr mkdir
				484	mknod more mount mv pivot_root printenv prompt pwd quit rm rmdir setenv source
				485	sum sync tar touch umask umount unalias where
				486	</b></blockquote>
				487
				488	<p>Plus sh because it's a shell. A dozen or so commands can only sanely be
				489	implemented as shell builtins (alias aliasall cd exec exit prompt quit setenv
				490	source umask unalias), where is an alias for which, and at triage time toybox
				491	already has chgrp, chmod, chown, cmp, cp, chroot, echo, help, kill, losetup,
				492	ln, ls, mkdir, mknod, printenv, pwd, rm, rmdir, sync, and touch.</p>
				493
				494	<p>This leaves:</p>
				495
				496	<blockquote><b>
				497	<span id=sash_cmd>
				498	ar chattr dd ed file find grep gunzip gzip lsattr more mount mv pivot_root
				499	sh sum tar umount
				500	</span>
				501	</b></blockquote>
				502
				503	<p>(For once, this project doesn't include a fork of gzip, instead
				504	it sucks in -lz from the host.)</p>
				505
				506	<hr />
				507	<a name=sbase />
				508	<h2>sbase:</h2>
				509
				510	<p>It's <a href=http://git.suckless.org/sbase>on suckless</a>. So far it's
				511	implemented:</p>
				512
				513	<blockquote><p>
Rob Landley	c166faf	2013-09-01 07:25:37 -0500	[diff] [blame]	514	<span id=sbase_cmd>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	515	basename cat chmod chown cksum cmp cp date dirname echo false fold grep head
				516	kill ln ls mc mkdir mkfifo mv nl nohup pwd rm seq sleep sort tail tee test
				517	touch true tty uname uniq wc yes
				518	</span>
				519	</p></blockquote>
				520
				521	<p>And has a TODO list:</p>
				522
				523	<blockquote><p>
Rob Landley	c166faf	2013-09-01 07:25:37 -0500	[diff] [blame]	524	<span id=sbase_cmd>
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	525	cal chgrp chvt comm cut df diff du env expand expr id md5sum nice paste
				526	printenv printf readlink rmdir seq sha1sum split sync test tr unexpand unlink
				527	who
				528	</span>
				529	</p></blockquote>
				530
				531	<p>At triage time, of the first list I still need to do: fold grep mc mv nl. Of
				532	the second list: diff expr paste printf split test tr unexpand who.</p>
				533
				534	<hr />
				535	<a name=s6 />
				536	<h2>s6</h2>
				537
				538	<p>The website <a href=http://skarnet.org/software/>skarnet</a> has a bunch
				539	of small utilities as part of something called "s6". This includes the
				540	<a href=http://skarnet.org/software/s6-portable-utils>s6-portabile-utils</a>
				541	and the <a href=http://skarnet.org/software/s6-linux-utils>s6-linux-utils</a>.
				542	</p>
				543
				544	<p>Both packages rely on multiple bespoke external libraries without which
				545	they can't compile. The source is completely uncommented and doesn't wrap at
				546	80 characters. Doing a find for *.c files brings up the following commands:</p>
				547
				548	<blockquote><b>
				549	<span id=s6>
				550	basename cat chmod chown chroot clock cut devd dirname echo env expr false
				551	format-filter freeramdisk grep halt head hiercopy hostname linkname ln
				552	logwatch ls maximumtime memoryhog mkdir mkfifo mount nice nuke pause
				553	pivotchroot poweroff printenv quote quote-filter reboot rename rmrf sleep
				554	sort swapoff swapon sync tail test touch true umount uniquename unquote
				555	unquote-filter update-symlinks
				556	</span>
				557	</b></blockquote>
				558
				559	<p>Triage: memoryhog isn't even listed on the website nor does it have
				560	a documentation file, clock seems like a subset
				561	of date, devd is some sort of netlink wrapper that spawns its command line
				562	every time it gets a message (maybe this is meant to implement part of
				563	udev/mdev?), format-filter is sort of awk's '{print $2}' function split out
				564	into its own command, hiercopy a subset of "cp -r", maximumtime is something
				565	I implemented as a shell script (more/timeout.sh in Aboriginal Linux),
				566	nuke isn't the same as klibc (this one's "kill SIG -1" only with hardwared
				567	SIG options), pause is a program that literally waits to be killed (I
				568	generally sleep 999999999 which is a little over 30 years),
				569	pivotchroot is a subset of switch_root, rmrf is rm -rf...</p>
				570
				571	<p>I see "nuke" resurface, and if "rmrf" wasn't also here I might think
				572	klibc had a point.</b>
				573
				574	<blockquote>
				575	basename cat chmod chown chroot cut dirname echo env expr false
				576	freeramdisk grep halt head hostname linkname ln
				577	logwatch ls mkdir mkfifo mount nice
				578	pivotchroot poweroff printenv quote quote-filter reboot rename sleep
				579	sort swapoff swapon sync tail test touch true umount uniquename unquote
				580	unquote-filter update-symlinks
				581	</blockquote>
				582
				583
				584	<hr />
				585	<a name=nash />
				586	<h2>nash:</h2>
				587
				588	<p>Red Hat's nash was part of its "mkinitrd" package, replacement for a shell
				589	and utilities on the boot floppy back in the 1990's (the same general idea
				590	as BusyBox, developed independently). Red Hat discontinued nash development
				591	in 2010, replacing it with dracut (which collects together existing packages,
				592	including busybox).</p>
				593
				594	<p>I couldn't figure out how to beat source code out of
				595	<a href=http://pkgs.fedoraproject.org/git/mkinitrd>Fedora's current git</a>
				596	repository. The last release version that used it was Fedora Core 12
				597	which has <a href=http://archive.fedoraproject.org/pub/archive/fedora/linux/releases/12/Fedora/source/SRPMS/mkinitrd-6.0.93-1.fc12.src.rpm>a source rpm</a>
				598	that can be unwound with "rpm2cpio mkinitrd.src.rpm \| cpio -i -d -H newc
				599	--no-absolute-filenames" and in there is a mkinitrd-6.0.93.tar.bz2 which
				600	has the source.</p>
				601
				602	<p>In addition to being a bit like a command shell, the nash man page lists the
				603	following commands:</p>
				604
				605	<blockquote><p>
				606	access echo find losetup mkdevices mkdir mknod mkdmnod mkrootdev mount
				607	pivot_root readlink raidautorun setquiet showlabels sleep switchroot umount
				608	</p></blockquote>
				609
				610	<p>Oddly, the only occurrence of the string pivot_root in the nash source code
				611	is in the man page, the command isn't there. (It seems to have been removed
				612	when the underscoreless switchroot went in.)</p>
				613
				614	<p>A more complete list seems to be the handlers[] array in nash.c:</p>
				615
				616	<blockquote><p>
				617	access buildEnv cat cond cp daemonize dm echo exec exit find kernelopt
				618	loadDrivers loadpolicy mkchardevs mkblktab mkblkdevs mkdir mkdmnod mknod
				619	mkrootdev mount netname network null plymouth hotplug killplug losetup
				620	ln ls raidautorun readlink resume resolveDevice rmparts setDeviceEnv
				621	setquiet setuproot showelfinterp showlabels sleep stabilized status switchroot
				622	umount waitdev
				623	</p></blockquote>
				624
				625	<p>This list is nuts: "plymouth" is an alias for "null" which is basically
				626	"true" (which thie above list doesn't have). Things like buildEnv and
				627	loadDrivers are bespoke Red Hat behavior that might as well be hardwired in
				628	to nash's main() without being called.</p>
				629
				630	<p>Instead of eliminating items
				631	from the list with an explanation for each, I'm just going to cherry pick
				632	a few: the device mapper (dm, raidautorun) is probably interesting,
				633	hotplug (may be obsolete due to kernel changes that now load firmware
				634	directly), and another "resume" ala klibc.</p>
				635
				636	<p>But mostly: I don't care about this one. And neither does Red Hat anymore.</p>
				637
Rob Landley	ca04c7f	2013-03-27 22:34:28 -0500	[diff] [blame]	638	<p>Verdict: ignore</p>
				639
Rob Landley	9a69a92	2013-02-23 18:32:08 -0600	[diff] [blame]	640	<hr />
				641	<a name=beastiebox />
				642	<h2>Beastiebox</h2>
				643
				644	<p>Back in 2008, the BSD guys vented some busybox-envy
				645	<a href=http://beastiebox.sourceforge.net>on sourceforge</a>. Then stopped.
				646	Their repository is still in CVS, hasn't been touched in years, it's a giant
				647	hairball of existing code sucked together. (The web page says the author
				648	is aware of crunchgen, but decided to do this by hand anyway. This is not
				649	a collection of new code, it's a katamari of existing code rolled up in a
				650	ball.)</p>
				651
				652	<p>Combining the set of commands listed on the web page with the set of
				653	man pages in the source gives us:</P>
				654
				655	<blockquote><p>
				656	[ cat chmod cp csh date df disklabel dmesg echo ex fdisk fsck fsck_ffs getty
				657	halt hostname ifconfig init kill less lesskey ln login ls lv mksh more mount
				658	mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh stty sysctl tar test
				659	traceroute umount vi wiconfig
				660	</p></blockquote>
				661
				662	<p>Apparently lv is the missing link ed and vi, copyright 1982-1997 (do not
				663	want), ex is another obsolete vi mode, lesskey is "used to
				664	specify a set of key bindings to be used with less", and csh is a shell they
				665	sucked in, [ is an alias for test. Several more bsd-isms that don't have Linux
				666	equivalents (even in the ubuntu "install this package" search) are
				667	disklabel, fsck_ffs, mount_ffs, and pfctl. And wiconfig is a wavelan interface
				668	network card driver utility. Subtracting all that and the commands toybox
				669	already implements at triage time, we get:</p>
				670
				671	<blockquote><p>
				672	<span id=beastiebox_cmd>
				673	fdisk fsck getty halt ifconfig init kill less mksh more mount mv ping poweroff
				674	ps reboot route sed sh stty sysctl tar test traceroute umount vi
				675	</span>
				676	</p></blockquote>
				677
				678	<p>Not a hugely interesting list, but eh.</p>
				679
Rob Landley	ca04c7f	2013-03-27 22:34:28 -0500	[diff] [blame]	680	<p>Verdict: ignore</p>
				681
				682	<hr />
				683	<a name=BsdBox />
				684	<h2>BsdBox</h2>
				685
				686	<p>Somebody decided to do a <a href=https://wiki.freebsd.org/AdrianChadd/BsdBox>multicall binary for freebsd</a>.</p>
				687
				688	<p>They based it on crunchgen, a tool that glues existing programs together
				689	into an archive and uses the name to execute the right one. It has no
				690	simplification or code sharing benefits whatsoever, it's basically an
				691	archiver that produces executables.</p>
				692
				693	<p>That's about where I stopped reading.</p>
				694
				695	<p>Verdict: ignore.</p>
				696
				697	<hr />
				698	<a name=slowaris />
				699	<h2>OpenSolaris Busybox</h2>
				700
				701	<p>Somebody <a href=http://hub.opensolaris.org/bin/view/Project+busybox/>wrote
				702	a wiki page</a> saying that Busybox for OpenSolaris would be a good idea.</p>
				703
				704	<p>The corresponding "files" tab is an auto-generated stub. The project never
				705	even got as far as suggesting commands to include before Oracle discontinued
				706	OpenSolaris.</p>
				707
				708	<p>Verdict: ignore.</p>
				709
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	710	<hr />
				711	<h2>Requests:</h2>
				712
Rob Landley	8fe1814	2014-10-07 14:11:54 -0500	[diff] [blame]	713	<p>The following additional commands have been requested (and often submitted)
				714	by various users:</p>
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	715	<blockquote><b>
				716	<span id=request>
Rob Landley	64b6319	2013-12-31 09:01:32 -0600	[diff] [blame]	717	dig freeramdisk getty halt hexdump hwclock klogd modprobe ping ping6 pivot_root
Rob Landley	ca04c7f	2013-03-27 22:34:28 -0500	[diff] [blame]	718	poweroff readahead rev sfdisk sudo syslogd taskset telnet telnetd tracepath
				719	traceroute unzip usleep vconfig zip free login modinfo unshare netcat help w
Rob Landley	64b6319	2013-12-31 09:01:32 -0600	[diff] [blame]	720	ntpd iwconfig iwlist rdate
Rob Landley	7dbb982	2014-02-21 22:24:02 -0600	[diff] [blame]	721	dos2unix unix2dos catv clear
				722	pmap realpath setsid timeout truncate
				723	mkswap swapon swapoff
				724	count oneit fstype
				725	acpi blkid eject pwdx
Rob Landley	8fe1814	2014-10-07 14:11:54 -0500	[diff] [blame]	726	sulogin rfkill bootchartd
				727	arp makedevs sysctl killall5 crond crontab deluser last mkpasswd watch
				728	ipaddr iplink iproute blockdev rpm2cpio arping brctl dumpleases fsck
				729	tcpsvd tftpd
				730	factor fallocate fsfreeze inotifyd lspci nbd-client partprobe strings
Rob Landley	3c99404	2015-01-01 17:37:57 -0600	[diff] [blame]	731	base64 mix
Rob Landley	8f90d3a	2012-07-21 23:58:40 -0500	[diff] [blame]	732	</span>
				733	</b></blockquote>
				734
				735	<!-- #include "footer.html" -->
				736