blob: 24157119777580a474e330a708dd5ee60ed9ea23 [file] [log] [blame]
Rob Landley349ff522014-01-04 13:09:42 -06001<html><head><title>toybox roadmap</title>
Rob Landley8f90d3a2012-07-21 23:58:40 -05002<!--#include file="header.html" -->
3<title>Toybox Roadmap</title>
4
5<h2>Goals and use cases</h2>
6
7<p>We have several potential use cases for a new set of command line
8utilities, and are using those to determine which commands to implement
9for Toybox's 1.0 release.</p>
10
Rob Landley9a69a922013-02-23 18:32:08 -060011<p>The most interesting standards are POSIX-2008 (also known as the Single
12Unix Specification version 4) and the Linux Standard Base (version 4.1).
13The main test harness including toybox in Aboriginal Linux and if that can
14build itself using the result to build Linux From Scratch (version 6.8).
15We also aim to replace Android's Toolbox.</p>
16
17<p>At a secondary level we'd like to meet other use cases. We've analyzed
18the commands provided by similar projects (klibc, sash, sbase, s6, embutils,
19nash, and beastiebox), along with various vendor configurations of busybox,
20and some end user requests.</p>
21
22<p>Finally, we'd like to provide a good replacement for the Bash shell,
23which was the first program Linux ever ran and remains the standard shell
24of Linux no matter what Ubuntu says. This doesn't mean including the full
25set of Bash 4.x functionality, but does involve {various,features} beyond
26posix.</p>
Rob Landley8f90d3a2012-07-21 23:58:40 -050027
28<p>See the <a href=status.html>status page</a> for the combined list
29and progress towards implementing it.</p>
30
Rob Landley9a69a922013-02-23 18:32:08 -060031<ul>
32<li><a href=#susv4>POSIX-2008/SUSv4</a></li>
33<li><a href=#sigh>Linux "Standard" Base</a></li>
34<li><a href=#dev_env>Development Environment</a></li>
35<li><a href=#android>Android Toolbox</a></li>
Rob Landley348a8002014-04-09 07:57:08 -050036<li>Miscelaneous: <a href=#klibc>klibc</a>, <a href=#glibc>glibc</a>,
37<a href=#sash>sash</a>, <a href=#sbase>sbase</a>, <a href=#s6>s6</a>...</li>
Rob Landley9a69a922013-02-23 18:32:08 -060038</ul>
39
Rob Landley8f90d3a2012-07-21 23:58:40 -050040<hr />
41<a name="standards">
42<h2>Use case: standards compliance.</h2>
43
Rob Landley571b0702012-11-13 16:13:45 -060044<h3><a name=susv4 /><a href="#susv4">POSIX-2008/SUSv4</a></h3>
Rob Landley8f90d3a2012-07-21 23:58:40 -050045<p>The best standards are the kind that describe reality, rather than
46attempting to impose a new one. (I.E. a good standard should document, not
47legislate.)</p>
48
49<p>The kind of standards which describe existing reality tend to be approved by
50more than one standards body, such ANSI and ISO both approving C. That's why
51the IEEE POSIX committee's 2008 standard, the Single Unix Specification version
524, and the Open Group Base Specification edition 7 are all the same standard
53from three sources.</p>
54
Rob Landley62f00212012-12-06 15:15:30 -060055<p>The <a href="http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html">"utilities"
Rob Landley8f90d3a2012-07-21 23:58:40 -050056section</a>
57of these standards is devoted to the unix command line, and are the best such
58standard for our purposes. (My earlier work on BusyBox was implemented with
59regard to SUSv3, an earlier version of this standard.)</p>
60
61<h3>Problems with the standard</h3>
62
63<p>Unfortunately, these standards describe a subset of reality, lacking any
64mention of commands such as init, login, or mount required to actually boot a
65system. It provides ipcrm and ipcs, but not ipcmk, so you can use System V IPC
66resources but not create them.</p>
67
68<p>These standards also contain a large number of commands that are
69inappropriate for toybox to implement in its 1.0 release. (Perhaps some of
70these could be reintroduced in later releases, but not now.)</p>
71
72<p>Starting with the full "utilities" list, we first remove generally obsolete
73commands (compess ed ex pr uncompress uccp uustat uux), commands for the
74pre-CVS "SCCS" source control system (admin delta get prs rmdel sact sccs unget
75val what), fortran support (asa fort77), and batch processing support (batch
76qalter qdel qhold qmove qmsg qrerun qrls qselect qsig qstat qsub).</p>
77
78<p>Some commands are for a compiler toolchain (ar c99 cflow ctags cxref gencat
79iconv lex m4 make nm strings strip tsort yacc), which is outside of toybox's
80mandate and should be supplied externally. (Again, some of these may be
81revisited later, but not for toybox 1.0.)</p>
82
83<p>Some commands are part of a command shell, and cannot be implemented as
84separate executables (alias bg cd command fc fg getopts hash jobs kill read
85type ulimit umask unalias wait). These may be revisited as part of a built-in
86toybox shell, but are not exported into $PATH via symlinks. (If you fork a
87child process and have it "cd" then exit, you've accomplished nothing.)</p>
88
89<p>A few other commands are judgement calls, providing command-line
90internationalization support (iconv locale localedef), System V inter-process
91communication (ipcrm ipcs), and cross-tty communication from the minicomputer
92days (talk mesg write). The "pax" utility was supplanted by tar, "mailx" is
93a command line email client, and "lp" submits files for printing to... what
94exactly? (cups?) The standard defines crontab but not crond.</p>
95
96<p>Removing all of that leaves the following commands, which toybox should
97implement:</p>
98
99<blockquote><b>
100<span id=posix>
101at awk basename bc cal cat chgrp chmod chown cksum cmp comm cp
102csplit cut date dd df diff dirname du echo env expand expr false file find
103fold fuser getconf grep head id join kill link ln logger logname ls man
104mkdir mkfifo more mv newgrp nice nl nohup od paste patch pathchk printf ps
105pwd renice rm rmdir sed sh sleep sort split stty tabs tail tee test time
106touch tput tr true tty uname unexpand uniq unlink uudecode uuencode vi wc
107who xargs zcat
108</span>
109</b></blockquote>
110
Rob Landley571b0702012-11-13 16:13:45 -0600111<h3><a name=sigh /><a href="#sigh">Linux Standard Base</a></h3>
Rob Landley8f90d3a2012-07-21 23:58:40 -0500112
113<p>One attempt to supplement POSIX towards an actual usable system was the
114Linux Standard Base. Unfortunately, the quality of this "standard" is
115fairly low.</p>
116
117<p>POSIX allowed its standards process to be compromised
118by leaving things out, thus allowing IBM mainframes and Windows NT to drive
119a truck through the holes and declare themselves compilant. But it means what
120they DID standardize tends to be respected.</p>
121
122<p>The Linux Standard Base's failure mode is different, they respond to
123pressure by including special-case crap, such as allowing Red Hat to shoehorn
Rob Landley62f00212012-12-06 15:15:30 -0600124RPM on the standard even though all sorts of distros (Debian, Slackware, Arch,
125Gentoo) don't use it and probably never will. This means anything in the LSB is
Rob Landley8f90d3a2012-07-21 23:58:40 -0500126at best a suggestion: arbitrary portions of this standard are widely
127ignored.</p>
128
129<p>The LSB does specify a <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/cmdbehav.html>list of command line
130utilities</a>:</p>
131
132<blockquote><b>
133ar at awk batch bc chfn chsh col cpio crontab df dmesg du echo egrep
134fgrep file fuser gettext grep groupadd groupdel groupmod groups
135gunzip gzip hostname install install_initd ipcrm ipcs killall lpr ls
136lsb_release m4 md5sum mknod mktemp more mount msgfmt newgrp od passwd
137patch pidof remove_initd renice sed sendmail seq sh shutdown su sync
138tar umount useradd userdel usermod xargs zcat
139</b></blockquote>
140
141<p>Where posix specifies one of those commands, LSB's deltas tend to be
142accomodations for broken tool versions which aren't up to date with the
143standard yet. (See <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/more.html>more</a> and <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/xargs.html>xargs</a>
144for examples.)</p>
145
146<p>Since we've already committed to using our own judgement to skip bits of
147POSIX, and LSB's "judgement" in this regard is purely bug workarounds to declare
148various legacy tool implementations "compliant", this means we're mostly
149interested in the set of tools that aren't specified in posix at all.</p>
150
151<p>Of these, gettext and msgfmt are internationalization, install_initd and
152remove_initd aren't present on ubuntu 10.04, lpr is out of scope, and
153lsb_release is a distro issue (it's a nice command, but the output of
154lsb_release -a is the name and version number of the linux distro you're
155running, which toybox doesn't know).</p>
156
157<p>This leaves:</p>
158
159<blockquote><b>
160<span id=lsb>
161chfn chsh dmesg egrep fgrep groupadd groupdel groupmod groups
162gunzip gzip hostname install killall md5sum
163mknod mktemp mount passwd pidof sendmail seq shutdown
164su sync tar umount useradd userdel usermod zcat
165</span>
166</b></blockquote>
167
168<hr />
169<a name="dev_env">
Rob Landley571b0702012-11-13 16:13:45 -0600170<h2><a href="#dev_env">Use case: provide a self-hosting development environment</a></h2>
Rob Landley8f90d3a2012-07-21 23:58:40 -0500171
172<p>The following commands are enough to build the Aboriginal Linux development
173environment, boot it to a shell prompt, and build Linux From Scratch 6.8 under
174it. (Aboriginal Linux currently uses BusyBox for this, thus provides a
175drop-in test environment for toybox. We install both implementations side
176by side, redirecting the symlinks a command at a time until the older
177package is no longer used, and can be removed.)</p>
178
179<p>This use case includes running init scripts and other shell scripts, running
180configure, make, and install in each package, and providing basic command line
181facilities such as a text editor. (It does not include a compiler toolchain or
182C library, those are outside the scope of this project.)</p>
183
184<blockquote><b>
185<span id=development>
186bzcat cat cp dirname echo env patch rmdir sha1sum sleep sort sync
187true uname wc which yes zcat
188awk basename bzip2 chmod chown cmp cut date dd diff
189egrep expr find grep gzip head hostname id install ln ls
190mkdir mktemp mv od readlink rm sed sh tail tar touch tr uniq
191wget whoami xargs chgrp comm gunzip less logname man split
192tee test time bunzip2 chgrp chroot comm cpio dmesg
193dnsdomainname ftpd ftpget ftpput gunzip ifconfig init less
194logname losetup man mdev mount mountpoint nc pgrep pkill
195pwd route split stat switch_root tac umount vi
196</span>
197</b></blockquote>
198
199<p>Note: Aboriginal Linux installs bash 2.05b as #!/bin/sh and its scripts
200require bash extensions not present in shells such as busybox ash.
201This means that toysh needs to supply several bash extensions _and_ work
202when called under the name "bash".</p>
203
Rob Landley8fe18142014-10-07 14:11:54 -0500204<p>The <a href=http://landley.net/aboriginal>Aboriginal Linux</a>
205self-bootstrapping build still uses the following busybox commands,
206not yet supplied by toybox:</p.
207
208<blockquote><p>
209ash awk bunzip2 bzip2 dd diff expr fdisk ftpd ftpget ftpput gunzip
210gzip less man pgrep ping pkill ps route sed sh sha512sum tar test tr unxz vi
211wget xzcat zcat</p></blockquote>
212
213<p>Many of those are in "pending". Most of the archive commands are needed
214because busybox tar doesn't call external versions. The remaining "difficult"
215commands are vi, awk, and ash.</p>
216
Rob Landley8f90d3a2012-07-21 23:58:40 -0500217<hr />
Rob Landley571b0702012-11-13 16:13:45 -0600218<h2><a name=android /><a href="#android">Use case: Replacing Android Toolbox</a></h2>
Rob Landley8f90d3a2012-07-21 23:58:40 -0500219
Rob Landleyfdc10c92012-10-16 17:09:30 -0500220<p>Android has a policy against GPL in userspace, so even though BusyBox
221predates Android by many years, they couldn't use it. Instead they grabbed
222an old version of ash and implemented their own command line utility set
223called "toolbox".</p>
Rob Landley8f90d3a2012-07-21 23:58:40 -0500224
Rob Landleyfdc10c92012-10-16 17:09:30 -0500225<p>Toolbox doesn't have its own repository, instead it's part of Android's
226<a href=https://android.googlesource.com/platform/system/core>system/core
227git repository</a> (this analysis looked at commit 51ccef27cab58).</p>
Rob Landley8f90d3a2012-07-21 23:58:40 -0500228
Rob Landleyfdc10c92012-10-16 17:09:30 -0500229<h3>Toolbox commands:</h3>
230
231<p>According to core/toolbox/Android.mk the toolbox directory builds the
232following commands:</p>
Rob Landley8f90d3a2012-07-21 23:58:40 -0500233
234<blockquote><b>
Rob Landleyfdc10c92012-10-16 17:09:30 -0500235ls mount cat ps kill ln insmod rmmod lsmod ifconfig setconsole
236rm mkdir rmdir reboot getevent sendevent date wipe sync umount
237start stop notify cmp dmesg route hd dd df getprop setprop watchprops
238log sleep renice printenv smd chmod chown newfs_msdos netstat ioctl
239mv schedtop top iftop id uptime vmstat nandread ionice touch lsof md5 r
Rob Landley9a69a922013-02-23 18:32:08 -0600240cp du grep watchdogd
Rob Landley8f90d3a2012-07-21 23:58:40 -0500241</b></blockquote>
242
Rob Landleyfdc10c92012-10-16 17:09:30 -0500243<p>If selinux is enabled, you also get:</p>
244<blockquote><b>
245getenforce setenforce chcon restorecon runcon getsebool setsebool load_policy
246</b></blockquote>
Rob Landley8f90d3a2012-07-21 23:58:40 -0500247
Rob Landleyfdc10c92012-10-16 17:09:30 -0500248<p>The Android.mk file also refers to dynarray.c and toolbox.c as library
249code. This leaves the following apparently unused C files in toolbox/*.c, each
250of which has a command_main() function and seems to implement a standalone
251command:</p>
252
253<blockquote><b>
254alarm exists lsusb readtty rotatefb setkey syren
255</b></blockquote>
256
257<h3>Command shell (ash)</h3>
258
259<p>The core/sh subdirectory contains a fork of ash 1.17, and sucks in
260liblinenoise to provide command line history/editing.</p>
261
262<h3>Other Android core commands</h3>
263
264<p>Other than the toolbox and sh directories, the currently interesting
265subdirectories in the core repository are fs_mgr, gpttool, init,
266logcat, logwrapper, mkbootimg, netcfg, run-as, and sdcard.</p>
267
268<ul>
269<li><b>fs_mgr</b> - subset of mount</li>
270<li><b>gpttool</b> - subset of fdisk</li>
271<li><b>init</b> - Android's PID 1</li>
272<li><b>logcat</b> - read android log format</li>
273<li><b>logwrapper</b> - redirect stdio to android log</li>
274<li><b>mkbootimg</b> - create signed boot image</li>
275<li><b>netcfg</b> - network configuration (sucks in libnetutils)</li>
276<li><b>run-as</b> - subset of sudo</li>
277<li><b>sdcard</b> - FUSE wrapper to squash UID/GID/permissions to what FAT supports.</li>
278</ul>
279
280<p>Almost all of these reinvent an existing wheel with less functionality and a
281different user interface. We may want to provide that interface, but
282implementing the full commands (mount, fdisk, init, ifconfig with dhcp,
283and sudo) come first.</p>
284
285<p>Although logcat/logwrapper also reinvent a wheel, Android did so in the
286kernel and these provide an interface to that.</p>
287
288<p>Also, gpttool and mkbootimg are install tools, and sdcard looks like a
289testing tool. These aren't a priority if android wants to use its own
290bespoke code to install itself.</p>
291
292<h3>Analysis</h3>
293
294<p>For reference, combining everything listed above, we get:</p>
295
296<blockquote><b>
Rob Landley9a69a922013-02-23 18:32:08 -0600297alarm ash cat chcon chmod chown cmp cp date dd df dmesg du exists fs_mgr
298getenforce
299getevent getprop getsebool gpttool grep hd id ifconfig iftop init insmod ioctl
Rob Landleyfdc10c92012-10-16 17:09:30 -0500300ionice kill ln load_policy log logcat logwrapper ls lsmod lsof lsusb md5
301mkbootimg mkdir mount mv nandread netcfg netstat newfs_msdos notify printenv
302ps r readtty reboot renice restorecon rm rmdir rmmod rotatefb route run-as
303runcon schedtop sdcard sendevent setconsole setenforce setkey setprop setsebool
Rob Landley9a69a922013-02-23 18:32:08 -0600304sleep smd start stop sync syren top touch umount uptime vmstat watchdogd
305watchprops wipe
Rob Landleyfdc10c92012-10-16 17:09:30 -0500306</b></blockquote>
307
308<p>We may eventually implement all of that, but for toybox 1.0 we need to
309focus a bit. For our first pass, let's ignore selinux, strip out the "unlisted"
310commands except lsusb, and grab just logcat and logwrapper from the "core"
311commands (since the rest have some full/standard version providing that
312functionality, which we can implement a shim interface for later).</p>
313
314<p>This means toybox should implement:</p>
Rob Landley8f90d3a2012-07-21 23:58:40 -0500315<blockquote><b>
316<span id=toolbox>
Rob Landley9a69a922013-02-23 18:32:08 -0600317cat chmod chown cmp cp date dd df dmesg du getevent getprop grep hd id ifconfig
318iftop insmod ioctl ionice kill ln log logcat logwrapper ls lsmod lsof lsusb md5
319mkdir mount mv nandread
Rob Landleyfdc10c92012-10-16 17:09:30 -0500320netstat newfs_msdos notify printenv ps r reboot renice rm rmdir rmmod route
321schedtop sendevent setconsole setprop sleep smd start stop sync top touch
Rob Landley9a69a922013-02-23 18:32:08 -0600322umount uptime vmstat watchprops watchdogd wipe
Rob Landley8f90d3a2012-07-21 23:58:40 -0500323</span>
Rob Landleyc26ca6e2013-01-31 04:05:56 -0600324</b></blockquote>
Rob Landleyfdc10c92012-10-16 17:09:30 -0500325
326<p>The following Toolbox commands are already covered in previous
327sections of this analysis:</p>
328
329<blockquote><b>
Rob Landley9a69a922013-02-23 18:32:08 -0600330cat chmod chown cmp cp date dd df dmesg du grep id ifconfig insmod kill ln ls
331lsmod mkdir mount mv ps renice rm rmdir rmmod route sleep sync top touch umount
Rob Landleyfdc10c92012-10-16 17:09:30 -0500332</b></blockquote>
333
334<p>Which leaves the following commands as new from Toolbox:</p>
335
336<blockquote><b>
337getevent getprop hd iftop ioctl ionice log lsof nandread netstat
338newfs_msdos notify printenv r reboot schedtop sendevent setconsole
Rob Landley9a69a922013-02-23 18:32:08 -0600339setprop smd start stop top uptime vmstat watchprops watchdogd wipe
Rob Landley8f90d3a2012-07-21 23:58:40 -0500340</b></blockquote>
341
Rob Landley9a69a922013-02-23 18:32:08 -0600342<hr /><a name=klibc />
Rob Landley934b2d32013-05-10 18:54:14 -0500343<h2>klibc:</h2>
Rob Landley9a69a922013-02-23 18:32:08 -0600344
345<p>Long ago some kernel developers came up with a project called
346<a href=http://en.wikipedia.org/wiki/Klibc>klibc</a>.
347After a decade of development it still has no web page or HOWTO,
348and nobody's quite sure if the license is BSD or GPL. It inexplicably
349<a href=http://www.infoworld.com/d/data-center/perl-isnt-going-anywhere-better-or-worse-211580>requires perl to build</a>, and seems like an ideal candidate for
350replacement.</p>
351
352<p>In addition to a C library even less capable than bionic (obsoleted by
353musl), klibc builds a random assortment of executables to run init scripts
354with. There's no multiplexer command, these are individual executables:</p>
355
356<blockquote><p>
357cat chroot cpio dd dmesg false fixdep fstype gunzip gzip halt ipconfig kill
358kinit ln losetup ls minips mkdir mkfifo mknodes
359mksyntax mount mv nfsmount nuke pivot_root poweroff readlink reboot resume
360run-init sh sha1hash sleep sync true umount uname zcat
361</p></blockquote>
362
363<p>To get that list, build klibc according to the instructions (I
364<a href=http://landley.net/notes-2013.html#23-01-2013>looked at</a> version
3652.0.2 and did cd klibc-*; ln -s /output/of/kernel/make/headers_install
366linux; make) then <b>echo $(for i in $(find . -type f); do file $i | grep -q
367executable && basename $i; done | grep -v '[.]g$' | sort -u)</b> to find
368executables, then eliminated the *.so files and *.shared duplicates.</p>
369
370<p>Some of those binaries are build-time tools that don't get installed,
371which removes mknodes, mksyntax, sha1hash, and fixdep from the list.
372(And sha1hash is just an unpolished sha1sum anyway.)</p>
373
374<p>The run-init command is more commonly called switch_root, nuke is just
375"rm -rf -- $@", and minips is more commonly called "ps". I'm not doing aliases
376for the oddball names.</p>
377
378<p>Yet more stale forks of dash and gzip sucked in here (see "dubious
379license terms" above), adding nothing to the other projects we've looked at.
380But we still need sh, gunzip, gzip, and zcat to replace this package.</p>
381
382<p>By the time I did the analysis toybox already had cat, chroot, dmesg, false,
383kill, ln, losetup, ls, mkdir, mkfifo, readlink, rm, switch_root, sleep, sync,
384true, and uname.</p>
385
386<p>The low hanging fruit is cpio, dd, ps, mv, and pivot_root.</p>
387
388<p>The "kinit" command is another gratuitous rename, it's init running as PID 1.
389The halt, poweroff, and reboot commands work with it.</p>
390
391<p>I've got mount and umount queued up already, fstype and nfsmount go with
392those. (And probably smbmount and p9mount, but this hasn't got one. Those
393are all about querying for login credentials, probably workable into the
394base mount command.)</p>
395
396<p>The ipconfig command here has a built in dhcp client, so it's ifconfig
397and dhcpcd and maybe some other stuff.</p>
398
399<p>The resume command is... weird. It finds a swap partition and reads data
400from it into a /proc file, something the kernel is capable of doing itself.
401(Even though the klibc author
402<a href=http://www.zytor.com/pipermail/klibc/2006-June/001748.html>attempted
403to remove</a> that capability from the kernel, current kernel/power/hibernate.c
404still parses "resume=" on the command line). And yet various distros seem to
405make use of klibc for this>
406Given the history of swsusp/hibernate (and
407<a href=http://lwn.net/Articles/333007>TuxOnIce</a>
408and <a href=http://lwn.net/Articles/242107>kexec jump</a>) I've lost track
409of the current state of the art here. Ah, Documentation/power/userland-swsusp.txt
410has the API docs, and <a href=http://suspend.sf.net>here's a better
411tool</a>...</p>
412
413<p>So the list of things actually in klibc are:</p>
414
415<blockquote><b>
Rob Landleyc166faf2013-09-01 07:25:37 -0500416<span id=klibc_cmd>
Rob Landley9a69a922013-02-23 18:32:08 -0600417cat chroot dmesg false kill ln losetup ls mkdir mkfifo readlink rm switch_root
418sleep sync true uname
419
420cpio dd ps mv pivot_root
421mount nfsmount fstype umount
422sh gunzip gzip zcat
423kinit halt poweroff reboot
424ipconfig
425resume
426</span>
427</b></blockquote>
428
429<hr />
Rob Landley348a8002014-04-09 07:57:08 -0500430<a name=glibc />
431<h2>glibc</h2>
432
433<p>Rather a lot of command line utilities come bundled with glibc:</p>
434
435<blockquote><b>
436catchsegv getconf getent iconv iconvconfig ldconfig ldd locale localedef
437mtrace nscd rpcent rpcinfo tzselect zdump zic
438</b></blockquote>
439
Isaac Dunham931425c2014-04-12 17:26:44 -0500440<p>Of those, musl libc only implements ldd.</p>
Rob Landleycbd77522014-04-12 20:39:33 -0500441
Isaac Dunham931425c2014-04-12 17:26:44 -0500442<p>catchsegv is a rudimentary debugger, probably out of scope for toybox.</p>
Rob Landleycbd77522014-04-12 20:39:33 -0500443
Isaac Dunham931425c2014-04-12 17:26:44 -0500444<p>iconv has been <a href="#susv4">previously discussed</a>.</p>
Rob Landleycbd77522014-04-12 20:39:33 -0500445
Isaac Dunham931425c2014-04-12 17:26:44 -0500446<p>iconvconfig is only relevant if iconv is user-configurable; musl uses a
447non-configurable iconv.</p>
Rob Landleycbd77522014-04-12 20:39:33 -0500448
Isaac Dunham931425c2014-04-12 17:26:44 -0500449<p>getconf is a posix utility which displays several variables from
450unistd.h; it probably belongs in the development toolchain.</p>
Rob Landleycbd77522014-04-12 20:39:33 -0500451
452<p>getent handles retrieving entries from passwd-style databases
453(in a rather lame way) and is trivially replacable by grep.</p>
454
Isaac Dunham931425c2014-04-12 17:26:44 -0500455<p>locale was discussed under <a href=#susv4>posix</a>.
456localedef compiles locale definitions, which musl currently does not use.</p>
457
458<p>mtrace is a perl script to use the malloc debugging that glibc has built-in;
459this is not relevant for musl, and would necessarily vary with libc. </p>
Rob Landleycbd77522014-04-12 20:39:33 -0500460
Isaac Dunham931425c2014-04-12 17:26:44 -0500461<p>nscd is a name service caching daemon, which is not yet relevant for musl.
462rpcinfo and rpcent are related to rpc, which musl does not include.</p>
463
Rob Landleycbd77522014-04-12 20:39:33 -0500464<p>The remaining commands involve glibc's bundled timezone database,
465which seems to be derived from the <a href=http://www.iana.org/time-zones>IANA
466timezone database</a>. Unless we want to maintain our own fork of the
467standards body's database like glibc does, these are of no interest,
468but for completeness:</p>
469
Isaac Dunham931425c2014-04-12 17:26:44 -0500470<p>tzselect outputs a TZ variable correponding to user input.
471The documentation does not indicate how to use it in a script, but it seems
472that Debian may have done so.
473zdump prints current time in each of several timezones, optionally
474outputting a great deal of extra information about each timezone.
Rob Landleycbd77522014-04-12 20:39:33 -0500475zic converts a description of a timezone to a file in tz format.</p>
Isaac Dunham931425c2014-04-12 17:26:44 -0500476
Rob Landleycbd77522014-04-12 20:39:33 -0500477<p>None of glibc's bundled commands are currently of interest to toybox.</p>
Isaac Dunham931425c2014-04-12 17:26:44 -0500478
Isaac Dunham931425c2014-04-12 17:26:44 -0500479</b></blockquote>
Rob Landley348a8002014-04-09 07:57:08 -0500480
481<hr />
Rob Landley9a69a922013-02-23 18:32:08 -0600482<a name=sash />
483<h2>Stand-Alone Shell</h2>
484
485<p>Wikipedia has <a href=http://en.wikipedia.org/wiki/Stand-alone_shell>a good
486summary of sash</a>, with links. The original Stand-Alone Shell project reached
487a stopping point, and then <a href=http://www.baiti.net/sash>"sash plus
488patches"</a> extended it a bit further. The result is a megabyte executable
489that provides 40 commands.</p>
490
491<p>Sash is a shell with built-in commands. It doesn't have a multiplexer
492command, meaning "sash ls -l" doesn't work (you have to go "sash -c 'ls -l'").
493</p>
494
495<p>The list of commands can be obtained via building it and doing
496"echo help | ./sash | awk '{print $1}' | sed 's/^-//' | xargs echo", which
497gives us:</p>
498
499<blockquote><b>
500alias aliasall ar cd chattr chgrp chmod chown cmp cp chroot dd echo ed exec
501exit file find grep gunzip gzip help kill losetup losetup ln ls lsattr mkdir
502mknod more mount mv pivot_root printenv prompt pwd quit rm rmdir setenv source
503sum sync tar touch umask umount unalias where
504</b></blockquote>
505
506<p>Plus sh because it's a shell. A dozen or so commands can only sanely be
507implemented as shell builtins (alias aliasall cd exec exit prompt quit setenv
508source umask unalias), where is an alias for which, and at triage time toybox
509already has chgrp, chmod, chown, cmp, cp, chroot, echo, help, kill, losetup,
510ln, ls, mkdir, mknod, printenv, pwd, rm, rmdir, sync, and touch.</p>
511
512<p>This leaves:</p>
513
514<blockquote><b>
515<span id=sash_cmd>
516ar chattr dd ed file find grep gunzip gzip lsattr more mount mv pivot_root
517sh sum tar umount
518</span>
519</b></blockquote>
520
521<p>(For once, this project doesn't include a fork of gzip, instead
522it sucks in -lz from the host.)</p>
523
524<hr />
525<a name=sbase />
526<h2>sbase:</h2>
527
528<p>It's <a href=http://git.suckless.org/sbase>on suckless</a>. So far it's
529implemented:</p>
530
531<blockquote><p>
Rob Landleyc166faf2013-09-01 07:25:37 -0500532<span id=sbase_cmd>
Rob Landley9a69a922013-02-23 18:32:08 -0600533basename cat chmod chown cksum cmp cp date dirname echo false fold grep head
534kill ln ls mc mkdir mkfifo mv nl nohup pwd rm seq sleep sort tail tee test
535touch true tty uname uniq wc yes
536</span>
537</p></blockquote>
538
539<p>And has a TODO list:</p>
540
541<blockquote><p>
Rob Landleyc166faf2013-09-01 07:25:37 -0500542<span id=sbase_cmd>
Rob Landley9a69a922013-02-23 18:32:08 -0600543cal chgrp chvt comm cut df diff du env expand expr id md5sum nice paste
544printenv printf readlink rmdir seq sha1sum split sync test tr unexpand unlink
545who
546</span>
547</p></blockquote>
548
549<p>At triage time, of the first list I still need to do: fold grep mc mv nl. Of
550the second list: diff expr paste printf split test tr unexpand who.</p>
551
552<hr />
553<a name=s6 />
554<h2>s6</h2>
555
556<p>The website <a href=http://skarnet.org/software/>skarnet</a> has a bunch
557of small utilities as part of something called "s6". This includes the
558<a href=http://skarnet.org/software/s6-portable-utils>s6-portabile-utils</a>
559and the <a href=http://skarnet.org/software/s6-linux-utils>s6-linux-utils</a>.
560</p>
561
562<p>Both packages rely on multiple bespoke external libraries without which
563they can't compile. The source is completely uncommented and doesn't wrap at
56480 characters. Doing a find for *.c files brings up the following commands:</p>
565
566<blockquote><b>
567<span id=s6>
568basename cat chmod chown chroot clock cut devd dirname echo env expr false
569format-filter freeramdisk grep halt head hiercopy hostname linkname ln
570logwatch ls maximumtime memoryhog mkdir mkfifo mount nice nuke pause
571pivotchroot poweroff printenv quote quote-filter reboot rename rmrf sleep
572sort swapoff swapon sync tail test touch true umount uniquename unquote
573unquote-filter update-symlinks
574</span>
575</b></blockquote>
576
577<p>Triage: memoryhog isn't even listed on the website nor does it have
578a documentation file, clock seems like a subset
579of date, devd is some sort of netlink wrapper that spawns its command line
580every time it gets a message (maybe this is meant to implement part of
581udev/mdev?), format-filter is sort of awk's '{print $2}' function split out
582into its own command, hiercopy a subset of "cp -r", maximumtime is something
583I implemented as a shell script (more/timeout.sh in Aboriginal Linux),
584nuke isn't the same as klibc (this one's "kill SIG -1" only with hardwared
585SIG options), pause is a program that literally waits to be killed (I
586generally sleep 999999999 which is a little over 30 years),
587pivotchroot is a subset of switch_root, rmrf is rm -rf...</p>
588
589<p>I see "nuke" resurface, and if "rmrf" wasn't also here I might think
590klibc had a point.</b>
591
592<blockquote>
593basename cat chmod chown chroot cut dirname echo env expr false
594freeramdisk grep halt head hostname linkname ln
595logwatch ls mkdir mkfifo mount nice
596pivotchroot poweroff printenv quote quote-filter reboot rename sleep
597sort swapoff swapon sync tail test touch true umount uniquename unquote
598unquote-filter update-symlinks
599</blockquote>
600
601
602<hr />
603<a name=nash />
604<h2>nash:</h2>
605
606<p>Red Hat's nash was part of its "mkinitrd" package, replacement for a shell
607and utilities on the boot floppy back in the 1990's (the same general idea
608as BusyBox, developed independently). Red Hat discontinued nash development
609in 2010, replacing it with dracut (which collects together existing packages,
610including busybox).</p>
611
612<p>I couldn't figure out how to beat source code out of
613<a href=http://pkgs.fedoraproject.org/git/mkinitrd>Fedora's current git</a>
614repository. The last release version that used it was Fedora Core 12
615which has <a href=http://archive.fedoraproject.org/pub/archive/fedora/linux/releases/12/Fedora/source/SRPMS/mkinitrd-6.0.93-1.fc12.src.rpm>a source rpm</a>
616that can be unwound with "rpm2cpio mkinitrd.src.rpm | cpio -i -d -H newc
617--no-absolute-filenames" and in there is a mkinitrd-6.0.93.tar.bz2 which
618has the source.</p>
619
620<p>In addition to being a bit like a command shell, the nash man page lists the
621following commands:</p>
622
623<blockquote><p>
624access echo find losetup mkdevices mkdir mknod mkdmnod mkrootdev mount
625pivot_root readlink raidautorun setquiet showlabels sleep switchroot umount
626</p></blockquote>
627
628<p>Oddly, the only occurrence of the string pivot_root in the nash source code
629is in the man page, the command isn't there. (It seems to have been removed
630when the underscoreless switchroot went in.)</p>
631
632<p>A more complete list seems to be the handlers[] array in nash.c:</p>
633
634<blockquote><p>
635access buildEnv cat cond cp daemonize dm echo exec exit find kernelopt
636loadDrivers loadpolicy mkchardevs mkblktab mkblkdevs mkdir mkdmnod mknod
637mkrootdev mount netname network null plymouth hotplug killplug losetup
638ln ls raidautorun readlink resume resolveDevice rmparts setDeviceEnv
639setquiet setuproot showelfinterp showlabels sleep stabilized status switchroot
640umount waitdev
641</p></blockquote>
642
643<p>This list is nuts: "plymouth" is an alias for "null" which is basically
644"true" (which thie above list doesn't have). Things like buildEnv and
645loadDrivers are bespoke Red Hat behavior that might as well be hardwired in
646to nash's main() without being called.</p>
647
648<p>Instead of eliminating items
649from the list with an explanation for each, I'm just going to cherry pick
650a few: the device mapper (dm, raidautorun) is probably interesting,
651hotplug (may be obsolete due to kernel changes that now load firmware
652directly), and another "resume" ala klibc.</p>
653
654<p>But mostly: I don't care about this one. And neither does Red Hat anymore.</p>
655
Rob Landleyca04c7f2013-03-27 22:34:28 -0500656<p>Verdict: ignore</p>
657
Rob Landley9a69a922013-02-23 18:32:08 -0600658<hr />
659<a name=beastiebox />
660<h2>Beastiebox</h2>
661
662<p>Back in 2008, the BSD guys vented some busybox-envy
663<a href=http://beastiebox.sourceforge.net>on sourceforge</a>. Then stopped.
664Their repository is still in CVS, hasn't been touched in years, it's a giant
665hairball of existing code sucked together. (The web page says the author
666is aware of crunchgen, but decided to do this by hand anyway. This is not
667a collection of new code, it's a katamari of existing code rolled up in a
668ball.)</p>
669
670<p>Combining the set of commands listed on the web page with the set of
671man pages in the source gives us:</P>
672
673<blockquote><p>
674[ cat chmod cp csh date df disklabel dmesg echo ex fdisk fsck fsck_ffs getty
675halt hostname ifconfig init kill less lesskey ln login ls lv mksh more mount
676mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh stty sysctl tar test
677traceroute umount vi wiconfig
678</p></blockquote>
679
680<p>Apparently lv is the missing link ed and vi, copyright 1982-1997 (do not
681want), ex is another obsolete vi mode, lesskey is "used to
682specify a set of key bindings to be used with less", and csh is a shell they
683sucked in, [ is an alias for test. Several more bsd-isms that don't have Linux
684equivalents (even in the ubuntu "install this package" search) are
685disklabel, fsck_ffs, mount_ffs, and pfctl. And wiconfig is a wavelan interface
686network card driver utility. Subtracting all that and the commands toybox
687already implements at triage time, we get:</p>
688
689<blockquote><p>
690<span id=beastiebox_cmd>
691fdisk fsck getty halt ifconfig init kill less mksh more mount mv ping poweroff
692ps reboot route sed sh stty sysctl tar test traceroute umount vi
693</span>
694</p></blockquote>
695
696<p>Not a hugely interesting list, but eh.</p>
697
Rob Landleyca04c7f2013-03-27 22:34:28 -0500698<p>Verdict: ignore</p>
699
700<hr />
701<a name=BsdBox />
702<h2>BsdBox</h2>
703
704<p>Somebody decided to do a <a href=https://wiki.freebsd.org/AdrianChadd/BsdBox>multicall binary for freebsd</a>.</p>
705
706<p>They based it on crunchgen, a tool that glues existing programs together
707into an archive and uses the name to execute the right one. It has no
708simplification or code sharing benefits whatsoever, it's basically an
709archiver that produces executables.</p>
710
711<p>That's about where I stopped reading.</p>
712
713<p>Verdict: ignore.</p>
714
715<hr />
716<a name=slowaris />
717<h2>OpenSolaris Busybox</h2>
718
719<p>Somebody <a href=http://hub.opensolaris.org/bin/view/Project+busybox/>wrote
720a wiki page</a> saying that Busybox for OpenSolaris would be a good idea.</p>
721
722<p>The corresponding "files" tab is an auto-generated stub. The project never
723even got as far as suggesting commands to include before Oracle discontinued
724OpenSolaris.</p>
725
726<p>Verdict: ignore.</p>
727
Rob Landley8f90d3a2012-07-21 23:58:40 -0500728<hr />
729<h2>Requests:</h2>
730
Rob Landley8fe18142014-10-07 14:11:54 -0500731<p>The following additional commands have been requested (and often submitted)
732by various users:</p>
Rob Landley8f90d3a2012-07-21 23:58:40 -0500733<blockquote><b>
734<span id=request>
Rob Landley64b63192013-12-31 09:01:32 -0600735dig freeramdisk getty halt hexdump hwclock klogd modprobe ping ping6 pivot_root
Rob Landleyca04c7f2013-03-27 22:34:28 -0500736poweroff readahead rev sfdisk sudo syslogd taskset telnet telnetd tracepath
737traceroute unzip usleep vconfig zip free login modinfo unshare netcat help w
Rob Landley64b63192013-12-31 09:01:32 -0600738ntpd iwconfig iwlist rdate
Rob Landley7dbb9822014-02-21 22:24:02 -0600739dos2unix unix2dos catv clear
740pmap realpath setsid timeout truncate
741mkswap swapon swapoff
742count oneit fstype
743acpi blkid eject pwdx
Rob Landley8fe18142014-10-07 14:11:54 -0500744sulogin rfkill bootchartd
745arp makedevs sysctl killall5 crond crontab deluser last mkpasswd watch
746ipaddr iplink iproute blockdev rpm2cpio arping brctl dumpleases fsck
747tcpsvd tftpd
748factor fallocate fsfreeze inotifyd lspci nbd-client partprobe strings
749
Rob Landley8f90d3a2012-07-21 23:58:40 -0500750</span>
751</b></blockquote>
752
753<!-- #include "footer.html" -->
754