Rob Landley | 349ff52 | 2014-01-04 13:09:42 -0600 | [diff] [blame^] | 1 | <html><head><title>toybox roadmap</title> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 2 | <!--#include file="header.html" --> |
| 3 | <title>Toybox Roadmap</title> |
| 4 | |
| 5 | <h2>Goals and use cases</h2> |
| 6 | |
| 7 | <p>We have several potential use cases for a new set of command line |
| 8 | utilities, and are using those to determine which commands to implement |
| 9 | for Toybox's 1.0 release.</p> |
| 10 | |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 11 | <p>The most interesting standards are POSIX-2008 (also known as the Single |
| 12 | Unix Specification version 4) and the Linux Standard Base (version 4.1). |
| 13 | The main test harness including toybox in Aboriginal Linux and if that can |
| 14 | build itself using the result to build Linux From Scratch (version 6.8). |
| 15 | We also aim to replace Android's Toolbox.</p> |
| 16 | |
| 17 | <p>At a secondary level we'd like to meet other use cases. We've analyzed |
| 18 | the commands provided by similar projects (klibc, sash, sbase, s6, embutils, |
| 19 | nash, and beastiebox), along with various vendor configurations of busybox, |
| 20 | and some end user requests.</p> |
| 21 | |
| 22 | <p>Finally, we'd like to provide a good replacement for the Bash shell, |
| 23 | which was the first program Linux ever ran and remains the standard shell |
| 24 | of Linux no matter what Ubuntu says. This doesn't mean including the full |
| 25 | set of Bash 4.x functionality, but does involve {various,features} beyond |
| 26 | posix.</p> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 27 | |
| 28 | <p>See the <a href=status.html>status page</a> for the combined list |
| 29 | and progress towards implementing it.</p> |
| 30 | |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 31 | <ul> |
| 32 | <li><a href=#susv4>POSIX-2008/SUSv4</a></li> |
| 33 | <li><a href=#sigh>Linux "Standard" Base</a></li> |
| 34 | <li><a href=#dev_env>Development Environment</a></li> |
| 35 | <li><a href=#android>Android Toolbox</a></li> |
| 36 | <li>Miscelaneous: <a href=#klibc>klibc</a>, <a href=#sash>sash</a>, |
Rob Landley | ca04c7f | 2013-03-27 22:34:28 -0500 | [diff] [blame] | 37 | <a href=#sbase>sbase</a>, <a href=#s6>s6</a>...</li> |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 38 | </ul> |
| 39 | |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 40 | <hr /> |
| 41 | <a name="standards"> |
| 42 | <h2>Use case: standards compliance.</h2> |
| 43 | |
Rob Landley | 571b070 | 2012-11-13 16:13:45 -0600 | [diff] [blame] | 44 | <h3><a name=susv4 /><a href="#susv4">POSIX-2008/SUSv4</a></h3> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 45 | <p>The best standards are the kind that describe reality, rather than |
| 46 | attempting to impose a new one. (I.E. a good standard should document, not |
| 47 | legislate.)</p> |
| 48 | |
| 49 | <p>The kind of standards which describe existing reality tend to be approved by |
| 50 | more than one standards body, such ANSI and ISO both approving C. That's why |
| 51 | the IEEE POSIX committee's 2008 standard, the Single Unix Specification version |
| 52 | 4, and the Open Group Base Specification edition 7 are all the same standard |
| 53 | from three sources.</p> |
| 54 | |
Rob Landley | 62f0021 | 2012-12-06 15:15:30 -0600 | [diff] [blame] | 55 | <p>The <a href="http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html">"utilities" |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 56 | section</a> |
| 57 | of these standards is devoted to the unix command line, and are the best such |
| 58 | standard for our purposes. (My earlier work on BusyBox was implemented with |
| 59 | regard to SUSv3, an earlier version of this standard.)</p> |
| 60 | |
| 61 | <h3>Problems with the standard</h3> |
| 62 | |
| 63 | <p>Unfortunately, these standards describe a subset of reality, lacking any |
| 64 | mention of commands such as init, login, or mount required to actually boot a |
| 65 | system. It provides ipcrm and ipcs, but not ipcmk, so you can use System V IPC |
| 66 | resources but not create them.</p> |
| 67 | |
| 68 | <p>These standards also contain a large number of commands that are |
| 69 | inappropriate for toybox to implement in its 1.0 release. (Perhaps some of |
| 70 | these could be reintroduced in later releases, but not now.)</p> |
| 71 | |
| 72 | <p>Starting with the full "utilities" list, we first remove generally obsolete |
| 73 | commands (compess ed ex pr uncompress uccp uustat uux), commands for the |
| 74 | pre-CVS "SCCS" source control system (admin delta get prs rmdel sact sccs unget |
| 75 | val what), fortran support (asa fort77), and batch processing support (batch |
| 76 | qalter qdel qhold qmove qmsg qrerun qrls qselect qsig qstat qsub).</p> |
| 77 | |
| 78 | <p>Some commands are for a compiler toolchain (ar c99 cflow ctags cxref gencat |
| 79 | iconv lex m4 make nm strings strip tsort yacc), which is outside of toybox's |
| 80 | mandate and should be supplied externally. (Again, some of these may be |
| 81 | revisited later, but not for toybox 1.0.)</p> |
| 82 | |
| 83 | <p>Some commands are part of a command shell, and cannot be implemented as |
| 84 | separate executables (alias bg cd command fc fg getopts hash jobs kill read |
| 85 | type ulimit umask unalias wait). These may be revisited as part of a built-in |
| 86 | toybox shell, but are not exported into $PATH via symlinks. (If you fork a |
| 87 | child process and have it "cd" then exit, you've accomplished nothing.)</p> |
| 88 | |
| 89 | <p>A few other commands are judgement calls, providing command-line |
| 90 | internationalization support (iconv locale localedef), System V inter-process |
| 91 | communication (ipcrm ipcs), and cross-tty communication from the minicomputer |
| 92 | days (talk mesg write). The "pax" utility was supplanted by tar, "mailx" is |
| 93 | a command line email client, and "lp" submits files for printing to... what |
| 94 | exactly? (cups?) The standard defines crontab but not crond.</p> |
| 95 | |
| 96 | <p>Removing all of that leaves the following commands, which toybox should |
| 97 | implement:</p> |
| 98 | |
| 99 | <blockquote><b> |
| 100 | <span id=posix> |
| 101 | at awk basename bc cal cat chgrp chmod chown cksum cmp comm cp |
| 102 | csplit cut date dd df diff dirname du echo env expand expr false file find |
| 103 | fold fuser getconf grep head id join kill link ln logger logname ls man |
| 104 | mkdir mkfifo more mv newgrp nice nl nohup od paste patch pathchk printf ps |
| 105 | pwd renice rm rmdir sed sh sleep sort split stty tabs tail tee test time |
| 106 | touch tput tr true tty uname unexpand uniq unlink uudecode uuencode vi wc |
| 107 | who xargs zcat |
| 108 | </span> |
| 109 | </b></blockquote> |
| 110 | |
Rob Landley | 571b070 | 2012-11-13 16:13:45 -0600 | [diff] [blame] | 111 | <h3><a name=sigh /><a href="#sigh">Linux Standard Base</a></h3> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 112 | |
| 113 | <p>One attempt to supplement POSIX towards an actual usable system was the |
| 114 | Linux Standard Base. Unfortunately, the quality of this "standard" is |
| 115 | fairly low.</p> |
| 116 | |
| 117 | <p>POSIX allowed its standards process to be compromised |
| 118 | by leaving things out, thus allowing IBM mainframes and Windows NT to drive |
| 119 | a truck through the holes and declare themselves compilant. But it means what |
| 120 | they DID standardize tends to be respected.</p> |
| 121 | |
| 122 | <p>The Linux Standard Base's failure mode is different, they respond to |
| 123 | pressure by including special-case crap, such as allowing Red Hat to shoehorn |
Rob Landley | 62f0021 | 2012-12-06 15:15:30 -0600 | [diff] [blame] | 124 | RPM on the standard even though all sorts of distros (Debian, Slackware, Arch, |
| 125 | Gentoo) don't use it and probably never will. This means anything in the LSB is |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 126 | at best a suggestion: arbitrary portions of this standard are widely |
| 127 | ignored.</p> |
| 128 | |
| 129 | <p>The LSB does specify a <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/cmdbehav.html>list of command line |
| 130 | utilities</a>:</p> |
| 131 | |
| 132 | <blockquote><b> |
| 133 | ar at awk batch bc chfn chsh col cpio crontab df dmesg du echo egrep |
| 134 | fgrep file fuser gettext grep groupadd groupdel groupmod groups |
| 135 | gunzip gzip hostname install install_initd ipcrm ipcs killall lpr ls |
| 136 | lsb_release m4 md5sum mknod mktemp more mount msgfmt newgrp od passwd |
| 137 | patch pidof remove_initd renice sed sendmail seq sh shutdown su sync |
| 138 | tar umount useradd userdel usermod xargs zcat |
| 139 | </b></blockquote> |
| 140 | |
| 141 | <p>Where posix specifies one of those commands, LSB's deltas tend to be |
| 142 | accomodations for broken tool versions which aren't up to date with the |
| 143 | standard yet. (See <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/more.html>more</a> and <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/xargs.html>xargs</a> |
| 144 | for examples.)</p> |
| 145 | |
| 146 | <p>Since we've already committed to using our own judgement to skip bits of |
| 147 | POSIX, and LSB's "judgement" in this regard is purely bug workarounds to declare |
| 148 | various legacy tool implementations "compliant", this means we're mostly |
| 149 | interested in the set of tools that aren't specified in posix at all.</p> |
| 150 | |
| 151 | <p>Of these, gettext and msgfmt are internationalization, install_initd and |
| 152 | remove_initd aren't present on ubuntu 10.04, lpr is out of scope, and |
| 153 | lsb_release is a distro issue (it's a nice command, but the output of |
| 154 | lsb_release -a is the name and version number of the linux distro you're |
| 155 | running, which toybox doesn't know).</p> |
| 156 | |
| 157 | <p>This leaves:</p> |
| 158 | |
| 159 | <blockquote><b> |
| 160 | <span id=lsb> |
| 161 | chfn chsh dmesg egrep fgrep groupadd groupdel groupmod groups |
| 162 | gunzip gzip hostname install killall md5sum |
| 163 | mknod mktemp mount passwd pidof sendmail seq shutdown |
| 164 | su sync tar umount useradd userdel usermod zcat |
| 165 | </span> |
| 166 | </b></blockquote> |
| 167 | |
| 168 | <hr /> |
| 169 | <a name="dev_env"> |
Rob Landley | 571b070 | 2012-11-13 16:13:45 -0600 | [diff] [blame] | 170 | <h2><a href="#dev_env">Use case: provide a self-hosting development environment</a></h2> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 171 | |
| 172 | <p>The following commands are enough to build the Aboriginal Linux development |
| 173 | environment, boot it to a shell prompt, and build Linux From Scratch 6.8 under |
| 174 | it. (Aboriginal Linux currently uses BusyBox for this, thus provides a |
| 175 | drop-in test environment for toybox. We install both implementations side |
| 176 | by side, redirecting the symlinks a command at a time until the older |
| 177 | package is no longer used, and can be removed.)</p> |
| 178 | |
| 179 | <p>This use case includes running init scripts and other shell scripts, running |
| 180 | configure, make, and install in each package, and providing basic command line |
| 181 | facilities such as a text editor. (It does not include a compiler toolchain or |
| 182 | C library, those are outside the scope of this project.)</p> |
| 183 | |
| 184 | <blockquote><b> |
| 185 | <span id=development> |
| 186 | bzcat cat cp dirname echo env patch rmdir sha1sum sleep sort sync |
| 187 | true uname wc which yes zcat |
| 188 | awk basename bzip2 chmod chown cmp cut date dd diff |
| 189 | egrep expr find grep gzip head hostname id install ln ls |
| 190 | mkdir mktemp mv od readlink rm sed sh tail tar touch tr uniq |
| 191 | wget whoami xargs chgrp comm gunzip less logname man split |
| 192 | tee test time bunzip2 chgrp chroot comm cpio dmesg |
| 193 | dnsdomainname ftpd ftpget ftpput gunzip ifconfig init less |
| 194 | logname losetup man mdev mount mountpoint nc pgrep pkill |
| 195 | pwd route split stat switch_root tac umount vi |
| 196 | </span> |
| 197 | </b></blockquote> |
| 198 | |
| 199 | <p>Note: Aboriginal Linux installs bash 2.05b as #!/bin/sh and its scripts |
| 200 | require bash extensions not present in shells such as busybox ash. |
| 201 | This means that toysh needs to supply several bash extensions _and_ work |
| 202 | when called under the name "bash".</p> |
| 203 | |
| 204 | <hr /> |
Rob Landley | 571b070 | 2012-11-13 16:13:45 -0600 | [diff] [blame] | 205 | <h2><a name=android /><a href="#android">Use case: Replacing Android Toolbox</a></h2> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 206 | |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 207 | <p>Android has a policy against GPL in userspace, so even though BusyBox |
| 208 | predates Android by many years, they couldn't use it. Instead they grabbed |
| 209 | an old version of ash and implemented their own command line utility set |
| 210 | called "toolbox".</p> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 211 | |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 212 | <p>Toolbox doesn't have its own repository, instead it's part of Android's |
| 213 | <a href=https://android.googlesource.com/platform/system/core>system/core |
| 214 | git repository</a> (this analysis looked at commit 51ccef27cab58).</p> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 215 | |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 216 | <h3>Toolbox commands:</h3> |
| 217 | |
| 218 | <p>According to core/toolbox/Android.mk the toolbox directory builds the |
| 219 | following commands:</p> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 220 | |
| 221 | <blockquote><b> |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 222 | ls mount cat ps kill ln insmod rmmod lsmod ifconfig setconsole |
| 223 | rm mkdir rmdir reboot getevent sendevent date wipe sync umount |
| 224 | start stop notify cmp dmesg route hd dd df getprop setprop watchprops |
| 225 | log sleep renice printenv smd chmod chown newfs_msdos netstat ioctl |
| 226 | mv schedtop top iftop id uptime vmstat nandread ionice touch lsof md5 r |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 227 | cp du grep watchdogd |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 228 | </b></blockquote> |
| 229 | |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 230 | <p>If selinux is enabled, you also get:</p> |
| 231 | <blockquote><b> |
| 232 | getenforce setenforce chcon restorecon runcon getsebool setsebool load_policy |
| 233 | </b></blockquote> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 234 | |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 235 | <p>The Android.mk file also refers to dynarray.c and toolbox.c as library |
| 236 | code. This leaves the following apparently unused C files in toolbox/*.c, each |
| 237 | of which has a command_main() function and seems to implement a standalone |
| 238 | command:</p> |
| 239 | |
| 240 | <blockquote><b> |
| 241 | alarm exists lsusb readtty rotatefb setkey syren |
| 242 | </b></blockquote> |
| 243 | |
| 244 | <h3>Command shell (ash)</h3> |
| 245 | |
| 246 | <p>The core/sh subdirectory contains a fork of ash 1.17, and sucks in |
| 247 | liblinenoise to provide command line history/editing.</p> |
| 248 | |
| 249 | <h3>Other Android core commands</h3> |
| 250 | |
| 251 | <p>Other than the toolbox and sh directories, the currently interesting |
| 252 | subdirectories in the core repository are fs_mgr, gpttool, init, |
| 253 | logcat, logwrapper, mkbootimg, netcfg, run-as, and sdcard.</p> |
| 254 | |
| 255 | <ul> |
| 256 | <li><b>fs_mgr</b> - subset of mount</li> |
| 257 | <li><b>gpttool</b> - subset of fdisk</li> |
| 258 | <li><b>init</b> - Android's PID 1</li> |
| 259 | <li><b>logcat</b> - read android log format</li> |
| 260 | <li><b>logwrapper</b> - redirect stdio to android log</li> |
| 261 | <li><b>mkbootimg</b> - create signed boot image</li> |
| 262 | <li><b>netcfg</b> - network configuration (sucks in libnetutils)</li> |
| 263 | <li><b>run-as</b> - subset of sudo</li> |
| 264 | <li><b>sdcard</b> - FUSE wrapper to squash UID/GID/permissions to what FAT supports.</li> |
| 265 | </ul> |
| 266 | |
| 267 | <p>Almost all of these reinvent an existing wheel with less functionality and a |
| 268 | different user interface. We may want to provide that interface, but |
| 269 | implementing the full commands (mount, fdisk, init, ifconfig with dhcp, |
| 270 | and sudo) come first.</p> |
| 271 | |
| 272 | <p>Although logcat/logwrapper also reinvent a wheel, Android did so in the |
| 273 | kernel and these provide an interface to that.</p> |
| 274 | |
| 275 | <p>Also, gpttool and mkbootimg are install tools, and sdcard looks like a |
| 276 | testing tool. These aren't a priority if android wants to use its own |
| 277 | bespoke code to install itself.</p> |
| 278 | |
| 279 | <h3>Analysis</h3> |
| 280 | |
| 281 | <p>For reference, combining everything listed above, we get:</p> |
| 282 | |
| 283 | <blockquote><b> |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 284 | alarm ash cat chcon chmod chown cmp cp date dd df dmesg du exists fs_mgr |
| 285 | getenforce |
| 286 | getevent getprop getsebool gpttool grep hd id ifconfig iftop init insmod ioctl |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 287 | ionice kill ln load_policy log logcat logwrapper ls lsmod lsof lsusb md5 |
| 288 | mkbootimg mkdir mount mv nandread netcfg netstat newfs_msdos notify printenv |
| 289 | ps r readtty reboot renice restorecon rm rmdir rmmod rotatefb route run-as |
| 290 | runcon schedtop sdcard sendevent setconsole setenforce setkey setprop setsebool |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 291 | sleep smd start stop sync syren top touch umount uptime vmstat watchdogd |
| 292 | watchprops wipe |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 293 | </b></blockquote> |
| 294 | |
| 295 | <p>We may eventually implement all of that, but for toybox 1.0 we need to |
| 296 | focus a bit. For our first pass, let's ignore selinux, strip out the "unlisted" |
| 297 | commands except lsusb, and grab just logcat and logwrapper from the "core" |
| 298 | commands (since the rest have some full/standard version providing that |
| 299 | functionality, which we can implement a shim interface for later).</p> |
| 300 | |
| 301 | <p>This means toybox should implement:</p> |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 302 | <blockquote><b> |
| 303 | <span id=toolbox> |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 304 | cat chmod chown cmp cp date dd df dmesg du getevent getprop grep hd id ifconfig |
| 305 | iftop insmod ioctl ionice kill ln log logcat logwrapper ls lsmod lsof lsusb md5 |
| 306 | mkdir mount mv nandread |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 307 | netstat newfs_msdos notify printenv ps r reboot renice rm rmdir rmmod route |
| 308 | schedtop sendevent setconsole setprop sleep smd start stop sync top touch |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 309 | umount uptime vmstat watchprops watchdogd wipe |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 310 | </span> |
Rob Landley | c26ca6e | 2013-01-31 04:05:56 -0600 | [diff] [blame] | 311 | </b></blockquote> |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 312 | |
| 313 | <p>The following Toolbox commands are already covered in previous |
| 314 | sections of this analysis:</p> |
| 315 | |
| 316 | <blockquote><b> |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 317 | cat chmod chown cmp cp date dd df dmesg du grep id ifconfig insmod kill ln ls |
| 318 | lsmod mkdir mount mv ps renice rm rmdir rmmod route sleep sync top touch umount |
Rob Landley | fdc10c9 | 2012-10-16 17:09:30 -0500 | [diff] [blame] | 319 | </b></blockquote> |
| 320 | |
| 321 | <p>Which leaves the following commands as new from Toolbox:</p> |
| 322 | |
| 323 | <blockquote><b> |
| 324 | getevent getprop hd iftop ioctl ionice log lsof nandread netstat |
| 325 | newfs_msdos notify printenv r reboot schedtop sendevent setconsole |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 326 | setprop smd start stop top uptime vmstat watchprops watchdogd wipe |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 327 | </b></blockquote> |
| 328 | |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 329 | <hr /><a name=klibc /> |
Rob Landley | 934b2d3 | 2013-05-10 18:54:14 -0500 | [diff] [blame] | 330 | <h2>klibc:</h2> |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 331 | |
| 332 | <p>Long ago some kernel developers came up with a project called |
| 333 | <a href=http://en.wikipedia.org/wiki/Klibc>klibc</a>. |
| 334 | After a decade of development it still has no web page or HOWTO, |
| 335 | and nobody's quite sure if the license is BSD or GPL. It inexplicably |
| 336 | <a href=http://www.infoworld.com/d/data-center/perl-isnt-going-anywhere-better-or-worse-211580>requires perl to build</a>, and seems like an ideal candidate for |
| 337 | replacement.</p> |
| 338 | |
| 339 | <p>In addition to a C library even less capable than bionic (obsoleted by |
| 340 | musl), klibc builds a random assortment of executables to run init scripts |
| 341 | with. There's no multiplexer command, these are individual executables:</p> |
| 342 | |
| 343 | <blockquote><p> |
| 344 | cat chroot cpio dd dmesg false fixdep fstype gunzip gzip halt ipconfig kill |
| 345 | kinit ln losetup ls minips mkdir mkfifo mknodes |
| 346 | mksyntax mount mv nfsmount nuke pivot_root poweroff readlink reboot resume |
| 347 | run-init sh sha1hash sleep sync true umount uname zcat |
| 348 | </p></blockquote> |
| 349 | |
| 350 | <p>To get that list, build klibc according to the instructions (I |
| 351 | <a href=http://landley.net/notes-2013.html#23-01-2013>looked at</a> version |
| 352 | 2.0.2 and did cd klibc-*; ln -s /output/of/kernel/make/headers_install |
| 353 | linux; make) then <b>echo $(for i in $(find . -type f); do file $i | grep -q |
| 354 | executable && basename $i; done | grep -v '[.]g$' | sort -u)</b> to find |
| 355 | executables, then eliminated the *.so files and *.shared duplicates.</p> |
| 356 | |
| 357 | <p>Some of those binaries are build-time tools that don't get installed, |
| 358 | which removes mknodes, mksyntax, sha1hash, and fixdep from the list. |
| 359 | (And sha1hash is just an unpolished sha1sum anyway.)</p> |
| 360 | |
| 361 | <p>The run-init command is more commonly called switch_root, nuke is just |
| 362 | "rm -rf -- $@", and minips is more commonly called "ps". I'm not doing aliases |
| 363 | for the oddball names.</p> |
| 364 | |
| 365 | <p>Yet more stale forks of dash and gzip sucked in here (see "dubious |
| 366 | license terms" above), adding nothing to the other projects we've looked at. |
| 367 | But we still need sh, gunzip, gzip, and zcat to replace this package.</p> |
| 368 | |
| 369 | <p>By the time I did the analysis toybox already had cat, chroot, dmesg, false, |
| 370 | kill, ln, losetup, ls, mkdir, mkfifo, readlink, rm, switch_root, sleep, sync, |
| 371 | true, and uname.</p> |
| 372 | |
| 373 | <p>The low hanging fruit is cpio, dd, ps, mv, and pivot_root.</p> |
| 374 | |
| 375 | <p>The "kinit" command is another gratuitous rename, it's init running as PID 1. |
| 376 | The halt, poweroff, and reboot commands work with it.</p> |
| 377 | |
| 378 | <p>I've got mount and umount queued up already, fstype and nfsmount go with |
| 379 | those. (And probably smbmount and p9mount, but this hasn't got one. Those |
| 380 | are all about querying for login credentials, probably workable into the |
| 381 | base mount command.)</p> |
| 382 | |
| 383 | <p>The ipconfig command here has a built in dhcp client, so it's ifconfig |
| 384 | and dhcpcd and maybe some other stuff.</p> |
| 385 | |
| 386 | <p>The resume command is... weird. It finds a swap partition and reads data |
| 387 | from it into a /proc file, something the kernel is capable of doing itself. |
| 388 | (Even though the klibc author |
| 389 | <a href=http://www.zytor.com/pipermail/klibc/2006-June/001748.html>attempted |
| 390 | to remove</a> that capability from the kernel, current kernel/power/hibernate.c |
| 391 | still parses "resume=" on the command line). And yet various distros seem to |
| 392 | make use of klibc for this> |
| 393 | Given the history of swsusp/hibernate (and |
| 394 | <a href=http://lwn.net/Articles/333007>TuxOnIce</a> |
| 395 | and <a href=http://lwn.net/Articles/242107>kexec jump</a>) I've lost track |
| 396 | of the current state of the art here. Ah, Documentation/power/userland-swsusp.txt |
| 397 | has the API docs, and <a href=http://suspend.sf.net>here's a better |
| 398 | tool</a>...</p> |
| 399 | |
| 400 | <p>So the list of things actually in klibc are:</p> |
| 401 | |
| 402 | <blockquote><b> |
Rob Landley | c166faf | 2013-09-01 07:25:37 -0500 | [diff] [blame] | 403 | <span id=klibc_cmd> |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 404 | cat chroot dmesg false kill ln losetup ls mkdir mkfifo readlink rm switch_root |
| 405 | sleep sync true uname |
| 406 | |
| 407 | cpio dd ps mv pivot_root |
| 408 | mount nfsmount fstype umount |
| 409 | sh gunzip gzip zcat |
| 410 | kinit halt poweroff reboot |
| 411 | ipconfig |
| 412 | resume |
| 413 | </span> |
| 414 | </b></blockquote> |
| 415 | |
| 416 | <hr /> |
| 417 | <a name=sash /> |
| 418 | <h2>Stand-Alone Shell</h2> |
| 419 | |
| 420 | <p>Wikipedia has <a href=http://en.wikipedia.org/wiki/Stand-alone_shell>a good |
| 421 | summary of sash</a>, with links. The original Stand-Alone Shell project reached |
| 422 | a stopping point, and then <a href=http://www.baiti.net/sash>"sash plus |
| 423 | patches"</a> extended it a bit further. The result is a megabyte executable |
| 424 | that provides 40 commands.</p> |
| 425 | |
| 426 | <p>Sash is a shell with built-in commands. It doesn't have a multiplexer |
| 427 | command, meaning "sash ls -l" doesn't work (you have to go "sash -c 'ls -l'"). |
| 428 | </p> |
| 429 | |
| 430 | <p>The list of commands can be obtained via building it and doing |
| 431 | "echo help | ./sash | awk '{print $1}' | sed 's/^-//' | xargs echo", which |
| 432 | gives us:</p> |
| 433 | |
| 434 | <blockquote><b> |
| 435 | alias aliasall ar cd chattr chgrp chmod chown cmp cp chroot dd echo ed exec |
| 436 | exit file find grep gunzip gzip help kill losetup losetup ln ls lsattr mkdir |
| 437 | mknod more mount mv pivot_root printenv prompt pwd quit rm rmdir setenv source |
| 438 | sum sync tar touch umask umount unalias where |
| 439 | </b></blockquote> |
| 440 | |
| 441 | <p>Plus sh because it's a shell. A dozen or so commands can only sanely be |
| 442 | implemented as shell builtins (alias aliasall cd exec exit prompt quit setenv |
| 443 | source umask unalias), where is an alias for which, and at triage time toybox |
| 444 | already has chgrp, chmod, chown, cmp, cp, chroot, echo, help, kill, losetup, |
| 445 | ln, ls, mkdir, mknod, printenv, pwd, rm, rmdir, sync, and touch.</p> |
| 446 | |
| 447 | <p>This leaves:</p> |
| 448 | |
| 449 | <blockquote><b> |
| 450 | <span id=sash_cmd> |
| 451 | ar chattr dd ed file find grep gunzip gzip lsattr more mount mv pivot_root |
| 452 | sh sum tar umount |
| 453 | </span> |
| 454 | </b></blockquote> |
| 455 | |
| 456 | <p>(For once, this project doesn't include a fork of gzip, instead |
| 457 | it sucks in -lz from the host.)</p> |
| 458 | |
| 459 | <hr /> |
| 460 | <a name=sbase /> |
| 461 | <h2>sbase:</h2> |
| 462 | |
| 463 | <p>It's <a href=http://git.suckless.org/sbase>on suckless</a>. So far it's |
| 464 | implemented:</p> |
| 465 | |
| 466 | <blockquote><p> |
Rob Landley | c166faf | 2013-09-01 07:25:37 -0500 | [diff] [blame] | 467 | <span id=sbase_cmd> |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 468 | basename cat chmod chown cksum cmp cp date dirname echo false fold grep head |
| 469 | kill ln ls mc mkdir mkfifo mv nl nohup pwd rm seq sleep sort tail tee test |
| 470 | touch true tty uname uniq wc yes |
| 471 | </span> |
| 472 | </p></blockquote> |
| 473 | |
| 474 | <p>And has a TODO list:</p> |
| 475 | |
| 476 | <blockquote><p> |
Rob Landley | c166faf | 2013-09-01 07:25:37 -0500 | [diff] [blame] | 477 | <span id=sbase_cmd> |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 478 | cal chgrp chvt comm cut df diff du env expand expr id md5sum nice paste |
| 479 | printenv printf readlink rmdir seq sha1sum split sync test tr unexpand unlink |
| 480 | who |
| 481 | </span> |
| 482 | </p></blockquote> |
| 483 | |
| 484 | <p>At triage time, of the first list I still need to do: fold grep mc mv nl. Of |
| 485 | the second list: diff expr paste printf split test tr unexpand who.</p> |
| 486 | |
| 487 | <hr /> |
| 488 | <a name=s6 /> |
| 489 | <h2>s6</h2> |
| 490 | |
| 491 | <p>The website <a href=http://skarnet.org/software/>skarnet</a> has a bunch |
| 492 | of small utilities as part of something called "s6". This includes the |
| 493 | <a href=http://skarnet.org/software/s6-portable-utils>s6-portabile-utils</a> |
| 494 | and the <a href=http://skarnet.org/software/s6-linux-utils>s6-linux-utils</a>. |
| 495 | </p> |
| 496 | |
| 497 | <p>Both packages rely on multiple bespoke external libraries without which |
| 498 | they can't compile. The source is completely uncommented and doesn't wrap at |
| 499 | 80 characters. Doing a find for *.c files brings up the following commands:</p> |
| 500 | |
| 501 | <blockquote><b> |
| 502 | <span id=s6> |
| 503 | basename cat chmod chown chroot clock cut devd dirname echo env expr false |
| 504 | format-filter freeramdisk grep halt head hiercopy hostname linkname ln |
| 505 | logwatch ls maximumtime memoryhog mkdir mkfifo mount nice nuke pause |
| 506 | pivotchroot poweroff printenv quote quote-filter reboot rename rmrf sleep |
| 507 | sort swapoff swapon sync tail test touch true umount uniquename unquote |
| 508 | unquote-filter update-symlinks |
| 509 | </span> |
| 510 | </b></blockquote> |
| 511 | |
| 512 | <p>Triage: memoryhog isn't even listed on the website nor does it have |
| 513 | a documentation file, clock seems like a subset |
| 514 | of date, devd is some sort of netlink wrapper that spawns its command line |
| 515 | every time it gets a message (maybe this is meant to implement part of |
| 516 | udev/mdev?), format-filter is sort of awk's '{print $2}' function split out |
| 517 | into its own command, hiercopy a subset of "cp -r", maximumtime is something |
| 518 | I implemented as a shell script (more/timeout.sh in Aboriginal Linux), |
| 519 | nuke isn't the same as klibc (this one's "kill SIG -1" only with hardwared |
| 520 | SIG options), pause is a program that literally waits to be killed (I |
| 521 | generally sleep 999999999 which is a little over 30 years), |
| 522 | pivotchroot is a subset of switch_root, rmrf is rm -rf...</p> |
| 523 | |
| 524 | <p>I see "nuke" resurface, and if "rmrf" wasn't also here I might think |
| 525 | klibc had a point.</b> |
| 526 | |
| 527 | <blockquote> |
| 528 | basename cat chmod chown chroot cut dirname echo env expr false |
| 529 | freeramdisk grep halt head hostname linkname ln |
| 530 | logwatch ls mkdir mkfifo mount nice |
| 531 | pivotchroot poweroff printenv quote quote-filter reboot rename sleep |
| 532 | sort swapoff swapon sync tail test touch true umount uniquename unquote |
| 533 | unquote-filter update-symlinks |
| 534 | </blockquote> |
| 535 | |
| 536 | |
| 537 | <hr /> |
| 538 | <a name=nash /> |
| 539 | <h2>nash:</h2> |
| 540 | |
| 541 | <p>Red Hat's nash was part of its "mkinitrd" package, replacement for a shell |
| 542 | and utilities on the boot floppy back in the 1990's (the same general idea |
| 543 | as BusyBox, developed independently). Red Hat discontinued nash development |
| 544 | in 2010, replacing it with dracut (which collects together existing packages, |
| 545 | including busybox).</p> |
| 546 | |
| 547 | <p>I couldn't figure out how to beat source code out of |
| 548 | <a href=http://pkgs.fedoraproject.org/git/mkinitrd>Fedora's current git</a> |
| 549 | repository. The last release version that used it was Fedora Core 12 |
| 550 | which has <a href=http://archive.fedoraproject.org/pub/archive/fedora/linux/releases/12/Fedora/source/SRPMS/mkinitrd-6.0.93-1.fc12.src.rpm>a source rpm</a> |
| 551 | that can be unwound with "rpm2cpio mkinitrd.src.rpm | cpio -i -d -H newc |
| 552 | --no-absolute-filenames" and in there is a mkinitrd-6.0.93.tar.bz2 which |
| 553 | has the source.</p> |
| 554 | |
| 555 | <p>In addition to being a bit like a command shell, the nash man page lists the |
| 556 | following commands:</p> |
| 557 | |
| 558 | <blockquote><p> |
| 559 | access echo find losetup mkdevices mkdir mknod mkdmnod mkrootdev mount |
| 560 | pivot_root readlink raidautorun setquiet showlabels sleep switchroot umount |
| 561 | </p></blockquote> |
| 562 | |
| 563 | <p>Oddly, the only occurrence of the string pivot_root in the nash source code |
| 564 | is in the man page, the command isn't there. (It seems to have been removed |
| 565 | when the underscoreless switchroot went in.)</p> |
| 566 | |
| 567 | <p>A more complete list seems to be the handlers[] array in nash.c:</p> |
| 568 | |
| 569 | <blockquote><p> |
| 570 | access buildEnv cat cond cp daemonize dm echo exec exit find kernelopt |
| 571 | loadDrivers loadpolicy mkchardevs mkblktab mkblkdevs mkdir mkdmnod mknod |
| 572 | mkrootdev mount netname network null plymouth hotplug killplug losetup |
| 573 | ln ls raidautorun readlink resume resolveDevice rmparts setDeviceEnv |
| 574 | setquiet setuproot showelfinterp showlabels sleep stabilized status switchroot |
| 575 | umount waitdev |
| 576 | </p></blockquote> |
| 577 | |
| 578 | <p>This list is nuts: "plymouth" is an alias for "null" which is basically |
| 579 | "true" (which thie above list doesn't have). Things like buildEnv and |
| 580 | loadDrivers are bespoke Red Hat behavior that might as well be hardwired in |
| 581 | to nash's main() without being called.</p> |
| 582 | |
| 583 | <p>Instead of eliminating items |
| 584 | from the list with an explanation for each, I'm just going to cherry pick |
| 585 | a few: the device mapper (dm, raidautorun) is probably interesting, |
| 586 | hotplug (may be obsolete due to kernel changes that now load firmware |
| 587 | directly), and another "resume" ala klibc.</p> |
| 588 | |
| 589 | <p>But mostly: I don't care about this one. And neither does Red Hat anymore.</p> |
| 590 | |
Rob Landley | ca04c7f | 2013-03-27 22:34:28 -0500 | [diff] [blame] | 591 | <p>Verdict: ignore</p> |
| 592 | |
Rob Landley | 9a69a92 | 2013-02-23 18:32:08 -0600 | [diff] [blame] | 593 | <hr /> |
| 594 | <a name=beastiebox /> |
| 595 | <h2>Beastiebox</h2> |
| 596 | |
| 597 | <p>Back in 2008, the BSD guys vented some busybox-envy |
| 598 | <a href=http://beastiebox.sourceforge.net>on sourceforge</a>. Then stopped. |
| 599 | Their repository is still in CVS, hasn't been touched in years, it's a giant |
| 600 | hairball of existing code sucked together. (The web page says the author |
| 601 | is aware of crunchgen, but decided to do this by hand anyway. This is not |
| 602 | a collection of new code, it's a katamari of existing code rolled up in a |
| 603 | ball.)</p> |
| 604 | |
| 605 | <p>Combining the set of commands listed on the web page with the set of |
| 606 | man pages in the source gives us:</P> |
| 607 | |
| 608 | <blockquote><p> |
| 609 | [ cat chmod cp csh date df disklabel dmesg echo ex fdisk fsck fsck_ffs getty |
| 610 | halt hostname ifconfig init kill less lesskey ln login ls lv mksh more mount |
| 611 | mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh stty sysctl tar test |
| 612 | traceroute umount vi wiconfig |
| 613 | </p></blockquote> |
| 614 | |
| 615 | <p>Apparently lv is the missing link ed and vi, copyright 1982-1997 (do not |
| 616 | want), ex is another obsolete vi mode, lesskey is "used to |
| 617 | specify a set of key bindings to be used with less", and csh is a shell they |
| 618 | sucked in, [ is an alias for test. Several more bsd-isms that don't have Linux |
| 619 | equivalents (even in the ubuntu "install this package" search) are |
| 620 | disklabel, fsck_ffs, mount_ffs, and pfctl. And wiconfig is a wavelan interface |
| 621 | network card driver utility. Subtracting all that and the commands toybox |
| 622 | already implements at triage time, we get:</p> |
| 623 | |
| 624 | <blockquote><p> |
| 625 | <span id=beastiebox_cmd> |
| 626 | fdisk fsck getty halt ifconfig init kill less mksh more mount mv ping poweroff |
| 627 | ps reboot route sed sh stty sysctl tar test traceroute umount vi |
| 628 | </span> |
| 629 | </p></blockquote> |
| 630 | |
| 631 | <p>Not a hugely interesting list, but eh.</p> |
| 632 | |
Rob Landley | ca04c7f | 2013-03-27 22:34:28 -0500 | [diff] [blame] | 633 | <p>Verdict: ignore</p> |
| 634 | |
| 635 | <hr /> |
| 636 | <a name=BsdBox /> |
| 637 | <h2>BsdBox</h2> |
| 638 | |
| 639 | <p>Somebody decided to do a <a href=https://wiki.freebsd.org/AdrianChadd/BsdBox>multicall binary for freebsd</a>.</p> |
| 640 | |
| 641 | <p>They based it on crunchgen, a tool that glues existing programs together |
| 642 | into an archive and uses the name to execute the right one. It has no |
| 643 | simplification or code sharing benefits whatsoever, it's basically an |
| 644 | archiver that produces executables.</p> |
| 645 | |
| 646 | <p>That's about where I stopped reading.</p> |
| 647 | |
| 648 | <p>Verdict: ignore.</p> |
| 649 | |
| 650 | <hr /> |
| 651 | <a name=slowaris /> |
| 652 | <h2>OpenSolaris Busybox</h2> |
| 653 | |
| 654 | <p>Somebody <a href=http://hub.opensolaris.org/bin/view/Project+busybox/>wrote |
| 655 | a wiki page</a> saying that Busybox for OpenSolaris would be a good idea.</p> |
| 656 | |
| 657 | <p>The corresponding "files" tab is an auto-generated stub. The project never |
| 658 | even got as far as suggesting commands to include before Oracle discontinued |
| 659 | OpenSolaris.</p> |
| 660 | |
| 661 | <p>Verdict: ignore.</p> |
| 662 | |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 663 | <hr /> |
| 664 | <h2>Requests:</h2> |
| 665 | |
| 666 | <p>The following additional commands have been requested by various users:</p> |
| 667 | <blockquote><b> |
| 668 | <span id=request> |
Rob Landley | 64b6319 | 2013-12-31 09:01:32 -0600 | [diff] [blame] | 669 | dig freeramdisk getty halt hexdump hwclock klogd modprobe ping ping6 pivot_root |
Rob Landley | ca04c7f | 2013-03-27 22:34:28 -0500 | [diff] [blame] | 670 | poweroff readahead rev sfdisk sudo syslogd taskset telnet telnetd tracepath |
| 671 | traceroute unzip usleep vconfig zip free login modinfo unshare netcat help w |
Rob Landley | 64b6319 | 2013-12-31 09:01:32 -0600 | [diff] [blame] | 672 | ntpd iwconfig iwlist rdate |
Rob Landley | 8f90d3a | 2012-07-21 23:58:40 -0500 | [diff] [blame] | 673 | </span> |
| 674 | </b></blockquote> |
| 675 | |
| 676 | <!-- #include "footer.html" --> |
| 677 | |