blob: c9bf1236bc6c6960acfc00c4a84e838059526a99 [file] [log] [blame]
Rob Landley349ff522014-01-04 13:09:42 -06001<html><head><title>toybox news</title>
Rob Landley589f5cd2010-01-05 10:41:52 -06002<!--#include file="header.html" -->
3
Rob Landley5cab9942013-09-18 10:50:38 -05004<p>Toybox combines common Linux command line utilities together
Rob Landleye566f3a2013-06-16 20:02:38 -05005into a single BSD-licensed executable that's simple, small, fast,
6reasonably standards-compliant, and powerful enough to turn Android into
Rob Landley49f8d6f2013-07-26 13:04:21 -05007a development environment. See the links on the left for details.</p>
8
9<h2>News</h2>
Rob Landley08e18fb2014-04-20 14:22:19 -050010
Rob Landleyba3e32b2015-02-27 08:13:24 -060011<hr><b>February 25, 2015</b>
12<blockquote><p>"A common mistake that people make when trying to design
13something completely foolproof is to underestimate the ingenuity of
14complete fools."</p><p>- The Hitchhiker's Guide to the Galaxy.</p></blockquote>
15
16<p><a href=downloads/toybox-0.5.2.tar.gz>Toybox 0.5.2</a>
17(<a href=/hg/toybox/shortlog/1702>commit 1702</a>) is out.</p>
18
19<p>New promoted commands: sed (finally fixed enough it builds Linux From
20Scratch), printf (cleaned up and promoted), shred and
21base64 (the Tizen guys wanted them), getenforce, setenforce, and chcon (android),
22mix (promoted with fixes from Isaac Dunham), nsenter (from
23Andy Lutomirski, merged into unshare).</p>
24
25<p>Elliott Hughes submited a bunch of patches to support Android (to
26both toybox and Bionic libc, which he maintains). On toybox's end this
27involved a lot of fixups to portability.[ch] and fixes to over a dozen
28commands, plus several new ones. Other portability fixes included working
29with buildroot's uclibc fork and building for nommu targets.</p>
30
31<p>The new "make change" target builds each toybox command as a standalone
32binary. Rather a lot of commands that didn't build by themselves (mv depending
33on cp and so on) were hit with a large rock until they built standalone.
34This involved rewriting bits of option parsing, more elaborate dependency
35generation, making each command have its own config
36symbol and main() function (even when it's just a wrapper calling another
37command's main()), and so on. Also, some commands can't be built standalone
38at a conceptual level: "help" describes other enabled commands and "sh"
39has a number of bulitin commands (cd, exit, set) that require the
40multiplexer infrastructure, so "make change" filters them out.</p>
41
42<p>The mailing list's web archive is still screwed up. Dreamhost has
43been trying to fix it since approximately September. There are
44<a href=http://www.mail-archive.com/toybox@lists.landley.net/>two</a>
45<a href=http://news.gmane.org/gmane.linux.toybox>other</a> less broken
46archives, but neither has quite the same UI as mailman.</p>
47
48<h3>Bugfixes and tweaks</h3>
49
50<p>Cynt Rynt sent in tests for ifconfig,
51Robert Thompson taught factor to accept whitespace separated arguments,
52Hyejin Kim pointed out that some of mktemp's longopts were attached to
53the wrong short options,
54Luis Felipe Strano Moraes fixed a wrong free() call in bootchartd in pending.
55Patches from Ashwini Sharma to make "df /dev/node" work, prevent du from
56looping endlessly following symlinks, and to make expr.c
57(in pending) understand == and regex matches. (Speaking of expr, it gets
58priority groupings wrong but the bug was actually in the posix spec's
59HTML conversion. They fixed the posix spec upstream for us. Still need
60to fix the expr code, but it's in pending for a reason...)</p>
61
62<p>Some commands grew new option flags, such as cp --remove-destination
63and touch -h.</p>
64
65<p>The parallel build has better error reporting now. When toybox needs to
66re-exec itself to regain suid root permissions and hasn't got the suid bit,
67it now gives the right error message ("not root" instead of "no such command").
68
69<p>Added a test to "mount" to not mount the same device/directory combination
70over itself (the OS catches this for block devices, but not for tmpfs).
71Make blkid distinguish ext3 from ext4. Added catv back into cat (because
72the Android guys wanted it, and they have historical usage on their side,
73so...). Handle nanoseconds in touch.</p>
74
75<p>Fixed a segfault when CP_MORE was disabled (the resulting option flag list
76no longer defined -d but still had it in option groups at the end).
77Workaround for glibc redefining dirname() and basename() to random non-posix
78semantics because gnu. (They could have created dirname_r() but didn't want
79to.)</p>
80
81<p>Fix an ifconfig test that was preventing assigning an ipv4 address to
82interface aliases. Several cleanup passes on hwclock but not quite
83promoted out of pending yet.<p>
84
85<p>Fixed a wrong error message in rm (if you had a chmod 000 directory and
86did rm -r on it without -f, after the prompt it would complain it was a
87directory, which was not the problem).</p>
88
89<p>The gzip compression code now does "store only" output to stdout, for
90what that's worth.</p>
91
92<p>Cleanup mountpoint and expand, and remove them from toys/pending/README
93(a list of commands that predate the toys/pending directory but needed
94another pass).</p>
95
96<h3>Library and infrastructure:</h3>
97
98<p>Reworked the option parsing infrastructure so more commands build
99standalone (via scripts/single.sh or "make change"). The option flag bit
100values are no longer packed, it leaves spaces where currently disabled
101flags go, and you can #define FORCE_FLAGS so disabled flags aren't zeroed.
102This allows multiple commands to more easily share infrastructure, even if
103your current flag context is for a disabled command (switched off in config),
104you can force them to stay on and as long as the flags read the same right
105to left they'll have the same values.</p>
106
107<p>We've started removing use of strncpy() because it's a hugely broken
108standard C function: the length is the maximum length to _append_, not
109the size of the destination buffer. It memsets the remaining space it didn't
110copy ala "memset(dest+strlen(dest), 0, len);" so
111if you think len is the size of dest you're guaranteed to stomp memory off the
112end). And if it runs out of space it won't null terminate because reasons.
113(Meanwhile sprintf("%*s", len, str) is counting wide characters in your current
114locale, so if you set a locale other than "C" it will also go past your
115allocated buffer size. Whoever is maintining the C library standards is really
116bad at strings.)
117Instead we have xstrncat() which will error_exit() if src+dest+1 doesn't
118fit in the buffer. (Because randomly truncating input data isn't necessarily
119an improvement.) And there's always xmprintf().</p>
120
121<p>Similarly, strtol() doesn't return an error indicator on overflow,
122you have to clear and then check errno. So new xstrtol() that cares
123about overflow.</p>
124
125<p>The bionic and musl guys agree faccessat(AT_SYMLINK_NOFOLLOW) is not
126supported, so stop using it.</p>
127
128<p>Fixed toy_exec() to detect when argc is in optargs, so we don't
129need a separate xexec_optargs().</p>
130
131<hr><b>February 18, 2015</b>
132<p>Dreamhost continues to be unable to make mailing list archives work, so
133here's <a href=http://www.mail-archive.com/toybox@lists.landley.net/>another
134list archive</a> with a less awkward interface than gmane.</p>
135
136<p>(Neither gives you the convenient historical monthly views of mailman,
137but I still have hopes dreamhost will someday figure out what they're doing
138wrong. They've only been trying since October. Last month they did a
139<a href=http://www.dreamhoststatus.com/2015/01/14/discussion-list-hardware-maintenance/>hardware upgrade to fix a software problem</a>, and the stale
140data loads much faster now, so that's something.)</p>
141
142<p>Update (Feb 19): the archive started updating again, by discarding
143all the pending data. So there are now _two_ giant holes in Dreamhost's
144web archive, from Dec 15-Jan 3, and then another hole from Jan 16-Feb 18.
145The relevant messages are in both of the other archives. Here's hoping
146the chronic archive constipation problem won't happen a sixth time.</p>
147
Rob Landley774c5992014-12-30 14:49:02 -0600148<hr><b>December 30, 2014</b>
149<p>Due to Dreamhost's <a href=http://landley.net/dreamhost.txt>ongoing</a>
150<a href=http://landley.net/dreamhost2.txt>inability</a> to make mailman
151work reliably, I've added a link to a backup web archive at
152<a href=http://news.gmane.org/gmane.linux.toybox>gmane</a> to the nav bar
153on the left.</p>
154
155<p>You still subscribe to the list through
156<a href=http://lists.landley.net/listinfo.cgi/toybox-landley.net>the first link</a>.</p>
157
Rob Landleyba3e32b2015-02-27 08:13:24 -0600158<p>Update (January 27, 2015): they're <a href=https://twitter.com/landley/status/558428839462703104>still working on it</a>.</p>
159
Rob Landleydc5bd762014-11-28 16:53:59 -0600160<hr><b>November 19, 2014</b>
161
162<blockquote><p>"This time it was right, it would work, and no one would have to get nailed to anything." - The Hitchhiker's Guide to the Galaxy.</p></blockquote>
163
164<p><a href=downloads/toybox-0.5.1.tar.bz2>Toybox 0.5.1</a>
165(<a href=/hg/toybox/shortlog/1566>commit 1566</a>) is out.</p>
166
167<p>It's an interim release, mostly bugfixes. There are several new commands,
168but they're all in pending.</p>
169
170<h3>Development</h3>
171
172<p>Finally implemented sed, which is still in pending because although
173it's feature complete according to posix, and even passes the parts of
174Busybox's sed test suite that aren't explicitly testing for gnu bugs we
175don't want to copy, it's not yet good enough to build Linux From Scratch.
176(The ./configure stages use very long sed scripts. 20 commits worth of
177implementation and debugging, just under 1000 lines of code, and there's
178still more to do. We're definitely up to some of the "fiddly" commands now.
179Did you know "echo hello | sed p - -" segfaults gnu sed in Ubuntu 12.04?
180Yeah...)</p>
181
182<p>Talked with the Tizen developers to follow up on their desire to
183make toybox a part of the base Tizen system, and got a list of commands
184to add to the roadmap. The tizen todo list is:</p>
185
186<blockquote><p>
187wget, sha256*, gzip, gunzip, bunzip2, rsync, zdiff*,
188less, ar, arch, base64, csplit, dir, fmt, join,
189nproc, shred, shuf, stdbuf, stty, test, tr, unexpand,
190users, vdir, diff3, sdiff, dosfsck (fsck.vfat), awk, fdisk
191</p></blockquote>
192
193<p>(Most of which was already on the todo list, but it helps prioritize.)</p>
194
195<p>Fixed md5sum and sha1sum on big endian systems (reported by James McMechan).
196Andy Lutomirski fixed unshare's help text and option parsing,
197and submitted nsenter (a tool to use setns(2)) to pending.
198Isaac Dunham implemented acpi -ctV options, and spotted the bug that ls -d
199was inappropraitely following command line symlinks without -H or -L (it
200should act like ls -l does), and ls -F handles symlinks wrong too.
201Lukasz Szpakowski sent in two bugfixes to tail.c. Cynt Rynt spotted an
202unnecessary assignment in lib/password.c.</p>
203
204<p>Ashwini Sharma's team was as busy as usual, submitting tr, crontab, and
205ipcrm, and hwclock to pending, more features to the pending ip.c, and a
206pile of bugfixes (to chgrp, killall, ifconfig, insmod,
207losetup, comm, cp, id, xwrap, netcat, modprobe, nohup...) mostly found by
208static analysis. (These fixes are mostly to seldom-used codepaths like the
209TOYBOX_FREE config option, but test coverage is always appreciated.) Ashwini
210also suggested upgrading ln -f to leave the original target alone if link
211creation fails, and reported that mv -f and -i weren't implemented (now fixed).</p>
212
213<p>New config option: TOYBOX_NORECURSE prevents xexec() from making internal
214function calls (for nommu systems with a finite stack).</p>
215
216<p>The "toybox" multiplexer command no longer adds a trailing space to each
217line of command names, so things like "./toybox | tr ' \n' '|'" to create
218a grep pattern snippet are easier to do. (Why you'd want to is your business,
219but the output is tidier now.)</p>
220
221<h3>Infrastructure</h3>
222
223<p>Isaac Dunham added Android support to portability.h, including compile
224probes for functions missing from bionic-libc, and annotated the commands that
225use those functions. We haven't really tested building against bionic,
226but in theory it's possible now.</p>
227
228<p>Running the test suite now color codes the PASS/SKIP/FAIL notifications
229if output is to a tty. (And in case you missed it last time, VERBOSE=fail
230to stop at the first failure is really useful.)</p>
231
232<p>In loopfiles_rw() use O_CLOEXEC instead of O_RDONLY to request the loop
233function close filehandles for us. (Otherwise the callback function must
234close each supplied filehandle itself.)</p>
235
236<p>The printf-style escape parsing ("\n" and friends) got factored out into
237a new unescape() function.</p>
238
Rob Landley037009f2014-10-02 07:53:27 -0500239<hr><b>October 2, 2014</b>
240<blockquote><p>"There is an art, it says, or rather, a knack to flying.
241The knack lies in learning how to throw yourself at the ground and miss...
242Clearly, it is this second part, the missing, which presents the
243difficulties." - The Hitchhiker's Guide to the Galaxy.<p></blockquote>
244
245<p><a href=downloads/toybox-0.5.0.tar.bz2>Toybox 0.5.0</a>
246(<a href=/hg/toybox/shortlog/1512>commit 1512</a>) is out.</p>
247
248<h3>New commands</h3>
249
250<p>The new commands are find, install, factor, and mount. Promoted commands
251(cleaned up and moved out of "pending") are lspci, inotifyd, and blockdev.</p>
252
253<p>cp now implements -HL and -F to force delete of pending files, cpio now
254ignores -m and implements -p, ls -C now has utf8 support (using wcwidth
255instead of strlen), and umount got a number of upgrades involving
256looking things up in /proc/mounts. Other minor cleanups happend to
257cut, touch, free, and id.</p>
258
259<p>In pending: Bradley Controy submitted mix (adjusts OSS sound volume). Ashwini
260Sharma submitted diff, userdel, blockdev, ipcs, and crond, upgraded
261fdisk, fsck, and ftpget, and ran a static analyzer on a lot of other code.
262Partial cleanup was done to useradd, userdel, groupadd, and groupdel.</p>
263
264<h3>Build infrastructure</h3>
265
266<p><b>Parallel builds</b></p>
267
268<p>The build now takes advantage of SMP, autodetecting the number of
269processors. (Export the environment variable CPUS to pick a specific number.)
270Other build changes: split out $LDOPTIMIZE because old compilers complain
271about linker options passed with -c, and the entire "generated" directory now
272gets deleted by clean (the README that was in there got merged into code.html).</p>
273
274<p><b>Standalone builds</b></p>
275
276<p>The standalone build infrastructure (scripts/single.sh) got upgraded to
277build more commands as standalone executables. In make.sh the source file
278selection uses a regex to find the source files with the NEWTOY/OLDTOY macro
279for the command. It enables each command's
280sub-options (so CP has CP_MORE), enables I18N and FLOAT support to build
281full-featured commands, and includes --help text (at least when
282the command doesn't use another command's help). The OLDTOY() macro
283now produces (redundant) function prototypes so you can build an OLDTOY
284without the NEWTOY</p>
285
286<p>It doesn't quite have complete coverage yet, the defconfig entries that
287aren't building standalone yet are:</p>
288
289<blockquote><p>chown, egrep, fgrep, fstype, halt, mv, nc, poweroff, unix2dos,
290whoami</p></blockquote>
291
292<p>The main reason for standalone build failures is NEWTOY() or OLDTOY()
293entries that don't have their own config symbol. Another problem is entries
294that depend on another entry in kconfig, usually because common infrastructure
295is using one command's flags (which the other commands copy): if that command
296is disabled, the FLAG macros become 0 so dead code elimination can remove the
297code. It's <a href=http://landley.net/hg/toybox/rev/1503>possible
298to untangle</a> this, but a bit awkward. (It boils down to conflicting
299design goals in the two contexts.)</p>
300
301<p>Standalone builds are used by the test suite when testing individual
302commands.
303
304<p><b>Snapshot builds</b></p>
305
306<p>A new addition to the "generated" directory is generated/build.sh
307containing a single compiler command line to build toybox in its current
308configuration. Combined with the generated/*.{h,sh} files from an
309exisiting build, this may let you build on a new system that hasn't quite
310got enough OS bits working to run a full configureand make.</p>
311
312<h3>Internals</h3>
313
314<p>Library code: xcreate/xopen now O_CLOEXEC by default to avoid leaking
315filehandles to child processes. DIRTREE_COMEAGAIN's second callback is now
316done with the directory filehandle still open (new dir->again variable added
317to distinguish first from second callback, and requesting DIRTREE_RECURSE now
318requires passing in the specific macro value, not just a true/false).
319Use daemon() out of libc instead of hand-rolled daemonize() in various
320pending commands. string_to_mode() now passes through type bits so you can
321use it to more easily modify a file's existing mode.
322Split xpopen() into xpopen_both(), xopen(), and xrun() depending on whether
323we want to redirect both, one, or neither of stdin/stdout.</p>
324
325<p>Bugfixes: Better error message when TOYBOX_SUID option can't drop priviliges
326(which happens when you suid something _other_ than root).
327The old pending version of nbd_client.c wasn't deleted when the
328command was promoted (and the build would break if both were enabled),
329toy_exec() sometimes needs to re-exec from $PATH rather than recurse
330internally (to gain dropped root permissions or limit stack depth),
331always call setlocale() when I18N is enabled to switch it back _off_ when
332we run commands that expect sscanf("%n") to return bytes,
333dirtree() had a memory leak in an error path, patch.c had some bugs in
334error paths (didn't report problem clearly). Ashwini Sharma spotted an
335option parsing bug where [-abc] would forget _all_ command line arguments
336saved in the GLOBALS() block (not just the ones for options being switched
337off), plus various minor fixes to nbd_client and cpio.
338Lukasz Szpakowski fixed rm -f on a broken symlink (failed), and killall
339with no arguments (segfaulted).</p>
340
341<p><b>Portability</b></p>
342
343<p>A somewhat fiddly fix to rm -rf (which needs to chmod directories to u+rwx
344to descend into them) which hit a musl bug in faccessat() which the musl
345maintainer refuses to fix. (He literally wants the man page changed
346instead, despite other libcs working.) Added an #ifdef __MUSL__ section
347to portability.h with a workaround, you may need CFLAGS=-D__MUSL__ in your
348build if your musl build's features.h doesn't #define that. (I may do
349a different workaround in future, but sometimes you've just got to make
350it work so you can ship. Also, toybox grep with multiple patterns
351requires <a href=http://landley.net/hg/aboriginal/rev/1692>a patch
352to musl's regex engine</a>, which applies to 1.1.4 but not to the current
353musl source control.)</p>
354
355<p>More portability.h fixes for uClibc too. (I don't expect that to ever have
356another release, so locally patching around posix-2008 violations is silly).</p>
357
358<p><b>Change to username filtering</b></p>
359
360<p>Posix recommends the username creation logic filter usernames to a small
361allowed set of characters (which even Red Hat breaks by explicitly allowing
362"$" at the end), but this prevents UTF-8 usernames. Posix' stated logic
363is to allow filesystems to create the user's home directory, but Linux
364filesystems can accept any character but NUL and "/". The only characters
365we actually _need_ to filter out are ":" (field separator in passwd),
366newline (line separator in passwd), and "/" (directory separator in
367filesystem).</p>
368
369<h3>Documentation</h3>
370
371<p>Web pages updated: cleanup.html documents more cleanup, code.html
372documents more code, and about.html now capitalizes "toybox" consistently
373(it's just a word, capitalize at start of sentence).</p>
374
375<p>The pending/README file now lists commands that needed review/cleanup
376before the pending directory was added.</p>
377
378<h3>Test Suite</h3>
379
380<p>Moved out of scripts/test into top level "tests" directory, and the
381testing.sh script is now in scripts rather than mixed into the *.test files.</p>
382
383<p>Johan Bergström requested VERBOSE=fail to make tests (telling it to
384stop at the first failure), and spotted a build bug where using gnu
385sort on the host broke in non-C locales.</p>
386
387<p>Divya Kothari submitted tests for chmod, link, tar, bzcat, xzcat, zcat,
388and hostname. (And more, but that's all that's merged so far.)</p>
389
Rob Landley8bae3142014-07-07 07:32:56 -0500390<hr><b>July 7, 2014</b>
391<blockquote><p>"This planet has - or rather had - a problem, which was this:
392most of the people living on it were unhappy for pretty much of the time. Many
393solutions were suggested for this problem, but most of these were largely
394concerned with the movement of small green pieces of paper, which was odd
395because on the whole it wasn't the small green pieces of paper that were
396unhappy." - The Hitchhiker's Guide to the Galaxy.</p></blockquote>
397
398<p><a href=downloads/toybox-0.4.9.tar.bz2>Toybox 0.4.9</a> (<a href=/hg/toybox/shortlog/1385>commit 1385</a>) is out.</p>
399
400<p><b>New commands</b> added to pending include:
401lsattr, chattr, inotifyd, rfkill, sulogin, strings, makedevs,
402killall5, and tar from Ashwini Sharma, arp from Kyungwan Han,
403sysctl by Bilal Qureshi, partprobe from Bertold Van den Bergh,
404host from Rich felker, and I did nbd-client and the first 2/3 of mount.</p>
405
406<p>Finished cleanups (commands promoted out of pending):
407sysctl, rfkill, strings, mkpasswd, makedevs, partprobe, killall5,
408fallocate, and nbd-client.</p>
409
410<p>(Along the way partial cleanups got made to: last, fold, lspci, ps,
411bootchartd, init, fsck, telnetd, telnet, vconfig, toysh, iconv, useradd,
412login, host, openvt, deallocvt, getty, tftpd, and modprobe. But there's
413still more to do on all of those.)</p>
414
415<p>This time around the <a href=bin>static binaries</a> are linked against
416musl instead of uClibc. (That's why there's no sparc version, musl doesn't
417support that target yet.)</p>
418
419<p><b>Documentation:</b></p>
420
421<p>The help text parser expects lower case "usage:" lines with
422a blank line after them, so go through and regularize those. Expand the
423"coding style" section in the docs and move it to design.html. (Not a show
424stopper for incoming
425contributions, just an explanation of some of the things I'll do to them
426during cleanup.) The help text for the "toybox" command now includes
427the shell script snippet to install symlinks to the toybox binary.</p>
428
429<p>The <a href=cleanup.html>cleanup page</a> now has descriptions for the
430full ifconfig cleanup series, among others.</p>
431
432<p>The new toys/examples directory contains hello.c and skeleton.c. The first is
433a simple hello world program in toybox style, the second is a much more
434elaborate example program using showing how to use the command line option
435parsing and how to provide multiple commands in the same C file.</p>
436
437<p><b>Fixes</b>:</p>
438
439<p>Fix od bug reported by Samuel Holland ("od -v -b" was appending the default
440output type even though an output type was specified). Ashwini Sharma reported
441bugs where readfile() was incorrectly freeing its buffer, and where toy_init()
442was zeroing the wrong data because the field it was using to measure (rebound)
443had moved (when I moved it back I added a comment why the field needs to be
444there), fixed a segfault in the dhcp client, and made a 0 length read at
445the start of password entry count as EOF. Make the "we are not root" test
446in the init code show the help text. Posix implies that fflush() can return
447success even when the stream's error bit is set, so call both fflush() and
448ferror() from xprintf().</p>
449
450<p>Isaac Dunham pointed out that bloatcheck couldn't deal with diff
451implementations that only implement "unified diff" format, and that some
452diff implementations can't handle nonseekable input (I.E. reading from
453a pipe). Bugfix so "help -a" works again. Option parsing on nohup now stops
454at first nonoption argument. Fix segfault in "which" if PATH wasn't set,
455which was actually a bug in lib function find_in_path(). Made rm -rf of
456chmod 000 directories actually remove them.</p>
457
458<p>The build now passes the same $CFLAGS to the library probe as the final
459build, because arch linux is so broken it provides different sets of
460libraries for static and dynamic linking.</p>
461
462<p>It turns out sprintf("%.123s", str) is counting characters, not bytes,
463so globally enabling locale support opens stack smashing vulnerabilities.
464So there's a new TOYFLAGS_LOCALE you set in toyflags when you want the
465setup code to setlocale().</p>
466
467<p><b>Upgrades:</b></p>
468
469<p>Isaac Dunham extended cpio to archive unreadable empty files, and I taught it
470to set uid/gid and timestamp when extracting archives. Isaac also
471added tests for cpio, link, and du, added lspci -i, made the pci database
472parsing skip # comment lines, merged logname and whoami into id.</p>
473
474<p>Daniel Verkamp sped up md5sum about 30% with some loop unrolling, making
475it actually smaller in the process. I added -b flags to md5sum and sha1sum
476for "brief" output that's just the hash with no filename. (I'm aware other
477implementations use that for MSDOS "binary" mode, and don't care.)</p>
478
479<p>When building standalone commands (scripts/singleconfig.sh commandname),
480the build now switches on all the sub-options of the command so we get
481a standalone version with all the bells and whistles enabled.</p>
482
483<p>Add -ds flags to date and document +FORMAT escapes. Add the shell NOP
484command ":" as an alias for true (for toysh).</p>
485
486<p>Add uClibc probe for iconv() and fallocate. (The fact it didn't always
487build against uClibc is why fallocate wasn't enabled in defconfig before.)</p>
488
489<p>The umount command now does an losetup -d on the device by default, so
490we don't leak loopback devices. Bugfix to losetup so "losetup /dev/loop0
491filename" actually works again.</p>
492
493<p>Divya Kothari sent in test suite entries for ls, ln, rm, mv, printf, dd,
494and renice. Then a second round for lsattr/chattr, mount, chmod, pgrep/pkill,
495groupadd, groupdel, and useradd. Several of these uncovered bugs, still
496working to fix them.</p>
497
498<p>There are now free() functions for the predefined llist types and a
499dlist_terminate() function to break doubly linked lists. The new
500generic_signal() handler either sets "toys.signal" or writes a byte
501to toys.signalfd with the signal number if signalfd isn't -1 (which it's
502initialized to in toy_init).</p>
503
504<p>The option parsing logic can now detect when a double fits in a long and
505use the more precise type for floating point arguments (the FLOAT macro
506contains the type used). The human_readable() function now just outputs
507decimal kilo/mega/gigabytes (so when du -u says 5.0G it means 5.0 billion
508bytes). The build infrastructure now notices duplicate commands (so if you
509cp toys/pending/command.c toys/other/command.c and forget to delete the
510first one, the build break is now more informative).</p>
511
Rob Landley08e18fb2014-04-20 14:22:19 -0500512<hr><b>April 20, 2014</b>
513<blockquote><p>And to this end they built themselves a stupendous supercomputer
514which was so amazingly intelligent that even before the data banks
515had been connected up it had started from "I think therefore I am" and got as
516far as the existence of rice pudding and income tax before anyone managed to
517turn it off. - The Hitchhiker's Guide to the Galaxy.</p></blockquote>
518
519<p><a href=downloads/toybox-0.4.8.tar.bz2>Toybox 0.4.8</a> is based on
520<a href=http://landley.net/hg/toybox/shortlog/1262>commit 1262</a>. And
521about time too.</p>
522
523<p>The big news is that the build no longer needs python to generate help.h,
524that's now done in C. The help text generation is also collating help text
525from multiple options, merging command line option blocks and usage: lines.
526There's even a new <a href=help.html>help web page</a>.</p>
527
528<p><b>New commands:</b> Ifconfig, cpio, and su were cleaned up the rest of the
529way and promoted out of pending. That saga is mostly explained on the
530<a href=cleanup.html>cleanup page</a>. Vivek Bhagat's freeramdisk,
531Isaac Dunham's fsfreeze, and Felix Janda's iconv are also new.</p>
532
533<p><b>In pending:</b>
534Ashwini Sharma's team submitted tcpsvd, udpsvd, telnet, telnetd, last, more,
535groupdel/delgroup, arping, brctl, ftpget, ftpput, printf, reset, and added
536ipv6 support to traceroute. Kyungwan Han's team submitted modprobe and getty.
537Vivek Bhagat submitted openvt and deacllocvt. Samuel Holland submitted fold.
538I wrote a new inflate (zip/zlib/gzip decompression) implementation in
539compress.c, and still need to do a corresponding deflate (compression-side)
540and plug them into gzip and zip and so on. (Right now it does zcat.)</p>
541
542<p>Several commands (vmstat, login, du, vconfig, mountpoint, free, chroot,
543cut, touch, modinfo, expand) predate the "pending" directory, and are thus
544in other directories but still need cleanup. Of these, vmstat got some
545work this time (which would be much easier other vmstat implementations
546documented what their output actually meant).</p>
547
548<p><b>Upgrades:</b> Ifconfig grew /prefix netmask support (ala 1.2.3.4/24). Grep now has -zZ to
549handle null terminated data, cksum grew -H for hex output. Upgraded od so the
550fields align better when producing multiple output types. Help has -a and -h
551options (all commands, html output).
552Bugfix to blkid building for a 32 bit target. The date command can actually
553set dates now. The O_NOFOLLOW compile time probe didn't work with cross
554compiling, so it's back to an #ifdef test in portability.h. Nathan McSween
555sent in a bugfix to od and a portability fix in the common library code.
556Ashwini Sharma spotted a bug in pidof -o, and added verbose (-v) options
557to mkdir and ln, and suggested killall should have an -s option and
558allow -l to take zero arguments. Ashwini Sharma and Felix Janda upgraded
559tftpd. Fixed dumpleases still using toynet.h after
560that was removed. Corrected killall return code and error reporting.
561Isacc Dunham fixed bugs all over the tree, did cleanup on a bunch of
562pending commands (getty, ftpget, init, openvt, modprobe...), and clarified
563find's help text. Tom Sparrow ran three different static analyzers on
564the code, which resulted in a few cleanups. The peek()/poke() functions
565now use "volatile" to prevent broken compiler "optimizations" to do with
566aliasing.</p>
567
568<p><b>Build stuff:</b> Each FOR_xxxx macro now has a complementary CLEANUP_xxxx macro, so you
569can put multiple commands with different command line options in the same
570.c file, so they can share infrastructure outside of lib. (This let the
571bunzip logic move out of lib into bzcat.c.) See XXX for example.
572i
573<p>The headers #included in toys.h are now grouped by standard, and headers
574not listed in Posix or LSB were moved to portability.h. The old xregcomp.h
575was folded into lib.h because it's posix (and supporting oddball uClibc
576configurations isn't as important as it once was).</p>
577
578<p>Regression tested against Ubuntu 8.04 to fix up bit-rot in defconfig
579build on older systems. (We depend on Posix-2008, but not necessarily
580the absolute latest build environment.)</p>
581
582<p><b>In lib</b>: lib/xwrap.c added xgetpwnam(), xchroot(), and lib/lib.c now has names_to_pid().
583xsetuid() was replaced with xsetuser() which takes a struct passwd
584and sets both gid and uid, mkpathat() got factored out into a library command,
585get_int_value() became atolx_range(), and
586xmsprintf() is now just xmprintf(). The bunzip2 logic moved from lib into
587bzcat.c.</p>
588
589<p><b>Documentation</b>: new <a href=help.html>help page</a> with the
590help text for all the defconfig commands, using the new help -ah output.
591The <a href=code.html>source code walkthrough</a> now says more about
592#including header files, and how the generated/* directory works. The
593<a href=design.html>design page</a> has some new paragraphs about trading
594of different kinds of simplicity, and why comments aren't a substitute for
595good code. The README no longer trails off into obvious unfinished confusion
596at the end. Each page on the website should now have its own title.</p>
597
Rob Landley78663502013-11-19 09:44:48 -0600598<hr><b>November 18, 2013</b>
599<blockquote><p>"Space," it says, "is big. Really big. You just won't believe how vastly, hugely, mindbogglingly big it is. I mean, you may think it's a long way down the street to the chemist's, but that's just peanuts to space." -
600The Hitchhiker's Guide to the Galaxy.</p></blockquote>
601
Rob Landley4d886d62014-03-08 19:26:33 -0600602<p><a href=downloads/toybox-0.4.7.tar.bz2>Toybox 0.4.7</a> is based on
Rob Landley78663502013-11-19 09:44:48 -0600603<a href=http://landley.net/hg/toybox/shortlog/1122>commit 1122</a>.</p>
604
605<p>New commands: Brad Conroy submitted blkid. Elie De Brauwer submitted
606reboot, halt, and poweroff. Strake's nl got cleaned up and promoted from
607pending to posix. In addition, the existing chvt and vconfig got some
608cleanup.</p>
609
610<p>That said, I haven't nearly kept up with the flood of new commands going
611into pending: Ashwini Sharma's team submitted
612dd, dumpleases, traceroute, top, useradd, groupadd, mkpasswd, tftpd, and
613an fsck wrapper (with no filesystem drivers yet). Isaac Dunham sent in cpio.</p>
614
615<p>Bugfixes: Jeroen van Rijn added a user count to uptime. Elie De Brauwer
616added -e to watch, removed a memory leak, and fixed a terminal size problem.
617William Haddon made xargs call its command line once even with blank input
618(the standard is vague, but builds expect it), and fixed an off by one bug
619where grep didn't malloc enough space with -E (leading to a segfault).
620I fixed a glitch in bunzip2 (same one as went into busybox since they're using
621the code I wrote), in od to fix -t co, -J, and -c options. Add uname -o as a
622synonym for -s. Build fix to never use $CC without prefixing it with
623$CROSS_COMPILE (since $HOSTCC could be different). Anca Emanuel spotted
624a typo in the web page.</p>
625
626<p>The compile-time command line option parsing got rewritten (ported from
627bash to C), which should speed up builds a bit and allow code controlled by
628--longopts to drop out properly when disabled in the configuration. Terminal
629querying got refactored. Patch's -x option is now more informative (a
630debug thing if you're trying to figure out why a patch didn't apply).
631The "toynet.h" file got folded into toys.h since musl supports it and
632micromanging uClibc options isn't very interesting anymore. The test suite
633now uses scripts/single.sh when testing a single command.</p>
634
Rob Landley5cab9942013-09-18 10:50:38 -0500635<hr><b>September 17, 2013</b>
636<blockquote><p>"Think of a number," said the computer, "any number."
637Arthur told the computer the telephone number of King's Cross railway
638station passenger inquiries, on the grounds that it must have some function,
639and this might turn out to be it. - The Hitchhiker's Guide to the Galaxy</p>
640</blockquote>
641
642<p><a href=downloads/toybox-0.4.6.tar.bz2>Toybox 0.4.6</a> is based on
643<a href=http://landley.net/hg/toybox/shortlog/1068>commit 1068</a>.</p>
644
645<p>This release adds
646several new commands: Felix Janda wrote paste and fallocate, Kyungwan Han
647submitted eject, Strake contributed grep, Ashwini Sharma added pmap (and
648a testsuite entry for grep), Lukasz Skalski sent pwdx, Isaac Dunham posted
649acpi, and I did timeout and umount.</p>
650
651<p>The ls command now has a --color=auto option (suggested by Rich Felker).
652The multiplexer now has a --help option so you can say "./toybox --help blah"
653instead of using the built-in "help" command. (Which is a shell built-in.
654Try it on your command line, it's like man for shell builtins. But a certain
655other project has conditioned people to expect --help, so...) I forget who
656heehooman at gmail is but they pointed out unshare needed PID and UID
657namespace support.</p>
658
659<h3>Pending</h3>
660
661<p>A lot of new commands in toys/pending, to the point the next release should
662probably just focus on cleanup and review of this backlog. We've got klogd,
663dhcp, dhcpd, watch, route, and ps from
Rob Landleyd2774142013-10-04 14:32:39 -0500664Ashwini Sharma (and an fsck wrapper but no fsck.fstype engines yet),
Rob Landley5cab9942013-09-18 10:50:38 -0500665syslogd, pgrep, and pkill from Madhur Verma, netstat by Ranjan Kumar,
666test by Felix Janda, lspci by Isaac Dunham, nl, su, and renice by strake (I.E.
667M. Farkas-Dyck), and sysvinit by Kyungwan Han.</p>
668
669<p>Some cleanup work on existing pending commands that aren't
670ready to promote yet: I did a few more rounds on ifconfig
671and Isaac Dunham's did several cleanups to xzcat, Felix Janda cleaned up
672logger and syslogd...</p>
673
674<p>Also some cleanup work on commands that predate the pending directory,
675but weren't quite polished when they went in, most prominently du,
676expand, and touch.</p>
677
678<h3>Infrastructure</h3>
679
680<p>The new scripts/single.sh builds a standalone command without the
681multiplexer, although not all commands can be built that way yet (NEWTOY yes,
682OLDTOY no) and the space savings aren't anything to write home about. (If a
683command needs the option parsing logic at all, it needs all of it.) If
684you're curious, you can do:</p>
685
686<blockquote><pre>
687make defconfig
688make
689mkdir singles
690for i in $(./toybox)
691do
692 echo $i
693 PREFIX=singles/ scripts/single.sh $i || break
694done
695</pre>
696<p>(And then wait a long time and watch almost half the builds fail.)</p>
697</blockquote>
698
699<p>There is now libbuf analogous to toybuf, another global 4k buffer this
700time for use by lib/ code instead of command code.</p>
701
702<p>The lib directory got split up a bit, lib/pending.c contains functions
703not yet used by anything outside of toys/pending/*, and lib/xwrap.c contains
704functions that wrap other functions and handle failures (via error_exit).
705This leaves lib/lib.c containing actual new functions.</p>
706
707<p>General improvements and bug fixes to argument parsing. The [-abc] exclude
708logic should now clear arguments slots when disabling options. Bare --longopts
709should work now and be able to report errors using their name, the new ;
710option allows optional arguments to longopts only suppliable with = (I.E.
711--color and --color=auto but not --color auto).</p>
712
713<p>I'm gradually weaning the code off of itoa()/utoa() because sprintf
714does this already. In this case "simple" probably means "let libc do it
715for us".</p>
716
717<p>Rewrote for_each_pid_with_name_in() and renamed it to just names_to_pid().
718It shouldn't get confused trying to compare absolute and relative paths quite
719so much anymore.</p>
720
721<p>lib/llist.c grew a new dlist_pop() function for removing a doubly
722linked list entry while maintaining a circular list; tail and patch are
723using it now.</p>
724
725<p>The musl guys suggested a new optimization flag
726(-fno-asynchronous-unwind-tables)
727that shaves about 10% off the binary size by removing a C++ism that crept
728into gcc's idea of C. While I don't normally try to micromanage the compiler,
729"-fstop-being-stupid" is a thing you have to hit gcc with from time to time.</p>
730
731<h3>Bugfixes</h3>
732
733<p>Felix Janda and I did a largeish rewrite of tail to
734finally make it work right (we think). Still need to implement tail -f
735someday (the tricky bit is making -f follow multiple files at once).
736Felix also reported a bug in xpidfile.</p>
737
738<p>Juhani Haverinen pointed out that
739python 3 doesn't work with config2help.py, so the detection logic looks
740for python2 (until I get around to rewriting that in C). Elie De Brauwer
741then fixed our first attempt at this, and also fixed uname's help string.</p>
742
743<p>Ashwini Sharma
744pointed out the build was making a FLAG_ macro for " " which broke some
745configurations. (That's a control character, not a command line option.)</p>
746
747<p>Jacek Bukarewicz pointed out a bug in chdir permission handling, and
748a way to make env segfault. Both should be fixed now.</p>
749
750<p>The new function xexec_optargs()
751replaces calls to xexec(toys.optargs) to avoid freeing and reusing optargs
752during option parsing screwing stuff up (such as netcat's exec mode).</p>
753
754<p>The stat command's %a output was padded with leading zeroes, which
755didn't match anybody else's behavior and thus made the test suite hiccup
756between TEST_HOST and testing toybox. (If you go "TEST_HOST=1 scripts/test.sh
757command" it sanity checks the tests against the host implementation.)</p>
758
759<p>Last release, "mkdir sub/sub && chmod 007 sub/sub && rm -rf sub" didn't
760delete sub and didn't exit with an error either. Neither was correct, rm
761should now be fixed.</p>
762
763<p>
Rob Landley49f8d6f2013-07-26 13:04:21 -0500764<hr><b>July 26, 2013</b>
765<p>Georgi Chorbadzhiyski maintains a <a href=https://github.com/gfto/toybox>git
766mirror</a> of the repository on github, automatically updated from the
767mercurial every 6 hours. The mirror is read only, but you can generate patches
768against it and post them to the list.</p>
Rob Landleye566f3a2013-06-16 20:02:38 -0500769
Isaac Dunhamc810f9f2013-07-06 11:26:15 -0500770<hr><b>July 2, 2013</b>
771<blockquote><p>"Time is an illusion. Lunchtime doubly so." "Very deep. You
772should send that in to the Reader's Digest. They've got a page for people
773like you." -
774The Hitchhiker's Guide to the Galaxy.</p></blockquote>
775
776<p><a href=downloads/toybox-0.4.5.tar.bz2>Toybox 0.4.5</a> is based on
777<a href=http://landley.net/hg/toybox/shortlog/941>commit 941</a>. It adds
778uuencode and uudecode from Erich Plondke, and enables Luis Morales' "who" by
779default. Felix Janda and I cleaned up last year's "stat" submission and
780enabled it. Ivo van Poorten added "groups".
781Andre Renaud added "lsusb". I implemented "split", "pivot_root", and "mv".
782</p>
783
784<p>The "help" command is implemented differently now (lib/help.c) and
785each command can now understand --help (including both "toybox --help"
786and "toybox --help command" in the multiplexer).</p>
787
788<p>The "pending" directory has several commands (find, xzcat, nbd-client,
789logger, expr) which work but are not enabled by default pending further cleanup.
790Ifconfig is enabled, but still in pending because it's only 2/3 cleaned up.
791(It's an awkward halfway state but I'm not holding up the release for it.)</p>
792
793<p>I'm <a href=cleanup.html>documenting the cleanups</a> to teach
794more people to do it, but the writeups aren't caught up yet. The
795<a href=roadmap.html>roadmap</a> also got updated a bit with further analysis
796of other projects, and the README and about pages got updated.</p>
797
798<p>Fixed _another_ "ls -C" segfault when terminal size can't be detected,
799condensed the ls help text to fit on one page, implented --color, and taught
800-l to print the major, minor numbers when showing block/char devices.
801Argument parsing now handles "--" properly (to end option checking),
802and the infrastructure can now handle bare --longopts that have no
803corresponding short option (both were implemented before but didn't work).
804Fixed an old bug in "patch", chmod grew -f, who grew -a. Isaac Dunham
805fixed "-" vs "_" handling in modinfo, added a "firmware" output
806field, added -b and -k support, and taught it that the ".ko" extension means
807to look for the file at the specified path instead of under /lib. Felix Janda
808moved file permission display code to lib so ls and
809stat could share it. Ashwini Sharma spotted a bug in xabspath when the
810last path component exists but we haven't got permissions to open it
811(ala readlink -f /dev/sda as a normal user).
812</p>
813
814<p>In the build infrastructure, scripts/findglobals.sh finds leaked global
815variables. (Leaked means they aren't part of the global union: Other than glibc
816debris, toybox should define "this", "toy_list", "toybuf", and "toys", and
817that's it; the rest add memory footprint to every command for the benefit of
818just one command; use GLOBALS() to stick 'em in the union.) Static linking
819against libraries other than the host's libc now applies to feature probes
820for unshare and such. Neuter stupid internationalization support that makes
821various host "sort" commands put things in an order other than alphabetical
822(breaking the multiplexer's binary search on command names).
823
824<p>You should now be able to build from a source control snapshot on a build
825system that hasn't got python: if you disable CONFIG_TOYBOX_HELP. (The
826release tarballs ship generated/help.h, but it's not in source control.
827Eventually I should rewrite that python script in C.)</p>
828</p>
829
830<p><b>LICENSE TWEAK</b>: After <a href=http://lists.landley.net/pipermail/toybox-landley.net/2013-March/000794.html>discussion</a> on the mailing list the "2 clause
831BSD" <a href=license.html>license</a> got slightly simplified so the first
832paragraph now says:</p>
833
834<blockquote><p>Permission to use, copy, modify, and/or distribute this
835software for any purpose with or without fee is hereby granted.</p></blockquote>
836
837<p>It used to continue "provided that the above copyright notice and this
838permission notice appear in all copies", but A) what's the point? B) does "all
839copies" mean binaries, or just source code, or what? C) lots of projects
840that consider BSD and GPL compatible have <a href=https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/crypto/aes_generic.c>files with
841both license notices</a> on them (sometimes at <a href=http://git.busybox.net/busybox/tree/shell/ash.c>opposite ends of the file</a> to make the conflict
842less obvious) because "all copies must include this function" would violate
843the GPL but "all copies must include this magic text blob" somehow don't?</p>
844
845<p>I don't want to have to care about this anymore. The tweaked version is more
846or less public domain with a liability disclaimer, but we're still calling it
847BSD (sometimes "0 clause BSD") to avoid explaining.</p>
848
Rob Landley6f5ef6f2013-03-21 20:21:12 -0500849<hr><b>March 21, 2013</b>
850<p>Video of my ELC talk
Rob Landleye566f3a2013-06-16 20:02:38 -0500851"<a href=http://youtu.be/SGmtP5Lg_t0>Why is Toybox?</a>"
Rob Landley6f5ef6f2013-03-21 20:21:12 -0500852is up on youtube. Related materials include the
853<a href=http://landley.net/talks/celf-2013.txt>talk outline</a> and an
854<a href=/aboriginal/about.html#selfhost>android self-hosting writeup</a>.</p>
855
Rob Landleye566f3a2013-06-16 20:02:38 -0500856<p>[Updated June 4] The following links jump to specific topics in the video. (Sorry about
857the ads, it's The Linux Foundation.)</p>
858
859<ul>
860<li>0m29s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=0m29s>The smartphone is replacing the PC</a></li>
861 <ul>
862 <li>4m22s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=4m22s>Software needed to become self-hosting</a></li>
863 <li>6m20s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=6m20s>Do we care if android or iphone wins?</a></li>
864 </ul>
865<li>9m45s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=9m45s>Android not vanilla: oppose or accept?</a></li>
866 <ul>
867 <li>11m30s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=11m30s>Open source can't do User Interfaces</a></li>
868 </ul>
869<li>15m09s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=15m09s>Android is not copyleft: oppose or accept?</a></li>
870<li>18m23s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=18m23s>Security issues</a></li>
871<li>21m15s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=21m15s>Solutions to the software problems</a></li>
872 <ul>
873 <li>22m55s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=22m55s>What toybox needs to be/do</a></li>
874 <li>28m17s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m17s>What is toybox?</a></li>
875 <ul>
876 <li>28m58s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m58s>Why toybox started...</a></li>
877 <li>37m50s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=37m50s>What does toybox actually implement?</a></li>
878 </ul>
879 </ul>
880</ul>
881</span>
882
883
Rob Landley6f5ef6f2013-03-21 20:21:12 -0500884<hr><b>March 14, 2013</b>
885<blockquote><p>"Ford, you're turning into a penguin. Stop it." -
886The Hitchhiker's Guide to the Galaxy.</p></blockquote>
887
888<p><a href=downloads/toybox-0.4.4.tar.bz2>Toybox 0.4.4</a> is based on
889<a href=http://landley.net/hg/toybox/shortlog/813>commit 813</a>, adding
890the "time" and "readahead" commands, plus some bugfixes.</p>
891
892<p>The "cp" command now implements the -s symlink option, plus bugfixes
893getting various corner cases right as used in actual package builds.
894"id -Gn root" should now print root's groups
895instead of the current user's. Several build fixes so toybox builds under
896Ubuntu 8.04 again (which is about as old a build environment as you
897can expect to find posix-2008 features in).</p>
898
899<p>Unfinished commands have generally been moved to "toys/pending".
900Everything else should "default y" to participate in make defconfig.
901Several of those pending commands got some basic cleanup so allyesconfig
902should at least compile (although defconfig is still what's useful).</p>
903
904<p>Significant roadmap updates, checking several other multicall binaries
905(klibc, sash, sbase, s6...) to see what commands they include.</p>
906
Rob Landley9a69a922013-02-23 18:32:08 -0600907<hr><b>January 18, 2013</b>
908<blockquote><p>This must be Thursday. I never could get the hang of Thursdays. - The Hitchhiker's Guide to the Galaxy.</p></blockquote>
909
910<p><a href=downloads/toybox-0.4.3.tar.bz2>Toybox 0.4.3</a> is based on
911<a href=http://landley.net/hg/toybox/shortlog/793>commit 793</a>. There
912are now exactly 100 commands in defconfig (of a little over 220 on the
913<a href=roadmap.html>todo list</a>).</p>
914
915<p>Elie De Brauwer added the rev command, cleaned up tac, implemented the -s
916and -f flags for seq, added -v and -i to killall (and fixed killall not to
917kill itself before finishing its pid list), and added to the test suite.
918Felix Janda added -m to mkdir, pwd -L and -P, and more test suite entries.</p>
919
920<p>Rob Landley added the losetup command, and fixed the existing ls, cp, and
921readlink commands. The segfault in ls
922happened when it couldn't determine the screen size (last release changed the
923default to -C and a screen size of 0 made column view unhappy), and cp got an
924extensive rewrite bringing it up to date with the dirtree changes and fixing
925a number of things it never did right in the first place. The xabspath()
926code in the library now handles a symlink after ".." properly (and the test
927suite checks for it).</p>
928
929<p>Infrastructure-wise the code is better about automatically setting the
930error return code properly. Now error_msg() sets the exit code to 1 if it's
931still defaulting to 0, and the global exit path does a fflush(NULL) with error
932bit check rather than trying to be quite so granular about flushing. (That
933means if we use printf() instead of xprintf() it still exits with the right
934error code, it just doesn't end the program early on an output error.)
935Minor bugfix so TOYBOX_DEBUG
936doesn't always warn about the lack of suid bit when toybox is built with
937at least one STAYROOT command. Bugfix for the option [grouping] logic
938(and then further fixes to the error reporting pointed out by Ashwini Sharma).
939dirtree_handle_callback() now has a prefix like the rest of the dirtree
940functions. A lot of stuff doing manual path handling was switched to using
941libc basename() (including, embarassingly, the basename command), which means
942it now correctly detects "/trailing/slash/" which the previous code didn't.</p>
943
944<p>Also, last release included some accidentally checked in debug code that
945disabled compiler optimization, so the binary size bloated a bit. It's back
946to -Os by default now.</p>
947
Rob Landley8390c652012-12-19 09:16:45 -0600948<hr><b>December 15, 2012</b>
949<blockquote><p>"The major difference between a thing that might go wrong and a
950thing that cannot possibly go wrong is that when a thing that cannot possibly
951go wrong goes wrong it usually turns out to be impossible to get at or repair."
952</p><p>- The Hitchhiker's Guide to the Galaxy.</p></blockquote>
953
954<p><a href=downloads/toybox-0.4.2.tar.bz2>Toybox 0.4.2</a> is based on
955<a href=http://landley.net/hg/toybox/shortlog/749>commit 749</a> and is
956just a resync. Linux 3.7 came out, meaning it's time to do an Aboriginal
957Linux release, and that should use a stable version of toybox. So here's
958a new stable version.</p>
959
960<p>The new commands are cut (from Jason Kyungwan Han), touch
961(from Choubey Ji), expand (from Jonathan Clairembault, and he fixed a
962bug in login), and rm (from Rob Landley). Felix Janda added UTF-8
963support infrastructure (for non-ascii character sets) with a config option.
964Elie De Brauwer added tests for cat and sha1sum, and -so options to pidof.
965The "ls" command defaults to -C (column view) now, and "readlink" now supports
966-fenq.</p>
967
968<p>Portability work: toybox should now build against the musl C library,
969and against older glibc versions (circa 2008, much before that and kernel
970features we depend on start to drop out).</p>
971
972<p>The whole codebase got reindented from "one tab" to "two spaces" per
973level. The option parsing logic now understands [groups] of commands (when more
974than one in a group is selected it can switch the others off, or error out,
975or other things). The error_exit() infrastructure can now longjmp back to an
976earlier point instead of exiting. Each toys/* directory now has a README,
977the first line of which is the fancy name menuconfig uses for the directory
978(so no more hardwired directory list in scripts/genconfig.sh).</p>
979
980<p>Fixed a filehandle leak in getmountlist().
981Pass parent pointer to dirtree_add_node() so it can give error messages with
982full path. The yesno() function now always reads from stdin and writes to
983stderr (we can retry tty checking complexity once we've got commands needing
984it).</p>
985
986<p>The open group broke their website so the
987<a href=http://opengroup.org/onlinepubs/9699919799>old links</a> to POSIX 2008
988now <a href=http://pubs.opengroup.org/onlinepubs/9699919799>need to start with
989pubs</a>. Some of the links in the tree have been updated, others haven't while
990I wait to see if their webmaster notices and fixes it.</p>
991
992<p>(I note that the current rm implementation is not technically posix compliant
993because the standard requires infinite recursion depth and the current
994implementation uses one filehandle per level. I can add a config option
995to do it Posix's way, which is more brittle and needs extra security checks,
996but am waiting for somebody to complain first. The default "ulimit -n" is 1024
997filehandles, so drilling down over 1000 nested subdirectories).</p>
998
Rob Landley571b0702012-11-13 16:13:45 -0600999<hr><b>November 13, 2012</b>
1000<blockquote><p>"Rule Six: The winning team shall be the first team that wins."
1001- The Hitchhiker's Guide to the Galaxy.</p></blockquote>
1002
1003<p><a href=downloads/toybox-0.4.1.tar.bz2>Toybox 0.4.1</a> is based on
1004<a href=http://landley.net/hg/toybox/shortlog/691>commit 691</a>.</p>
1005
1006<p>Elie De Brauwer contributed usleep, Ashwini Kumar contributed du, and
1007Kyungwan Han contributed vconfig. Other new commands include switch_root and
1008md5sum, and the remaining shell wrappers are now proper commands (dos2unix,
1009unix2dos).</p>
1010
1011<p>The patch command now supports -l, and gethostname is now enabled by
1012default. The df command follows symlinks to get the actual device name.
1013Felix Janda added -m support to wc (for utf8).</p>
1014
1015<p>On the infrastructure side, the commands have now been grouped into
1016"posix", "lsb", and "other" subdirectories (for things required by Posix-2008,
1017the Linux Standard Base 4.1, and commands in neither). This affects menuconfig
1018and the actual source layout (toys/cp.c is now toys/posix/cp.c, and so on).
1019An android directory is planned (see the updated
1020<a href=roadmap.html#android>android roadmap analysis</a>).</p>
1021
1022<p>The FLAG_ macros for command option parsing and TT alias for the command's
1023global block are now automatically generated, commands should
1024#define FOR_commandname before #including <toys.h> to get the macros for that
1025command.</p>
1026
1027<p>An upgrade to the build infrastructure now allows commands with _ and -
1028in them, such as switch_root.</p>
1029
1030<p>Bugfixes: Avery Pennarun spotted a case where ls showed uid twice instead of
1031uid and gid, and that nice was using the wrong range of numbers.
1032The ls command also recursed inappropriately last time (not quite
1033properly converted for the dirtree changes last release), and now it's
1034fixed. Roy Tam pointed out a glitch in sh, and fixed df's percentage
1035calculation to match the POSIX spec. The kernel build didn't like our mktemp
1036and it does now. The wc command wasn't quite posix compliant (trailing spaces
1037break stuff). The ls command recursed inappropriately last time (not quite
1038properly converted for the dirtree changes last release), and now it's
1039fixed. The catv command wasn't displaying byte 255 correctly. Some lib
1040fixes (thinko in xpidfile). Fixed uname -m when running a 32 bit x86 binary
1041on an x86-64 host (it lies and says the system is i686, i586, or i486 depending
1042on what the toolchain that built the binary supported. This makes builds in
1043a 32 bit chroot on a 64 bit kernel break less.) The df command was checking
1044partitions in the wrong order (displaying undermounts instead of overmounts:
1045this used to work but some library code changed out from under it and it
1046wasn't updated to match until now). Felix Janda filled out the test suite
1047some more. The patch file creation logic got tweaked several times to
1048successfully apply more patches. Support for older (pre 2.10) glibc
1049versions was added to portability.h.</p>
1050
1051<p>Miscelaneous cleanups all around (mknod, sha1sum, logname), including a
1052rewrite of taskset to be less dependent on libc getting the headers right. All
1053the command headers should now point to the current relevant standards
1054document, where applicable.</p>
1055
1056<p>This news page had old news entries from before the relaunch moved into
1057a separate <a href=oldnews.html>oldnews</a> page.</p>
1058
1059<p>I forgot to create <a href=bin>static binaries</a> last time, but they're
1060back now.</p>
1061</span>
1062
Rob Landley31103f92012-08-25 11:51:25 -05001063<hr><b>July 23, 2012</b>
1064<blockquote><p>"Ford", Arthur said. "There's an infinite number of monkeys
1065out here who want to talk to us about this script for Hamlet they've worked
1066out." - The Hitchhiker's Guide to the Galaxy.</p></blockquote>
1067
Rob Landley571b0702012-11-13 16:13:45 -06001068<p><a href=downloads/toybox-0.4.0.tar.bz2>Toybox 0.4.0</a> is based on
1069<a href=http://landley.net/hg/toybox/shortlog/640>commit 640</a>.</p>
Rob Landley31103f92012-08-25 11:51:25 -05001070
1071<p>The new <a href=status.html>status page</a> is calculated from
1072the roadmap info, and should be easier to keep up to date in future.</p>
1073
1074<p>Andre Renaud contributed od and modinfo. Elie De Brauwer contributed
1075taskset, bugfixes to cmp and tail, and tests for sort and tail. Kyungwan Han
1076contributed passwd. Gaurang Shastri contributed w. Ashwini Sharma spotted a
1077case where dirtree was adding extra slashes to a path.</p>
1078
1079<p>I rewrote od, cleaned up comm, documented the
1080<a href=code.html#lib_llist>llist</a> and
1081<a href=code.html#lib_dirtree>dirtree</a> infrastructure, added an -r option
1082to date (and fixed a bug where -u wouldn't override /etc/localtime),
1083fixed bugs in chmod +stw, fixed ls to show suid bits properly when the
1084corresponding executable bit wasn't set, and worked around a longstanding
1085glibc bug where static linking prevents stdout from automatically flushing
1086pending output on exit.</p>
1087
Rob Landleyb1cc1d12012-06-25 06:42:24 -05001088<hr><b>June 25, 2012</b>
1089<blockquote><p>"For a moment, nothing happened. Then, after a second or so, nothing continued to happen." - The Hitchhiker's Guide to the Galaxy.</p></blockquote>
1090
1091<p><a href=downloads/toybox-0.3.1.tar.bz2>Toybox 0.3.1</a> is based on commit
1092<a href=http://landley.net/hg/toybox/shortlog/607>commit 607</a>. It's
1093mostly a bugfix release for ls -l (which was unhappy on targets other than
1094x86-64), plus a new "date" from Andre Renaud and rewritten chgrp/chown which
1095now support the full set of posix flags, plus a little work on the test
1096suite and some more header tweaks towards eventual compatability with the
1097musl libc.</p>
1098
1099<p>The todo list runneth over, but "release early, release often", so here
1100it is. The roadmap and documentation are a bit behind, and I've got ~40
1101pending submissions to review. I need to catch up...</p>
1102</span>
1103
1104<hr><b>June 12, 2012</b>
1105<blockquote><p>"For instance, on the planet Earth, man had always assumed that
1106he was more intelligent than dolphins because he had achieved so much - the
1107wheel, New York, wars and so on - whilst all the dolphins had ever done was
1108muck about in the water having a good time. But conversely, the dolphins had
1109always believed that they were far more intelligent than man - for precisely
1110the same reasons." - The Hitchhiker's Guide to the Galaxy.</p></blockquote>
1111
1112<p>It's well past time for <a href=downloads/toybox-0.3.0.tar.bz2>toybox 0.3.0</a>,
1113so here it is, based
1114on <a href=http://landley.net/hg/toybox/shortlog/595>commit 595</a>, and the
1115statically linked <a href=downloads/binaries>prebuilt binaries</a> should
1116actually be statically linked this time (thanks Ashwini Sharma for spotting
1117that).</p>
1118
1119<p>It's hard to figure out where to cut a release, because development
1120doesn't stop. "Long before now" is the obviuos answer, of course.
1121The project's maintainer also moved house during this development cycle, which
1122threw things off for a bit (so many boxes). Releases should hopefully be a bit
1123more frequent from here on.</p>
1124
1125<p>The big things Rob worked on this time were the new dirtree (directory
1126tree traversal) infrastructure, and a complete rewrite of ls using that
1127which should now implement all 26 posix options.</p>
1128
1129<p>Georgi Chorbadzhiyski added printenv, whoami, mkdir, mkfifo, chmod, chown,
1130chgrp, and uniq. He also added fraction and extension support to sleep (so if
1131you need a quarter-second sleep, it can do that now), and fixed a build bug
1132on slackware.</p>
1133
1134<p>Daniel Walter contributed a string to mode_t parser (in use by chmod and
1135mkdir -m). Ilya Kuzmich contributed comm. Elie De Brauwer added mountpoint,
1136vmstat, logname, login, and mktemp. Kevin Chase did some portability cleanups.
1137Pere Orga fixed some documentation.</p>
1138
1139<p>The "tac" and "clear" commands are now normal commands instead of shell
1140wrappers, and the header #includes have been cleaned up a bit to remove
1141deprecated functions and attempt to increase compatability with the bionic and
1142musl C libraries, "tail" should now use lseek() for large files, and "id" got
1143some cleanups and bugfixes.</p>
1144
1145<p>The new TOYBOX_FLOAT configuration option selects whether or not
1146to include floating point support (for embedded targets where that's
1147problematic).</p>
1148
1149<p>Several random bugfixes: unshare() might actually build portably now,
1150yes 'n' | cp -i should no longer bypass stdin and prompt via the tty, the
1151SUID support no longer drops permissions going through the toybox
1152multiplexer command, and a bugfix to xargs -0 means it should no longer
1153segfault. (I have a pending bug report about xargs not doing the full
1154posix whitespace handling that -0 obsoleted, but I'll deal with that next
1155release.)</p>
1156
1157<p>The build infrastructure is now automatically generating FLAG_ macros
1158for the options, but currently with the wrong names. Some more macro glue
1159is necessary, which I haven't quite figured out how to do yet.</p>
1160
1161<p>A defconfig toybox at the start of the $PATH has successfully built
1162Linux From Scratch (in my Aboriginal Linux project). The commands that
1163'default n' in the config are often still broken, cleanup is ongoing.
1164(The new dirtree stuff broke several of them that haven't been converted
1165yet, but if I wait until everything works we won't have a release before
11661.0, so here's a checkpoint.)</p>
1167
1168
Rob Landleyed6ed622012-03-06 20:49:03 -06001169<hr><b>March 3, 2012</b>
1170
1171<blockquote><p>"They went unnoticed at Goonhilly, passed over Cape Canaveral
1172without a blip, and Woomera and Jodrell Bank looked straight through them.
1173Which was a pity, because it was exactly the sort of thing they'd been looking
1174for all these years."</p></p>- The Hitchhiker's Guide to the Galaxy.</p>
1175</p></blockquote>
1176
1177<p>Here's <a href=downloads/toybox-0.2.1.tar.bz2>toybox 0.2.1</a> based
1178on <a href=http://landley.net/hg/toybox/shortlog/512>commit 512</a>. This
1179time around, there are statically linked <a href=downloads/binaries>prebuilt
1180binaries</a> for various embedded targets.</p>
1181
1182<p>It's been a busy few weeks, almost entirely due to new contributors. (I
1183have not quite been keeping up.)</p>
1184
1185<p>Elie De Brauwer contributed free, uptime, swapon, swapoff, lsmod, mknod,
1186insmod, rmmod, and fixed a bug in basename. Andre Renaud contributed ls, ln,
1187realpath, and hostname. Andres Heck contributed pidof and killall. Daniel
1188Walter wrote kill and extended id. Timothy Elliott contributed tail and tests
1189for cmp. Frank Bergmann sent a warning fix. Bryce Fricke added -i to cp.
1190Nathan McSween pointed out an optimization. Georgi Chorbadzhiyski fixed
1191cross compiling to work more reliably.</p>
1192
1193<p>(My own contribution this time around was just tightening up other people's
1194code, a build fix to unshare, some random bugfixes, and so on. My only new
1195code this time around was writing a bash replacement for the existing python
1196bloat-o-meter.)</p>
1197
1198<p>Last time (the 0.2.0 release) included the first pass at an id command from
1199Tim Bird, env and basename from Tryn Mirell, cmp and head from Timothy Elliott,
1200more bugfixes from Nathan McSween and Elie De Brauwer, and Luis Felipe Strano
1201Moraes did a first pass at the who command plus other bugfixes and
1202optimizations.</p>
1203
1204<p>(For that release I did xargs, cal, truncate, unlink, nohup, tty, wc, link,
1205dirname, unshare, and various infrastructure tweaks, but it took me 3 months
1206and those guys did their stuff in a week or so.)</p>
1207
Rob Landleye258af32008-01-05 18:09:49 -06001208
Rob Landleyd11ac702012-02-13 21:16:03 -06001209<hr><b>February 12, 2012</b>
1210<blockquote><p>
1211"for though it has many omissions and contains much that is apocryphal, or at
1212least wildly inaccurate, it scores over the older, more pedestrian work in two
1213important respects..."</p>
1214<p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
1215
1216<p>Here's the first BSD licensed release,
1217<a href=downloads/toybox-0.2.0.tar.bz2>toybox-0.2.0</a>, more a synchronization
1218point than anything particularly useful. 47 commands in a reasonably
1219ready-to-use state (what "make defconfig" builds), another ten or so partially
1220finished stubs ("make allyesconfig"), and several
1221patches pending on the mailing list I need to review and merge.</p>
1222
1223<p>More to come...</p>
1224
Rob Landleyce8a2672012-02-02 07:27:05 -06001225<hr>
1226<p><b>November 15, 2011</b> - Back from the dead, Toybox is now under a 2
1227clause BSD license, and aiming to become the default command line
1228implementation of Android systems everywhere.</p>
1229
1230<p>More to come...</p>
1231
1232<hr>
Rob Landley589f5cd2010-01-05 10:41:52 -06001233
Rob Landleyaba51702012-08-26 16:33:06 -05001234<p><a href=oldnews.html>Old news</a> from before the relaunch.</p>
Rob Landley589f5cd2010-01-05 10:41:52 -06001235
1236<!--#include file="footer.html" -->