Various fixes/updates

- Update HOWTO to note that directio and fallocate don't work with ZFS
  on Solaris.

Refactor the Makefile's to add CPPFLAGS and LIBS.
- Add -fno-omit-frame-pointer from Linux Makefile to every platform
- Change undefined $(ALL_CFLAGS) to $(CFLAGS)
- Pass -std=gnu99, without which OS X fails to build.
- Add -D__EXTENSIONS__ on Solaris since some functions we need are
  behind it.

- Pull in <limits.h> in fio.c to get PTHREAD_STACK_MIN.
- NetBSD doesn't define PTHREAD_STACK_MIN so set it to 4k in
   os-netbsd.h

- If we have posix_fallocate don't error out if it fails during mutex
  and malloc operations since it will fail on Solaris with a ZFS
  filesystem. As I understand it these aren't performance-critical
  operations so do they need to be considered critical?

- Remove fio_unused from os-* files since it's defined in fio.h and we
  don't really need it.

- FreeBSD has an idprio command but not the API so don't claim it does.

- OS X doesn't have the timer_* API so emulate it using
  setitimer/sigaction.

- NetBSD and Solaris don't support CLOCK_MONOTONIC in timer_create so
  remove FIO_HAVE_CLOCK_MONOTONIC from their os-* files.

I've noticed that a change I made a while ago to use fmin/fmax could
cause issues on older OSes - I had a CD with NetBSD 5.0.2 and found
they had only been implemented in 5.1 so I'm not sure if I should
revert it.

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
diff --git a/HOWTO b/HOWTO
index 35dccfb..e54382b 100644
--- a/HOWTO
+++ b/HOWTO
@@ -351,7 +351,8 @@
 fallocate=bool	By default, fio will use fallocate() to advise the system
 		of the size of the file we are going to write. This can be
 		turned off with fallocate=0. May not be available on all
-		supported platforms.
+		supported platforms.  If using ZFS on Solaris this must be
+		set to 0 because ZFS doesn't support it.
 
 fadvise_hint=bool By default, fio will use fadvise() to advise the kernel
 		on what IO patterns it is likely to issue. Sometimes you
@@ -555,7 +556,7 @@
 		job, can be overridden with a larger value for higher
 		concurrency. Note that increasing iodepth beyond 1 will not
 		affect synchronous ioengines (except for small degress when
-		verify_async is in use). Even async engines my impose OS
+		verify_async is in use). Even async engines may impose OS
 		restrictions causing the desired depth not to be achieved.
 		This may happen on Linux when using libaio and not setting
 		direct=1, since buffered IO is not async on that OS. Keep an
@@ -585,7 +586,7 @@
 		the depth drain down to 4 before starting to fill it again.
 
 direct=bool	If value is true, use non-buffered io. This is usually
-		O_DIRECT.
+		O_DIRECT. Note that ZFS on Solaris doesn't support direct io.
 
 buffered=bool	If value is true, use buffered io. This is the opposite
 		of the 'direct' option. Defaults to true.