evict all broadcast support

Libwebsockets is fundamentally singlethreaded... the existence of the
fork and broadcast support, especially in the sample server is
giving the wrong idea about how to use it.

This replaces broadcast in the sample server with
libwebsocket_callback_on_writable_all_protocol().  The whole idea of
'broadcast' is removed.

All of the broadcast proxy stuff is removed: data must now be sent
from the callback only.  Doing othherwise is not reliable since the
service loop may close the socket and free the wsi at any time,
invalidating a wsi pointer held by another thread (don't do that!)

Likewise the confirm_legit_wsi api added recently does not help the
other thread case, since if the wsi has been freed dereferencing the
wsi to study if it is legit or not will segfault in that case.  So
this is removed too.

The overall effect is to push user code to only operate inside the
protocol callbacks or external poll loops, ie, single thread context.

Signed-off-by: Andy Green <andy.green@linaro.org>
diff --git a/README.build b/README.build
index c277b23..d0c22e9 100644
--- a/README.build
+++ b/README.build
@@ -50,11 +50,6 @@
 
 There are several other possible configure options
 
---enable-nofork		disables the fork into the background API
-			and removes all references to fork() and
-			pr_ctl() from the sources.  Use it if your
-			platform doesn't support forking.
-
 --enable-libcrypto	by default libwebsockets uses its own
 			built-in md5 and sha-1 implementation for
 			simplicity.  However the libcrypto ones
diff --git a/README.coding b/README.coding
index 05bc09f..dfe6a41 100644
--- a/README.coding
+++ b/README.coding
@@ -25,24 +25,46 @@
 libwebsockets will adapt accordingly.
 
 
-Procedure for sending data from other threads or process contexts
------------------------------------------------------------------
+Libwebsockets is singlethreaded
+-------------------------------
 
-Libwebsockets is carefully designed to work with no blocking in a single thread.
-In some cases where you will add libwebsockets to something else that uses the
-same single thread approach, you can so a safe implementation by combining the
-poll() loops as described in "External Polling loop support" below.
+Directly performing websocket actions from other threads is not allowed.
+Aside from the internal data being inconsistent in forked() processes,
+the scope of a wsi (struct websocket) can end at any time during service
+with the socket closing and the wsi freed.
 
-In other cases, you find you have asynchronous events coming from other thread
-or process contexts and there's not much you can do about it.  If you just try
-to randomly send, or broadcast using libwebsockets_broadcast() from these other
-places things will blow up either quickly or when the events on the two threads
-interefere with each other.  It's not legal to do this.
+Websocket write activities should only take place in the
+"LWS_CALLBACK_SERVER_WRITEABLE" callback as described below.
 
-For those situations, you can use libwebsockets_broadcast_foreign().  This
-serializes the data you're sending using a private, per-protocol socket, so the
-service thread picks it up when it's ready, and it is serviced from the service
-thread context only.
+Only live connections appear in the user callbacks, so this removes any
+possibility of trying to used closed and freed wsis.
+
+If you need to service other socket or file descriptors as well as the
+websocket ones, you can combine them together with the websocket ones
+in one poll loop, see "External Polling Loop support" below, and
+still do it all in one thread / process context.
+
+
+Only send data when socket writeable
+------------------------------------
+
+You should only send data on a websocket connection from the user callback
+"LWS_CALLBACK_SERVER_WRITEABLE" (or "LWS_CALLBACK_CLIENT_WRITEABLE" for
+clients).
+
+If you want to send something, do not just send it but request a callback
+when the socket is writeable using
+
+ - libwebsocket_callback_on_writable(context, wsi) for a specific wsi, or
+ - libwebsocket_callback_on_writable_all_protocol(protocol) for all connections
+using that protocol to get a callback when next writeable.
+
+Usually you will get called back immediately next time around the service
+loop, but if your peer is slow or temporarily inactive the callback will be
+delayed accordingly.  Generating what to write and sending it should be done
+in the ...WRITEABLE callback.
+
+See the test server code for an example of how to do this.
 
 
 Fragmented messages
diff --git a/README.test-apps b/README.test-apps
index bcc295f..57afeb7 100644
--- a/README.test-apps
+++ b/README.test-apps
@@ -58,17 +58,6 @@
 serving both the script html over http and websockets.
 
 
-Forkless operation
-------------------
-
-If your target device does not offer fork(), you can use
-libwebsockets from your own main loop instead.  Use the
-configure option --nofork and simply call libwebsocket_service()
-from your own main loop as shown in the test app sources.
-
-
-
-
 Testing websocket client support
 --------------------------------
 
diff --git a/configure.ac b/configure.ac
index 5cc5554..d6868ec 100644
--- a/configure.ac
+++ b/configure.ac
@@ -34,20 +34,6 @@
 #
 #
 #
-AC_ARG_ENABLE(nofork,
- [  --enable-nofork  Disables fork-related options],
- [ nofork=yes
- ])
-
-if test "x$nofork" = "xyes" ; then
-CFLAGS="$CFLAGS -DLWS_NO_FORK"
-else
-AC_FUNC_FORK
-fi
-
-#
-#
-#
 AC_ARG_ENABLE(libcrypto,
  [  --enable-libcrypto  Use libcrypto MD5 and SHA1 implementations],
  [ libcrypto=yes
diff --git a/lib/client.c b/lib/client.c
index 7a01a4a..cf56b03 100644
--- a/lib/client.c
+++ b/lib/client.c
@@ -627,7 +627,7 @@
 	int ext_count = 0;
 #endif
 	unsigned char buf[LWS_SEND_BUFFER_PRE_PADDING + 1 +
-			 MAX_BROADCAST_PAYLOAD + LWS_SEND_BUFFER_POST_PADDING];
+			 MAX_USER_RX_BUFFER + LWS_SEND_BUFFER_POST_PADDING];
 	static const char magic_websocket_guid[] =
 					 "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
 
diff --git a/lib/libwebsockets.c b/lib/libwebsockets.c
index 728d5e9..e6c87fd 100644
--- a/lib/libwebsockets.c
+++ b/lib/libwebsockets.c
@@ -507,6 +507,7 @@
 				struct libwebsocket *wsi, struct pollfd *pollfd)
 {
 	int n;
+
 #ifndef LWS_NO_EXTENSIONS
 	struct lws_tokens eff_buf;
 	int ret;
@@ -686,11 +687,12 @@
 							  struct pollfd *pollfd)
 {
 	struct libwebsocket *wsi;
-	unsigned char buf[LWS_SEND_BUFFER_PRE_PADDING + 1 +
-			 MAX_BROADCAST_PAYLOAD + LWS_SEND_BUFFER_POST_PADDING];
 	int n;
 	int m;
 	struct timeval tv;
+	unsigned char buf[LWS_SEND_BUFFER_PRE_PADDING + 1 +
+			 MAX_USER_RX_BUFFER + LWS_SEND_BUFFER_POST_PADDING];
+
 #ifndef LWS_NO_EXTENSIONS
 	int more = 1;
 #endif
@@ -781,11 +783,7 @@
 #ifndef LWS_NO_SERVER
 	case LWS_CONNMODE_HTTP_SERVING:
 	case LWS_CONNMODE_SERVER_LISTENER:
-	case LWS_CONNMODE_BROADCAST_PROXY_LISTENER:
-	case LWS_CONNMODE_BROADCAST_PROXY:
 	case LWS_CONNMODE_SSL_ACK_PENDING:
-
-lwsl_debug("*\n");
 		return lws_server_socket_service(context, wsi, pollfd);
 #endif
 
@@ -1001,10 +999,7 @@
  *
  *	1) Accept new connections to our context's server
  *
- *	2) Perform pending broadcast writes initiated from other forked
- *	   processes (effectively serializing asynchronous broadcasts)
- *
- *	3) Call the receive callback for incoming frame data received by
+ *	2) Call the receive callback for incoming frame data received by
  *	    server or client connections.
  *
  *	You need to call this service function periodically to all the above
@@ -1423,11 +1418,6 @@
 	struct sockaddr_in serv_addr;
 	int opt = 1;
 	struct libwebsocket_context *context = NULL;
-#ifndef LWS_NO_FORK
-	unsigned int slen;
-	struct sockaddr_in cli_addr;
-	int fd;
-#endif
 	char *p;
 	struct libwebsocket *wsi;
 #ifndef LWS_NO_EXTENSIONS
@@ -1445,7 +1435,6 @@
 	lwsl_info(" LWS_INITIAL_HDR_ALLOC: %u\n", LWS_INITIAL_HDR_ALLOC);
 	lwsl_info(" LWS_ADDITIONAL_HDR_ALLOC: %u\n", LWS_ADDITIONAL_HDR_ALLOC);
 	lwsl_info(" MAX_USER_RX_BUFFER: %u\n", MAX_USER_RX_BUFFER);
-	lwsl_info(" MAX_BROADCAST_PAYLOAD: %u\n", MAX_BROADCAST_PAYLOAD);
 	lwsl_info(" LWS_MAX_PROTOCOLS: %u\n", LWS_MAX_PROTOCOLS);
 #ifndef LWS_NO_EXTENSIONS
 	lwsl_info(" LWS_MAX_EXTENSIONS_ACTIVE: %u\n", LWS_MAX_EXTENSIONS_ACTIVE);
@@ -1868,7 +1857,7 @@
 			lwsl_warn("setuid: %s\n", strerror(errno));
 #endif
 
-	/* set up our internal broadcast trigger sockets per-protocol */
+	/* initialize supported protocols */
 
 	for (context->count_protocols = 0;
 			protocols[context->count_protocols].callback;
@@ -1880,64 +1869,6 @@
 		protocols[context->count_protocols].owning_server = context;
 		protocols[context->count_protocols].protocol_index =
 						       context->count_protocols;
-
-#ifndef LWS_NO_FORK
-		fd = socket(AF_INET, SOCK_STREAM, 0);
-		if (fd < 0) {
-			lwsl_err("ERROR opening socket\n");
-			return NULL;
-		}
-
-		/* allow us to restart even if old sockets in TIME_WAIT */
-		setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (const void *)&opt,
-								  sizeof(opt));
-
-		bzero((char *) &serv_addr, sizeof(serv_addr));
-		serv_addr.sin_family = AF_INET;
-		serv_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
-		serv_addr.sin_port = 0; /* pick the port for us */
-
-		n = bind(fd, (struct sockaddr *) &serv_addr, sizeof(serv_addr));
-		if (n < 0) {
-			lwsl_err("ERROR on binding to port %d (%d %d)\n",
-								port, n, errno);
-			return NULL;
-		}
-
-		slen = sizeof cli_addr;
-		n = getsockname(fd, (struct sockaddr *)&cli_addr, &slen);
-		if (n < 0) {
-			lwsl_err("getsockname failed\n");
-			return NULL;
-		}
-		protocols[context->count_protocols].broadcast_socket_port =
-						       ntohs(cli_addr.sin_port);
-		listen(fd, 5);
-
-		lwsl_debug("  Protocol %s broadcast socket %d\n",
-				protocols[context->count_protocols].name,
-						      ntohs(cli_addr.sin_port));
-
-		/* dummy wsi per broadcast proxy socket */
-
-		wsi = (struct libwebsocket *)malloc(sizeof(struct libwebsocket));
-		if (wsi == NULL) {
-			lwsl_err("Out of mem\n");
-			close(fd);
-			return NULL;
-		}
-		memset(wsi, 0, sizeof (struct libwebsocket));
-		wsi->sock = fd;
-		wsi->mode = LWS_CONNMODE_BROADCAST_PROXY_LISTENER;
-#ifndef LWS_NO_EXTENSIONS
-		wsi->count_active_extensions = 0;
-#endif
-		/* note which protocol we are proxying */
-		wsi->protocol_index_for_broadcast_proxy =
-						       context->count_protocols;
-
-		insert_wsi_socket_into_fds(context, wsi);
-#endif
 	}
 
 #ifndef LWS_NO_EXTENSIONS
@@ -1963,76 +1894,14 @@
 	return context;
 }
 
-
-#ifndef LWS_NO_FORK
-
-/**
- * libwebsockets_fork_service_loop() - Optional helper function forks off
- *				  a process for the websocket server loop.
- *				You don't have to use this but if not, you
- *				have to make sure you are calling
- *				libwebsocket_service periodically to service
- *				the websocket traffic
- * @context:	server context returned by creation function
- */
-
-int
-libwebsockets_fork_service_loop(struct libwebsocket_context *context)
-{
-	int n;
-
-	n = fork();
-	if (n < 0)
-		return n;
-
-	if (n) {
-
-		/* main process context */
-
-		return 0;
-	}
-
-#ifdef HAVE_SYS_PRCTL_H
-	/* we want a SIGHUP when our parent goes down */
-	signal(SIGHUP, SIG_DFL);
-	prctl(PR_SET_PDEATHSIG, SIGHUP);
-#endif
-
-	/* in this forked process, sit and service websocket connections */
-
-	while (1) {
-		if (libwebsocket_service(context, 1000))
-			break;
-//#ifndef HAVE_SYS_PRCTL_H
-/*
- * on systems without prctl() (i.e. anything but linux) we can notice that our
- * parent is dead if getppid() returns 1. FIXME apparently this is not true for
- * solaris, could remember ppid right after fork and wait for it to change.
- */
-
-		/* if our parent went down, don't linger around */
-		if (context->started_with_parent && kill(context->started_with_parent, 0) < 0)
-			kill(getpid(), SIGTERM);
-
-	        if (getppid() == 1)
-	            break;
-//#endif
-	}
-
-
-	return 1;
-}
-
-#endif
-
 /**
  * libwebsockets_get_protocol() - Returns a protocol pointer from a websocket
  *				  connection.
  * @wsi:	pointer to struct websocket you want to know the protocol of
  *
  *
- *	This is useful to get the protocol to broadcast back to from inside
- * the callback.
+ *	Some apis can act on all live connections of a given protocol,
+ *	this is how you can get a pointer to the active protocol if needed.
  */
 
 const struct libwebsocket_protocols *
@@ -2041,159 +1910,6 @@
 	return wsi->protocol;
 }
 
-/**
- * libwebsockets_broadcast() - Sends a buffer to tx callback for all connections of given protocol from single thread
- *
- * @protocol:	pointer to the protocol you will broadcast to all members of
- * @buf:  buffer containing the data to be broadcase.  NOTE: this has to be
- *		allocated with LWS_SEND_BUFFER_PRE_PADDING valid bytes before
- *		the pointer and LWS_SEND_BUFFER_POST_PADDING afterwards in the
- *		case you are calling this function from callback context.
- * @len:	length of payload data in buf, starting from buf.
- *
- *	This function allows bulk sending of a packet to every connection using
- * the given protocol.  It does not send the data directly; instead it calls
- * the callback with a reason type of LWS_CALLBACK_BROADCAST.  If the callback
- * wants to actually send the data for that connection, the callback itself
- * should call libwebsocket_write().
- *
- * This version only works from the same thread / process context as the service
- * loop.  Use libwesockets_broadcast_foreign(...) to do the same job from a different
- * thread in a safe way.
- */
-
-
-int
-libwebsockets_broadcast(const struct libwebsocket_protocols *protocol,
-						 unsigned char *buf, size_t len)
-{
-	struct libwebsocket_context *context = protocol->owning_server;
-	int n;
-	struct libwebsocket *wsi;
-
-	if (!context)
-		return 1;
-
-	/*
-	 * We are either running unforked / flat, or we are being
-	 * called from poll thread context
-	 * eg, from a callback.  In that case don't use sockets for
-	 * broadcast IPC (since we can't open a socket connection to
-	 * a socket listening on our own thread) but directly do the
-	 * send action.
-	 *
-	 * Locking is not needed because we are by definition being
-	 * called in the poll thread context and are serialized.
-	 */
-
-	for (n = 0; n < context->fds_count; n++) {
-
-		wsi = context->lws_lookup[context->fds[n].fd];
-		if (!wsi)
-			continue;
-
-		if (wsi->mode != LWS_CONNMODE_WS_SERVING)
-			continue;
-
-		/*
-		 * never broadcast to non-established connections
-		 */
-		if (wsi->state != WSI_STATE_ESTABLISHED)
-			continue;
-
-		/* only broadcast to guys using
-		 * requested protocol
-		 */
-		if (wsi->protocol != protocol)
-			continue;
-
-		user_callback_handle_rxflow(wsi->protocol->callback,
-			 context, wsi,
-			 LWS_CALLBACK_BROADCAST,
-			 wsi->user_space,
-			 buf, len);
-	}
-
-	return 0;
-}
-
-
-#ifndef LWS_NO_FORK
-/**
- * libwebsockets_broadcast_foreign() - Sends a buffer to the callback for all active
- *				  connections of the given protocol.
- * @protocol:	pointer to the protocol you will broadcast to all members of
- * @buf:  buffer containing the data to be broadcase.  NOTE: this has to be
- *		allocated with LWS_SEND_BUFFER_PRE_PADDING valid bytes before
- *		the pointer and LWS_SEND_BUFFER_POST_PADDING afterwards in the
- *		case you are calling this function from callback context.
- * @len:	length of payload data in buf, starting from buf.
- *
- *	This function allows bulk sending of a packet to every connection using
- * the given protocol.  It does not send the data directly; instead it calls
- * the callback with a reason type of LWS_CALLBACK_BROADCAST.  If the callback
- * wants to actually send the data for that connection, the callback itself
- * should call libwebsocket_write().
- *
- * This ..._foreign() version is designed to be randomly called from other thread or
- * process contexts than the main libwebsocket service one.  A private socket is used
- * to serialize accesses here with the main service loop. 
- */
-
-int
-libwebsockets_broadcast_foreign(struct libwebsocket_protocols *protocol,
-						 unsigned char *buf, size_t len)
-{
-	struct libwebsocket_context *context = protocol->owning_server;
-	int n;
-	int fd;
-	struct sockaddr_in cli_addr;
-
-	if (!context)
-		return 1;
-
-	/*
-	 * We're being called from a different process context than the server
-	 * loop.  Instead of broadcasting directly, we send our
-	 * payload on a socket to do the IPC; the server process will serialize
-	 * the broadcast action in its main poll() loop.
-	 *
-	 * There's one broadcast socket listening for each protocol supported
-	 * set up when the websocket server initializes
-	 */
-
-
-	/*
-	 * autoconnect to this protocol's broadcast proxy socket for this
-	 * thread if needed
-	 */
-
-	if (protocol->broadcast_socket_user_fd <= 0) {
-		fd = socket(AF_INET, SOCK_STREAM, 0);
-		if (fd < 0) {
-			lwsl_err("Unable to create socket\n");
-			return -1;
-		}
-		cli_addr.sin_family = AF_INET;
-		cli_addr.sin_port = htons(protocol->broadcast_socket_port);
-		cli_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
-		n = connect(fd, (struct sockaddr *)&cli_addr, sizeof cli_addr);
-		if (n < 0) {
-			lwsl_err("Unable to connect to "
-					"broadcast socket %d, %s\n",
-					n, strerror(errno));
-			return -1;
-		}
-
-		protocol->broadcast_socket_user_fd = fd;
-	}
-
-	n = send(protocol->broadcast_socket_user_fd, buf, len, MSG_NOSIGNAL);
-
-	return n;
-}
-#endif
-
 int
 libwebsocket_is_final_fragment(struct libwebsocket *wsi)
 {
@@ -2224,39 +1940,6 @@
 	return wsi->user_space;
 }
 
-/**
- * lws_confirm_legit_wsi: returns nonzero if the wsi looks bad
- *
- * @wsi: struct libwebsocket to assess
- *
- * Performs consistecy checks on what the wsi claims and what the
- * polling arrays hold.  This'll catch a closed wsi still in use.
- * Don't try to use on the listen (nonconnection) wsi as it will
- * fail it.  Otherwise 0 return == wsi seems consistent.
- */
-
-int lws_confirm_legit_wsi(struct libwebsocket *wsi)
-{
-	struct libwebsocket_context *context;
-
-	if (!(wsi && wsi->protocol && wsi->protocol->owning_server))
-		return 1;
-
-	context = wsi->protocol->owning_server;
-
-	if (!context)
-		return 2;
-
-	if (!wsi->position_in_fds_table)
-		return 3; /* position in fds table looks bad */
-	if (context->fds[wsi->position_in_fds_table].fd != wsi->sock)
-		return 4; /* pollfd entry does not wait on our socket descriptor */
-	if (context->lws_lookup[wsi->sock] != wsi)
-		return 5; /* lookup table does not agree with wsi */
-
-	return 0;
-}
-
 static void lwsl_emit_stderr(int level, const char *line)
 {
 	char buf[300];
diff --git a/lib/libwebsockets.h b/lib/libwebsockets.h
index 4d3342c..0b08fcb 100644
--- a/lib/libwebsockets.h
+++ b/lib/libwebsockets.h
@@ -726,9 +726,6 @@
 libwebsocket_context_destroy(struct libwebsocket_context *context);
 
 LWS_EXTERN int
-libwebsockets_fork_service_loop(struct libwebsocket_context *context);
-
-LWS_EXTERN int
 libwebsocket_service(struct libwebsocket_context *context, int timeout_ms);
 
 LWS_EXTERN int
diff --git a/lib/output.c b/lib/output.c
index 69bf459..5bf0404 100644
--- a/lib/output.c
+++ b/lib/output.c
@@ -286,10 +286,6 @@
 	int m;
 #endif
 
-	if (lws_confirm_legit_wsi(wsi)) {
-		lwsl_err("libwebsocket_write on illegitimate wsi\n");
-		return -1;
-	}
 	if (len == 0 && protocol != LWS_WRITE_CLOSE) {
 		lwsl_warn("zero length libwebsocket_write attempt\n");
 		return 0;
diff --git a/lib/private-libwebsockets.h b/lib/private-libwebsockets.h
index 7e42202..0e013b3 100644
--- a/lib/private-libwebsockets.h
+++ b/lib/private-libwebsockets.h
@@ -107,9 +107,6 @@
 #ifndef MAX_USER_RX_BUFFER
 #define MAX_USER_RX_BUFFER 4096
 #endif
-#ifndef MAX_BROADCAST_PAYLOAD
-#define MAX_BROADCAST_PAYLOAD 4096
-#endif
 #ifndef LWS_MAX_PROTOCOLS
 #define LWS_MAX_PROTOCOLS 10
 #endif
@@ -217,8 +214,6 @@
 
 	/* special internal types */
 	LWS_CONNMODE_SERVER_LISTENER,
-	LWS_CONNMODE_BROADCAST_PROXY_LISTENER,
-	LWS_CONNMODE_BROADCAST_PROXY
 };
 
 struct libwebsocket_protocols;
@@ -356,9 +351,6 @@
 	struct lws_tokens utf8_token[WSI_TOKEN_COUNT];
 	
 	enum libwebsocket_write_protocol rx_frame_type;
-#ifndef LWS_NO_FORK
-	int protocol_index_for_broadcast_proxy;
-#endif
 
 #ifndef LWS_NO_CLIENT
 	char *c_path;
@@ -450,9 +442,6 @@
 _libwebsocket_rx_flow_control(struct libwebsocket *wsi);
 
 extern int
-lws_confirm_legit_wsi(struct libwebsocket *wsi);
-
-extern int
 user_callback_handle_rxflow(callback_function, struct libwebsocket_context * context,
 			struct libwebsocket *wsi,
 			 enum libwebsocket_callback_reasons reason, void *user,
diff --git a/lib/server.c b/lib/server.c
index 9aa1fe9..0a7c0dd 100644
--- a/lib/server.c
+++ b/lib/server.c
@@ -134,7 +134,7 @@
 			struct libwebsocket *wsi, struct pollfd *pollfd)
 {
 	unsigned char buf[LWS_SEND_BUFFER_PRE_PADDING + 1 +
-			 MAX_BROADCAST_PAYLOAD + LWS_SEND_BUFFER_POST_PADDING];
+			 MAX_USER_RX_BUFFER + LWS_SEND_BUFFER_POST_PADDING];
 	struct libwebsocket *new_wsi;
 	int accept_fd;
 	unsigned int clilen;
@@ -368,130 +368,6 @@
 		break;
 #endif
 
-
-#ifndef LWS_NO_FORK
-	case LWS_CONNMODE_BROADCAST_PROXY_LISTENER:
-
-		/* as we are listening, POLLIN means accept() is needed */
-
-		if (!(pollfd->revents & POLLIN))
-			break;
-
-		/* listen socket got an unencrypted connection... */
-
-		clilen = sizeof(cli_addr);
-		accept_fd  = accept(pollfd->fd, (struct sockaddr *)&cli_addr,
-								       &clilen);
-		if (accept_fd < 0) {
-			lwsl_warn("ERROR on accept %d\n", accept_fd);
-			return 0;
-		}
-
-		/* create a dummy wsi for the connection and add it */
-
-		new_wsi = (struct libwebsocket *)malloc(sizeof(struct libwebsocket));
-		if (new_wsi == NULL) {
-			lwsl_err("Out of mem\n");
-			goto bail_prox_listener;
-		}
-		memset(new_wsi, 0, sizeof (struct libwebsocket));
-		new_wsi->sock = accept_fd;
-		new_wsi->mode = LWS_CONNMODE_BROADCAST_PROXY;
-		new_wsi->state = WSI_STATE_ESTABLISHED;
-#ifndef LWS_NO_EXTENSIONS
-		new_wsi->count_active_extensions = 0;
-#endif
-		/* note which protocol we are proxying */
-		new_wsi->protocol_index_for_broadcast_proxy =
-					wsi->protocol_index_for_broadcast_proxy;
-
-		insert_wsi_socket_into_fds(context, new_wsi);
-		break;
-
-bail_prox_listener:
-		compatible_close(accept_fd);
-		break;
-
-	case LWS_CONNMODE_BROADCAST_PROXY:
-
-		/* handle session socket closed */
-
-		if (pollfd->revents & (POLLERR | POLLHUP)) {
-
-			lwsl_debug("Session Socket %p (fd=%d) dead\n",
-				(void *)wsi, pollfd->fd);
-
-			libwebsocket_close_and_free_session(context, wsi,
-						       LWS_CLOSE_STATUS_NORMAL);
-			return 0;
-		}
-
-		/*
-		 * either extension code with stuff to spill, or the user code,
-		 * requested a callback when it was OK to write
-		 */
-
-		if (pollfd->revents & POLLOUT)
-			if (lws_handle_POLLOUT_event(context, wsi,
-								 pollfd) < 0) {
-				libwebsocket_close_and_free_session(
-					context, wsi, LWS_CLOSE_STATUS_NORMAL);
-				return 0;
-			}
-
-		/* any incoming data ready? */
-
-		if (!(pollfd->revents & POLLIN))
-			break;
-
-		/* get the issued broadcast payload from the socket */
-
-		len = read(pollfd->fd, buf + LWS_SEND_BUFFER_PRE_PADDING,
-							 MAX_BROADCAST_PAYLOAD);
-		if (len < 0) {
-			lwsl_err("Error reading broadcast payload\n");
-			break;
-		}
-
-		/* broadcast it to all guys with this protocol index */
-
-		for (n = 0; n < context->fds_count; n++) {
-
-			new_wsi = context->lws_lookup[context->fds[n].fd];
-			if (new_wsi == NULL)
-				continue;
-
-			/* only to clients we are serving to */
-
-			if (new_wsi->mode != LWS_CONNMODE_WS_SERVING)
-				continue;
-
-			/*
-			 * never broadcast to non-established
-			 * connection
-			 */
-
-			if (new_wsi->state != WSI_STATE_ESTABLISHED)
-				continue;
-
-			/*
-			 * only broadcast to connections using
-			 * the requested protocol
-			 */
-
-			if (new_wsi->protocol->protocol_index !=
-				wsi->protocol_index_for_broadcast_proxy)
-				continue;
-
-			/* broadcast it to this connection */
-
-			user_callback_handle_rxflow(new_wsi->protocol->callback, context, new_wsi,
-				LWS_CALLBACK_BROADCAST,
-				new_wsi->user_space,
-				buf + LWS_SEND_BUFFER_PRE_PADDING, len);
-		}
-		break;
-#endif
 	default:
 		break;
 	}
diff --git a/libwebsockets-api-doc.html b/libwebsockets-api-doc.html
index 156afe7..402c36d 100644
--- a/libwebsockets-api-doc.html
+++ b/libwebsockets-api-doc.html
@@ -204,10 +204,7 @@
 <p>
 1) Accept new connections to our context's server
 <p>
-2) Perform pending broadcast writes initiated from other forked
-processes (effectively serializing asynchronous broadcasts)
-<p>
-3) Call the receive callback for incoming frame data received by
+2) Call the receive callback for incoming frame data received by
 server or client connections.
 <p>
 You need to call this service function periodically to all the above
@@ -400,16 +397,6 @@
 one place; they're all handled in the user callback.
 </blockquote>
 <hr>
-<h2>libwebsockets_fork_service_loop - Optional helper function forks off a process for the websocket server loop. You don't have to use this but if not, you have to make sure you are calling libwebsocket_service periodically to service the websocket traffic</h2>
-<i>int</i>
-<b>libwebsockets_fork_service_loop</b>
-(<i>struct libwebsocket_context *</i> <b>context</b>)
-<h3>Arguments</h3>
-<dl>
-<dt><b>context</b>
-<dd>server context returned by creation function
-</dl>
-<hr>
 <h2>libwebsockets_get_protocol - Returns a protocol pointer from a websocket connection.</h2>
 <i>const struct libwebsocket_protocols *</i>
 <b>libwebsockets_get_protocol</b>
@@ -422,87 +409,8 @@
 <h3>Description</h3>
 <blockquote>
 <p>
-This is useful to get the protocol to broadcast back to from inside
-the callback.
-</blockquote>
-<hr>
-<h2>libwebsockets_broadcast - Sends a buffer to tx callback for all connections of given protocol from single thread</h2>
-<i>int</i>
-<b>libwebsockets_broadcast</b>
-(<i>const struct libwebsocket_protocols *</i> <b>protocol</b>,
-<i>unsigned char *</i> <b>buf</b>,
-<i>size_t</i> <b>len</b>)
-<h3>Arguments</h3>
-<dl>
-<dt><b>protocol</b>
-<dd>pointer to the protocol you will broadcast to all members of
-<dt><b>buf</b>
-<dd>buffer containing the data to be broadcase.  NOTE: this has to be
-allocated with LWS_SEND_BUFFER_PRE_PADDING valid bytes before
-the pointer and LWS_SEND_BUFFER_POST_PADDING afterwards in the
-case you are calling this function from callback context.
-<dt><b>len</b>
-<dd>length of payload data in buf, starting from buf.
-</dl>
-<h3>Description</h3>
-<blockquote>
-This function allows bulk sending of a packet to every connection using
-the given protocol.  It does not send the data directly; instead it calls
-the callback with a reason type of LWS_CALLBACK_BROADCAST.  If the callback
-wants to actually send the data for that connection, the callback itself
-should call <b>libwebsocket_write</b>.
-<p>
-This version only works from the same thread / process context as the service
-loop.  Use libwesockets_broadcast_foreign(...) to do the same job from a different
-thread in a safe way.
-</blockquote>
-<hr>
-<h2>libwebsockets_broadcast_foreign - Sends a buffer to the callback for all active connections of the given protocol.</h2>
-<i>int</i>
-<b>libwebsockets_broadcast_foreign</b>
-(<i>struct libwebsocket_protocols *</i> <b>protocol</b>,
-<i>unsigned char *</i> <b>buf</b>,
-<i>size_t</i> <b>len</b>)
-<h3>Arguments</h3>
-<dl>
-<dt><b>protocol</b>
-<dd>pointer to the protocol you will broadcast to all members of
-<dt><b>buf</b>
-<dd>buffer containing the data to be broadcase.  NOTE: this has to be
-allocated with LWS_SEND_BUFFER_PRE_PADDING valid bytes before
-the pointer and LWS_SEND_BUFFER_POST_PADDING afterwards in the
-case you are calling this function from callback context.
-<dt><b>len</b>
-<dd>length of payload data in buf, starting from buf.
-</dl>
-<h3>Description</h3>
-<blockquote>
-This function allows bulk sending of a packet to every connection using
-the given protocol.  It does not send the data directly; instead it calls
-the callback with a reason type of LWS_CALLBACK_BROADCAST.  If the callback
-wants to actually send the data for that connection, the callback itself
-should call <b>libwebsocket_write</b>.
-<p>
-This ...<b>_foreign</b> version is designed to be randomly called from other thread or
-process contexts than the main libwebsocket service one.  A private socket is used
-to serialize accesses here with the main service loop. 
-</blockquote>
-<hr>
-<h2>lws_confirm_legit_wsi - </h2>
-<i>int</i>
-<b>lws_confirm_legit_wsi</b>
-(<i>struct libwebsocket *</i> <b>wsi</b>)
-<h3>Arguments</h3>
-<dl>
-<dt><b>wsi</b>
-<dd>struct libwebsocket to assess
-</dl>
-<h3>Description</h3>
-<blockquote>
-Performs consistecy checks on what the wsi claims and what the
-polling arrays hold.  This'll catch a closed wsi still in use.
-Don't try to use on the listen (nonconnection) wsi as it will
-fail it.  Otherwise 0 return == wsi seems consistent.
+Some apis can act on all live connections of a given protocol,
+this is how you can get a pointer to the active protocol if needed.
 </blockquote>
 <hr>
 <h2>lws_set_log_level - Set the logging bitfield</h2>
diff --git a/test-server/test-server.c b/test-server/test-server.c
index 2fa0eae..505c6ee 100644
--- a/test-server/test-server.c
+++ b/test-server/test-server.c
@@ -31,20 +31,12 @@
 #include "../lib/libwebsockets.h"
 
 static int close_testing;
-
-#ifdef EXTERNAL_POLL
-#ifndef LWS_NO_FORK
-#define LWS_NO_FORK
-#endif
-
 int max_poll_elements;
 
 struct pollfd *pollfds;
 int *fd_lookup;
 int count_pollfds;
 
-#endif /* EXTERNAL_POLL */
-
 
 /*
  * This demo server shows how to use libwebsockets for one or more
@@ -277,13 +269,7 @@
 		pss->number = 0;
 		break;
 
-	/*
-	 * in this protocol, we just use the broadcast action as the chance to
-	 * send our own connection-specific data and ignore the broadcast info
-	 * that is available in the 'in' parameter
-	 */
-
-	case LWS_CALLBACK_BROADCAST:
+	case LWS_CALLBACK_SERVER_WRITEABLE:
 		n = sprintf((char *)p, "%d", pss->number++);
 		n = libwebsocket_write(wsi, p, n, LWS_WRITE_TEXT);
 		if (n < 0) {
@@ -396,12 +382,6 @@
 		}
 		break;
 
-	case LWS_CALLBACK_BROADCAST:
-		n = libwebsocket_write(wsi, in, len, LWS_WRITE_TEXT);
-		if (n < 0)
-			lwsl_err("mirror write failed\n");
-		break;
-
 	case LWS_CALLBACK_RECEIVE:
 
 		if (((ringbuffer_head - pss->ringbuffer_tail) &
@@ -504,8 +484,6 @@
 			    LOCAL_RESOURCE_PATH"/libwebsockets-test-server.pem";
 	const char *key_path =
 			LOCAL_RESOURCE_PATH"/libwebsockets-test-server.key.pem";
-	unsigned char buf[LWS_SEND_BUFFER_PRE_PADDING + 1024 +
-						  LWS_SEND_BUFFER_POST_PADDING];
 	int port = 7681;
 	int use_ssl = 0;
 	struct libwebsocket_context *context;
@@ -513,9 +491,8 @@
 	char interface_name[128] = "";
 	const char *interface = NULL;
 	int syslog_options = LOG_PID | LOG_PERROR;
-#ifdef LWS_NO_FORK
 	unsigned int oldus = 0;
-#endif
+
 	int debug_level = 7;
 #ifndef LWS_NO_DAEMONIZE
 	int daemonize = 0;
@@ -605,16 +582,6 @@
 		return -1;
 	}
 
-	buf[LWS_SEND_BUFFER_PRE_PADDING] = 'x';
-
-#ifdef LWS_NO_FORK
-
-	/*
-	 * This example shows how to work with no forked service loop
-	 */
-
-	lwsl_info(" Using no-fork service loop\n");
-
 	n = 0;
 	while (n >= 0) {
 		struct timeval tv;
@@ -622,22 +589,13 @@
 		gettimeofday(&tv, NULL);
 
 		/*
-		 * This broadcasts to all dumb-increment-protocol connections
-		 * at 20Hz.
-		 *
-		 * We're just sending a character 'x', in these examples the
-		 * callbacks send their own per-connection content.
-		 *
-		 * You have to send something with nonzero length to get the
-		 * callback actions delivered.
-		 *
-		 * We take care of pre-and-post padding allocation.
+		 * This provokes the LWS_CALLBACK_SERVER_WRITEABLE for every
+		 * live websocket connection using the DUMB_INCREMENT protocol,
+		 * as soon as it can take more packets (usually immediately)
 		 */
 
 		if (((unsigned int)tv.tv_usec - oldus) > 50000) {
-			libwebsockets_broadcast(
-					&protocols[PROTOCOL_DUMB_INCREMENT],
-					&buf[LWS_SEND_BUFFER_PRE_PADDING], 1);
+			libwebsocket_callback_on_writable_all_protocol(&protocols[PROTOCOL_DUMB_INCREMENT]);
 			oldus = tv.tv_usec;
 		}
 
@@ -679,49 +637,6 @@
 #endif
 	}
 
-#else /* !LWS_NO_FORK */
-
-	/*
-	 * This example shows how to work with the forked websocket service loop
-	 */
-
-	lwsl_info(" Using forked service loop\n");
-
-	/*
-	 * This forks the websocket service action into a subprocess so we
-	 * don't have to take care about it.
-	 */
-
-	n = libwebsockets_fork_service_loop(context);
-	if (n < 0) {
-		lwsl_err("Unable to fork service loop %d\n", n);
-		return 1;
-	}
-	if (n)
-		exit(0);
-
-	while (1) {
-
-		usleep(50000);
-
-		/*
-		 * This broadcasts to all dumb-increment-protocol connections
-		 * at 20Hz.
-		 *
-		 * We're just sending a character 'x', in these examples the
-		 * callbacks send their own per-connection content.
-		 *
-		 * You have to send something with nonzero length to get the
-		 * callback actions delivered.
-		 *
-		 * We take care of pre-and-post padding allocation.
-		 */
-
-		libwebsockets_broadcast_foreign(&protocols[PROTOCOL_DUMB_INCREMENT],
-					&buf[LWS_SEND_BUFFER_PRE_PADDING], 1);
-	}
-
-#endif
 #ifdef EXTERNAL_POLL
 done:
 #endif