blob: 8ce883c2ecffacfcd544ff3fc623e3c8b071dbc0 [file] [log] [blame]
Andy Greena35c86f2013-01-31 10:16:44 +08001Changelog
2---------
3
Andy Greenac032542018-11-16 09:10:41 +08004v3.1.0
5======
6
Andy Green502130d2018-05-10 16:13:26 +08007 - CHANGE: REMOVED: lws_client_connect() and lws_client_connect_extended()
8 compatibility apis for lws_client_connect_via_info() have been marked as
9 deprecated for several versions and are now removed. Use
Andy Greenac032542018-11-16 09:10:41 +080010 lws_client_connect_via_info() directly instead.
Andy Green502130d2018-05-10 16:13:26 +080011
Andy Greenac032542018-11-16 09:10:41 +080012 - CHANGE: CMAKE:
13 - LWS_WITH_HTTP2: now defaults ON
14
Andy Greend9f982a2018-11-19 07:40:47 +080015 - CHANGE: Minimal examples updated to use Content Security Policy best
16 practices, using
17 `LWS_SERVER_OPTION_HTTP_HEADERS_SECURITY_BEST_PRACTICES_ENFORCE` vhost
18 option flag and disabling of inline style and scripts.
19
Andy Greenac032542018-11-16 09:10:41 +080020 - NEW: CMAKE
21 - LWS_FOR_GITOHASHI: sets various cmake options suitable for gitohashi
22 - LWS_WITH_ASAN: for Linux, enable build with ASAN
23
24 Don't forget LWS_WITH_DISTRO_RECOMMENDED, which enables a wide range of lws
25 options suitable for a distro build of the library.
26
27 - NEW: lws threadpool - lightweight pool of pthreads integrated to lws wsi, with
28 all synchronization to event loop handled internally, queue for excess tasks
29 [threadpool docs](https://libwebsockets.org/git/libwebsockets/tree/lib/misc/threadpool)
30 [threadpool minimal example](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-threadpool)
31 Cmake config: `-DLWS_WITH_THREADPOOL=1`
32
33 - NEW: libdbus support integrated on lws event loop
34 [lws dbus docs](https://libwebsockets.org/git/libwebsockets/tree/lib/roles/dbus)
35 [lws dbus client minimal examples](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/dbus-client)
36 [lws dbus server minimal examples](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/dbus-server)
37 Cmake config: `-DLWS_ROLE_DBUS=1`
38
39 - NEW: lws allocated chunks (lwsac) - helpers for optimized mass allocation of small
40 objects inside a few larger malloc chunks... if you need to allocate a lot of
41 inter-related structs for a limited time, this removes per-struct allocation
42 library overhead completely and removes the need for any destruction handling
43 [lwsac docs](https://libwebsockets.org/git/libwebsockets/tree/lib/misc/lwsac)
44 [lwsac minimal example](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/api-tests/api-test-lwsac)
45 Cmake Config: `-DLWS_WITH_LWSAC=1`
46
47 - NEW: lws tokenizer - helper api for robustly tokenizing your own strings without
48 allocating or adding complexity. Configurable by flags for common delimiter
49 sets and comma-separated-lists in the tokenizer. Detects and reports syntax
50 errors.
51 [lws_tokenize docs](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-tokenize.h)
52 [lws_tokenize minimal example / api test](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/api-tests/api-test-lws_tokenize)
53
54 - NEW: lws full-text search - optimized trie generation, serialization,
55 autocomplete suggestion generation and instant global search support extensible
56 to huge corpuses of UTF-8 text while remaining super lightweight on resources.
57 [full-text search docs](https://libwebsockets.org/git/libwebsockets/tree/lib/misc/fts)
58 [full-text search minimal example / api test](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/api-tests/api-test-fts)
59 [demo](https://libwebsockets.org/ftsdemo/)
60 [demo sources](https://libwebsockets.org/git/libwebsockets/tree/plugins/protocol_fulltext_demo.c)
61 Cmake config: `-DLWS_WITH_FTS=1 -DLWS_WITH_LWSAC=1`
62
63 - NEW: gzip + brotli http server-side compression - h1 and h2 automatic advertising
64 of server compression and application to files with mimetypes "text/*",
65 "application/javascript" and "image/svg.xml".
66 Cmake config: `-DLWS_WITH_HTTP_STREAM_COMPRESSION=1`, `-DLWS_WITH_HTTP_BROTLI=1`
67
68 - NEW: managed disk cache - API for managing a directory containing cached files
69 with hashed names, and automatic deletion of LRU files once the cache is
70 above a given limit.
71 [lws diskcache docs](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-diskcache.h)
72 Cmake config: `-DLWS_WITH_DISKCACHE=1`
73
74 - NEW: http reverse proxy - lws mounts support proxying h1 or h2 requests to
75 a local or remote IP, or unix domain socket over h1. This allows microservice
76 type architectures where parts of the common URL space are actually handled
77 by external processes which may be remote or on the same machine.
78 [lws gitohashi serving](https://libwebsockets.org/git/) is handled this way.
79 CMake config: `-DLWS_WITH_HTTP_PROXY=1`
80
81 - NEW: lws_buflist - internally several types of ad-hoc malloc'd buffer have
82 been replaced by a new, exported api `struct lws_buflist`. This allows
83 multiple buffers to be chained and drawn down in strict FIFO order.
84
85 - NEW: In the case of h1 upgrade, the connection header is checked to contain
86 "upgrade". The vhost flag LWS_SERVER_OPTION_VHOST_UPG_STRICT_HOST_CHECK
87 also causes the Host: header to be confirmed to match the vhost name and
88 listen port.
Andy Greenc027c3d2018-11-21 06:46:23 +080089
90 - NEW: If no 404 redirect for `lws_return_http_status()` is specified for the vhost,
91 the status page produced will try to bring in a stylesheet `/error.css`. This allows
92 you to produce styled 404 or other error pages with logos, graphics etc. See
93 https://libwebsockets.org/git/badrepo for an example of what you can do with it.
Andy Green502130d2018-05-10 16:13:26 +080094
Andy Green1c08a962018-05-04 13:27:12 +080095v3.0.0
96======
97
98 - CHANGE: Clients used to call LWS_CALLBACK_CLOSED same as servers...
99 LWS_CALLBACK_CLIENT_CLOSED has been introduced and is called for clients
100 now.
101
102 - CHANGE: LWS_CALLBACK_CLIENT_CONNECTION_ERROR used to only be directed at
103 protocols[0]. However in many cases, the protocol to bind to was provided
104 at client connection info time and the wsi bound accordingly. In those
105 cases, CONNECTION_ERROR is directed at the bound protocol, not protcols[0]
106 any more.
107
108 - CHANGE: CMAKE: the following cmake defaults have changed with this version:
109
110 - LWS_WITH_ZIP_FOPS: now defaults OFF
111 - LWS_WITH_RANGES: now defaults OFF
112 - LWS_WITH_ZLIB: now defaults OFF
113 - LWS_WITHOUT_EXTENSIONS: now defaults ON
114
115 - CHANGE: REMOVED: lws_alloc_vfs_file() (read a file to malloc buffer)
116
117 - CHANGE: REMOVED: lws_read() (no longer useful outside of lws internals)
118
119 - CHANGE: REMOVED: ESP8266... ESP32 is now within the same price range and much
120 more performant
121
122 - CHANGE: soname bump... don't forget to `ldconfig`
123
124 - NEW: all event libraries support "foreign" loop integration where lws itself
125 if just a temporary user of the loop unrelated to the actual loop lifecycle.
126
127 See `minimal-http-server-eventlib-foreign` for example code demonstrating
128 this for all the event libraries.
129
130 Internal loop in lws is also supported and demonstrated by
131 `minimal-http-server-eventlib`.
132
133 - NEW: ws-over-h2 support. This is a new RFC-on-the-way supported by Chrome
134 and shortly firefox that allows ws connections to be multiplexed back to the
135 server on the same tcp + tls wrapper h2 connection that the html and scripts
136 came in on. This is hugely faster that discrete connections.
137
138 - NEW: UDP socket adoption and related event callbacks
139
140 - NEW: Multi-client connection binding, queuing and pipelining support.
141
142 Lws detects multiple client connections to the same server and port, and
143 optimizes how it handles them according to the server type and provided
144 flags. For http/1.0, all occur with individual parallel connections. For
145 http/1.1, you can enable keepalive pipelining, so the connections occur
146 sequentially on a single network connection. For http/2, they all occur
147 as parallel streams within a single h2 network connection.
148
149 See minimal-http-client-multi for example code.
150
151 - NEW: High resolution timer API for wsi, get a callback on your wsi with
152 LWS_CALLBACK_TIMER, set and reset the timer with lws_set_timer_usecs(wsi, us)
153 Actual resolution depends on event backend. Works with all backends, poll,
154 libuv, libevent, and libev.
155
156 - NEW: Protocols can arrange vhost-protocol instance specific callbacks with
157 second resolution using `lws_timed_callback_vh_protocol()`
158
159 - NEW: ACME client plugin for self-service TLS certificates
160
161 - NEW: RFC7517 JSON Web Keys RFC7638 JWK thumbprint, and RFC7515 JSON Web
162 signatures support
163
164 - NEW: lws_cancel_service() now provides a generic way to synchronize events
165 from other threads, which appear as a LWS_CALLBACK_EVENT_WAIT_CANCELLED
166 callback on all protocols. This is compatible with all the event libraries.
167
168 - NEW: support BSD poll() where changes to the poll wait while waiting are
169 undone.
170
171 - NEW: Introduce generic hash, hmac and RSA apis that operate the same
172 regardless of OpenSSL or mbedTLS tls backend
173
174 - NEW: Introduce X509 element query api that works the same regardless of
175 OpenSSL or mbedTLS tls backend
176
177 - NEW: Introduce over 30 "minimal examples" in ./minimal-examples... these
178 replace most of the old test servers
179
180 - test-echo -> minimal-ws-server-echo and minimal-ws-client-echo
181
182 - test-server-libuv / -libevent / -libev ->
183 minimal-https-server-eventlib / -eventlib-foreign / -eventlib-demos
184
185 - test-server-v2.0 -> folded into all the minimal servers
186
187 - test-server direct http serving -> minimal-http-server-dynamic
188
189 The minimal examples allow individual standalone build using their own
190 small CMakeLists.txt.
191
192 - NEW: lws now detects any back-to-back writes that did not go through the
193 event loop inbetween and reports them. This will flag any possibility of
194 failure rather than wait until the problem happens.
195
196 - NEW: CMake has LWS_WITH_DISTRO_RECOMMENDED to select features that are
197 appropriate for distros
198
199 - NEW: Optional vhost URL `error_document_404` if given causes a redirect there
200 instead of serve the default 404 page.
201
202 - NEW: lws_strncpy() wrapper guarantees NUL in copied string even if it was
203 truncated to fit.
204
205 - NEW: for client connections, local protocol binding name can be separated
206 from the ws subprotocol name if needed, using .local_protocol_name
207
208 - NEW: Automatic detection of time discontiguities
209
210 - NEW: Applies TCP_USER_TIMEOUT for Linux tcp keepalive where available
211
212 - QA: 1600 tests run on each commit in Travis CI, including almost all
213 Autobahn in client and server mode, various h2load tests, h2spec, attack.sh
214 the minimal example selftests and others.
215
216 - QA: fix small warnings introduced on gcc8.x (eg, Fedora 28)
217
218 - QA: Add most of -Wextra on gcc (-Wsign-compare, -Wignored-qualifiers,
219 -Wtype-limits, -Wuninitialized)
220
221 - QA: clean out warnings on windows
222
223 - QA: pass all 146 h2spec tests now on strict
224
225 - QA: introduce 35 selftests that operate different minimal examples against
226 each other and confirm the results.
227
228 - QA: LWS_WITH_MINIMAL_EXAMPLES allows mass build of all relevant minimal-
229 examples with the LWS build, for CI and to make all the example binaries
230 available from the lws build dir ./bin
231
232 - REFACTOR: the lws source directory layout in ./lib has been radically
233 improved, and there are now README.md files in selected subdirs with extra
234 documentation of interest to people working on lws itself.
235
236 - REFACTOR: pipelined transactions return to the event loop before starting the
237 next part.
238
239 - REFACTOR: TLS: replace all TLS library constants with generic LWS ones and
240 adapt all the TLS library code to translate to these common ones.
241
242 Isolated all the tls-related private stuff in `./lib/tls/private.h`, and all
243 the mbedTLS stuff in `./lib/tls/mbedtls` + openSSL stuff in
244 `./lib/tls/openssl`
245
246 - REFACTOR: the various kinds of wsi possible with lws have been extracted
247 from the main code and isolated into "roles" in `./lib/roles` which
248 communicate with the core code via an ops struct. Everything related to
249 ah is migrated to the http role.
250
251 wsi modes are eliminated and replaced by the ops pointer for the role the
252 wsi is performing. Generic states for wsi are available to control the
253 lifecycle using core code.
254
255 Adding new "roles" is now much easier with the changes and ops struct to
256 plug into.
257
258 - REFACTOR: reduce four different kinds of buffer management in lws into a
259 generic scatter-gather struct lws_buflist.
260
261 - REFACTOR: close notifications go through event loop
262
263
Andy Greenfcf5b2c2017-10-16 20:09:58 +0800264v2.4.0
265======
266
267 - HTTP/2 server support is now mature and usable! LWS_WITH_HTTP2=1 enables it.
268 Uses ALPN to serve HTTP/2, HTTP/1 and ws[s] connections all from the same
269 listen port seamlessly. (Requires ALPN-capable OpenSSL 1.1 or mbedTLS).
270
271 - LWS_WITH_MBEDTLS=1 at CMake now builds and works against mbedTLS instead of
272 OpenSSL. Most things work identically, although on common targets where
273 OpenSSL has acceleration, mbedTLS is many times slower in operation. However
274 it is a lot smaller codewise.
275
276 - Generic hash apis introduced that work the same on mbedTLS or OpenSSL backend
277
278 - LWS_WITH_PEER_LIMITS tracks IPs across all vhosts and allows restrictions on
279 both the number of simultaneous connections and wsi in use for any single IP
280
281 - lws_ring apis provide a generic single- or multi-tail ringbuffer... mirror
282 protocol now uses this. Features include ring elements may be sized to fit
283 structs in the ringbuffer, callback when no tail any longer needs an element
284 and it can be deleted, and zerocopy options to write new members directly
285 into the ringbuffer, and use the ringbuffer element by address too.
286
287 - abstract ssh 2 server plugin included, with both plugin and standalone
288 demos provided. You can bind the plugin to a vhost and also serve full-
289 strength ssh from the vhost. IO from the ssh server is controlled by an
290 "ops" struct of callbacks for tx, rx, auth etc.
291
292 - Many fixes, cleanups, source refactors and other improvements.
293
294
Andy Greene6bd6292017-07-28 14:13:42 +0800295v2.3.0
296======
297
298 - ESP32 OpenSSL support for client and server
299
300 - ESP32 4 x WLAN credential slots may be configured
301
302 - Libevent event loop support
303
304 - SOCKS5 proxy support
305
306 - lws_meta protocol for websocket connection multiplexing
307
308 - lws_vhost_destroy() added... allows dynamic removal of listening
309 vhosts. Vhosts with shared listen sockets adopt the listen socket
310 automatically if the owner is destroyed.
311
312 - IPv6 on Windows
313
314 - Improved CGI handling suitable for general CGI scripting, eg, PHP
315
316 - Convert even the "old style" test servers to use statically included
317 plugin sources
318
319 - LWS_WITH_STATS cmake option dumps resource usage and timing information
320 every few seconds to debug log, including latency information about
321 delay from asking for writeable callback to getting it
322
323 - Large (> 2GB) files may be served
324
325 - LWS_WITH_HTTP_PROXY Cmake option adds proxying mounts
326
327 - Workaround for libev build by disabling -Werror on the test app
328
329 - HTTP2 support disabled since no way to serve websockets on it
330
331
Andy Green6be573f2017-03-06 15:35:45 +0800332v2.2.0
333======
334
335Major new features
336
337 - A mount can be protected by Basic Auth... in lwsws it looks like this
338
339 ```
340{
341 "mountpoint": "/basic-auth",
342 "origin": "file://_lws_ddir_/libwebsockets-test-server/private",
343 "basic-auth": "/var/www/balogins-private"
344}
345```
346
347The text file named in `basic-auth` contains user:password information
348one per line.
349
350See README.lwsws.md for more information.
351
352 - RFC7233 RANGES support in lws server... both single and multipart.
353 This allows seeking for multimedia file serving and download resume.
354 It's enabled by default but can be disabled by CMake option.
355
356 - On Linux, lwsws can reload configuration without dropping ongoing
357 connections, when sent a SIGHUP. The old configuration drops its
358 listen sockets so the new configuration can listen on them.
359 New connections connect to the server instance with the new
360 configuration. When all old connections eventually close, the old
361 instance automatically exits. This is equivalent to
362 `systemctl reload apache`
363
364 - New `adopt` api allow adoption including SSL negotiation and
365 for raw sockets and file descriptors.
366
367 - Chunked transfer encoding supported for client and server
368
369 - Adaptations to allow operations inside OPTEE Secure World
370
371 - ESP32 initial port - able to do all test server functions. See
372 README.build.md
373
374 - Serving gzipped files from inside a ZIP file is supported... this
375 includes directly serving the gzipped content if the client
376 indicated it could accept it (ie, almost all browsers) saving
377 bandwidth and time. For clients that can't accept it, lws
378 automatically decompresses and serves the content in memory-
379 efficient chunks. Only a few hundred bytes of heap are needed
380 to serve any size file from inside the zip. See README.coding.md
381
382 - RAW file descriptors may now be adopted into the lws event loop,
383 independent of event backend (including poll service).
384 See README.coding.md
385
386 - RAW server socket descriptors may now be enabled on the vhost if
387 the first thing sent on the connection is not a valid http method.
388 The user code can associate these with a specific protocol per
389 vhost, and RAW-specific callbacks appear there for creation, rx,
390 writable and close. See libwebsockets-test-server-v2.0 for an example.
391 See README.coding.md
392
393 - RAW client connections are now possible using the method "RAW".
394 After connection, the socket is associated to the protocol
395 named in the client connection info and RAW-specific callbacks
396 appear there for creation, rx, writable and close.
397 See libwebsockets-test-client (with raw://) for an example.
398 See README.coding.md
399
400
Andy Green73557502016-10-06 21:48:20 +0800401v2.1.0
402======
403
404Major new features
405
406 - Support POST arguments, including multipart and file attachment
407
408 - Move most of lwsws into lws, make the stub CC0
409
410 - Add loopback test plugin to confirm client ws / http coexistence
411
412 - Integrate lwsws testing on Appveyor (ie, windows)
413
414 - Introduce helpers for sql, urlencode and urldecode sanitation
415
416 - Introduce LWS_CALLBACK_HTTP_BIND_PROTOCOL / DROP_PROTOCOL that
417 are compatible with http:/1.1 pipelining and different plugins
418 owning different parts of the URL space
419
420 - lwsgs - Generic Sessions plugin supports serverside sessions,
421 cookies, hashed logins, forgot password etc
422
423 - Added APIs for sending email to SMTP servers
424
425 - Messageboard example plugin for lwsgs
426
427 - Automatic PING sending at fixed intervals and close if no response
428
429 - Change default header limit in ah to 4096 (from 1024)
430
431 - Add SNI matching for wildcards if no specific wildcard vhost name match
432
433 - Convert docs to Doxygen
434
435 - ESP8266 support ^^
436
Enno Boland7731a3e2016-05-05 22:08:41 +0200437Fixes
438-----
439
Andy Green73557502016-10-06 21:48:20 +0800440See git log v2.0.0..
Enno Boland7731a3e2016-05-05 22:08:41 +0200441
Andy Greenfb8be052016-05-12 19:39:29 +0800442
Enno Boland7731a3e2016-05-05 22:08:41 +0200443
Andy Green55006432016-05-05 09:40:18 +0800444v2.0.0
445======
446
447Summary
448-------
449
450 - There are only api additions, the api is compatible with v1.7.x. But
451 there is necessarily an soname bump to 8.
452
453 - If you are using lws client, you mainly need to be aware the option
454 LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT is needed at context-creation time
455 if you will use SSL.
456
457 - If you are using lws for serving, the above is also true but there are
458 many new features to simplify your code (and life). There is a
459 summany online here
460
461 https://libwebsockets.org/lws-2.0-new-features.html
462
463 but basically the keywords are vhosts, mounts and plugins. You can now
464 do the web serving part from lws without any user callback code at all.
465 See ./test-server/test-server-v2.0.c for an example, it has no user
466 code for ws either since it uses the protocol plugins... that one C file
467 is all that is needed to do the whole test server function.
468
469 You now have the option to use a small generic ws-capable webserver
470 "lwsws" and write your ws part as a plugin. That eliminates even
471 cut-and-pasting the test server code and offers more configurable
472 features like control over http cacheability in JSON.
473
474
Andy Green0c3cc2e2016-02-20 09:12:52 +0800475Fixes
476-----
477
Andy Green55006432016-05-05 09:40:18 +0800478These are already in 1.7.x series
479
Andy Green0c3cc2e2016-02-20 09:12:52 +08004801) MAJOR (Windows-only) fix assert firing
481
4822) MAJOR http:/1.1 connections handled by lws_return_http_status() did not
483get sent a content-length resulting in the link hanging until the peer closed
484it. attack.sh updated to add a test for this.
485
Andy Green26d42492016-02-24 12:40:21 +08004863) MINOR An error about hdr struct in _lws_ws_related is corrected, it's not
487known to affect anything until after it was fixed
488
Andy Green5c0bcf42016-02-24 21:27:46 +08004894) MINOR During the close shutdown wait state introduced at v1.7, if something
490requests callback on writeable for the socket it will busywait until the
491socket closes
492
Andy Greendbfbbb42016-02-24 20:58:19 +08004935) MAJOR Although the test server has done it for a few versions already, it
494is now required for the user code to explicitly call
495
496 if (lws_http_transaction_completed(wsi))
497 return -1;
498
499when it finishes replying to a transaction in http. Previously the library
500did it for you, but that disallowed large, long transfers with multiple
501trips around the event loop (and cgi...).
502
Andy Green4ba798d2016-02-25 21:50:49 +08005036) MAJOR connections on ah waiting list that closed did not get removed from
504the waiting list...
505
Andy Green83af28a2016-02-28 10:55:31 +08005067) MAJOR since we added the ability to hold an ah across http keepalive
507transactions where more headers had already arrived, we broke the ability
508to tell if more headers had arrived. Result was if the browser didn't
509close the keepalive, we retained ah for the lifetime of the keepalive,
510using up the pool.
511
Andy Green442e1c82016-02-29 10:10:42 +08005128) MAJOR windows-only-POLLHUP was not coming
513
Andy Green4f5ebec2016-03-09 23:13:31 +08005149) Client should not send ext hdr if no exts
Andy Greendbfbbb42016-02-24 20:58:19 +0800515
Andy Green0c3cc2e2016-02-20 09:12:52 +0800516Changes
517-------
518
5191) MINOR test-server gained some new switches
520
521 -C <file> use external SSL cert file
522 -K <file> use external SSL key file
523 -A <file> use external SSL CA cert file
524
525 -u <uid> set effective uid
526 -g <gid> set effective gid
527
528together you can use them like this to have the test-server work with the
529usual purchased SSL certs from an official CA.
530
531 --ssl -C your.crt -K your.key -A your.cer -u 99 -g 99
532
5332) MINOR the OpenSSL magic to setup ECDH cipher usage is implemented in the
534library, and the ciphers restricted to use ECDH only.
535Using this, the lws test server can score an A at SSLLABS test
536
5373) MINOR STS (SSL always) header is added to the test server if you use --ssl. With
538that, we score A+ at SSLLABS test
539
5404) MINOR daemonize function (disabled at cmake by default) is updated to work
541with systemd
542
5435) MINOR example systemd .service file now provided for test server
544(not installed by default)
545
Andy Green0ad1a6e2016-02-20 14:05:55 +08005466) test server html is updated with tabs and a new live server monitoring
547feature. Input sanitization added to the js.
548
Andy Green2d8d35a2016-02-29 14:19:16 +08005497) client connections attempted when no ah is free no longer fail, they are
550just deferred until an ah becomes available.
551
Andy Greena661ee52016-02-29 13:18:30 +08005528) The test client pays attention to if you give it an http:/ or https://
553protocol string to its argument in URL format. If so, it stays in http[s]
554client mode and doesn't upgrade to ws[s], allowing you to do generic http client
Andy Green5c8906e2016-03-13 16:44:19 +0800555operations. Receiving transfer-encoding: chunked is supported.
Andy Greena661ee52016-02-29 13:18:30 +0800556
Andy Green1e5a9ad2016-03-20 11:59:53 +08005579) If you enable -DLWS_WITH_HTTP_PROXY=1 at cmake, the test server has a
558new URI path http://localhost:7681/proxytest If you visit here, a client
559connection to http://example.com:80 is spawned, and the results piped on
560to your original connection.
561
56210) Also with LWS_WITH_HTTP_PROXY enabled at cmake, lws wants to link to an
563additional library, "libhubbub". This allows lws to do html rewriting on the
564fly, adjusting proxied urls in a lightweight and fast way.
Andy Greenfb5f33b2016-03-01 07:19:01 +0800565
Andy Greenc6fd3602016-03-23 09:22:11 +080056611) There's a new context creation flag LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT,
567this is included automatically if you give any other SSL-related option flag.
568If you give no SSL-related option flag, nor this one directly, then even
569though SSL support may be compiled in, it is never initialized nor used for the
570whole lifetime of the lws context.
571
572Conversely in order to prepare the context to use SSL, even though, eg, you
573are not listening on SSL but will use SSL client connections later, you must
574give this flag explicitly to make sure SSL is initialized.
575
Andy Green2d8d35a2016-02-29 14:19:16 +0800576
Andy Greend7fddad2016-02-18 20:36:55 +0800577User API additions
578------------------
579
Andy Green0c3cc2e2016-02-20 09:12:52 +08005801) MINOR APIBREAK There's a new member in struct lws_context_creation_info, ecdh_curve,
Andy Greend7fddad2016-02-18 20:36:55 +0800581which lets you set the name of the ECDH curve OpenSSL should use. By
582default (if you leave ecdh_curve NULL) it will use "prime256v1"
583
Andy Green51d9afa2016-02-24 11:05:56 +08005842) MINOR NEWAPI It was already possible to adopt a foreign socket that had not
585been read from using lws_adopt_socket() since v1.7. Now you can adopt a
586partially-used socket if you don't need SSL, by passing it what you read
587so it can drain that before reading from the socket.
588
589LWS_VISIBLE LWS_EXTERN struct lws *
590lws_adopt_socket_readbuf(struct lws_context *context, lws_sockfd_type accept_fd,
591 const char *readbuf, size_t len);
Andy Greend7fddad2016-02-18 20:36:55 +0800592
Andy Green6a8099b2016-02-21 21:25:48 +08005933) MINOR NEWAPI CGI type "network io" subprocess execution is now possible from
594a simple api.
595
596LWS_VISIBLE LWS_EXTERN int
Andy Green1a138852016-03-20 11:55:25 +0800597lws_cgi(struct lws *wsi, char * const *exec_array, int script_uri_path_len,
598 int timeout_secs);
Andy Green6a8099b2016-02-21 21:25:48 +0800599
600LWS_VISIBLE LWS_EXTERN int
601lws_cgi_kill(struct lws *wsi);
602
603To use it, you must first set the cmake option
604
605$ cmake .. -DLWS_WITH_CGI=1
606
607See test-server-http.c and test server path
608
609http://localhost:7681/cgitest
610
611stdin gets http body, you can test it with wget
612
613$ echo hello > hello.txt
614$ wget http://localhost:7681/cgitest --post-file=hello.txt -O- --quiet
615lwstest script
616read="hello"
617
Andy Green1a138852016-03-20 11:55:25 +0800618The test script returns text/html table showing /proc/meminfo. But the cgi
619support is complete enough to run cgit cgi.
620
Andy Greend61bed32016-02-25 15:01:55 +08006214) There is a helper api for forming logging timestamps
622
623LWS_VISIBLE int
624lwsl_timestamp(int level, char *p, int len)
625
626this generates this kind of timestamp for use as logging preamble
627
628lwsts[13116]: [2016/01/25 14:52:52:8386] NOTICE: Initial logging level 7
629
Andy Greena661ee52016-02-29 13:18:30 +08006305) struct lws_client_connect_info has a new member
631
632 const char *method
633
634If it's NULL, then everything happens as before, lws_client_connect_via_info()
635makes a ws or wss connection to the address given.
636
637If you set method to a valid http method like "GET", though, then this method
638is used and the connection remains in http[s], it's not upgraded to ws[s].
639
640So with this, you can perform http[s] client operations as well as ws[s] ones.
641
642There are 4 new related callbacks
643
644 LWS_CALLBACK_ESTABLISHED_CLIENT_HTTP = 44,
645 LWS_CALLBACK_CLOSED_CLIENT_HTTP = 45,
646 LWS_CALLBACK_RECEIVE_CLIENT_HTTP = 46,
647 LWS_CALLBACK_COMPLETED_CLIENT_HTTP = 47,
Andy Green6a8099b2016-02-21 21:25:48 +0800648
Andy Green494418a2016-03-02 09:17:22 +08006496) struct lws_client_connect_info has a new member
650
651 const char *parent_wsi
652
653if non-NULL, the client wsi is set to be a child of parent_wsi. This ensures
654if parent_wsi closes, then the client child is closed just before.
655
Andy Green0f9904f2016-03-17 15:26:49 +08006567) If you're using SSL, there's a new context creation-time option flag
657LWS_SERVER_OPTION_REDIRECT_HTTP_TO_HTTPS. If you give this, non-ssl
658connections to the server listen port are accepted and receive a 301
659redirect to / on the same host and port using https://
660
Andy Greenc5376b12016-04-08 09:45:49 +08006618) User code may set per-connection extension options now, using a new api
662"lws_set_extension_option()".
663
664This should be called from the ESTABLISHED callback like this
665
666 lws_set_extension_option(wsi, "permessage-deflate",
667 "rx_buf_size", "12"); /* 1 << 12 */
668
669If the extension is not active (missing or not negotiated for the
670connection, or extensions are disabled on the library) the call is
671just returns -1. Otherwise the connection's extension has its
672named option changed.
673
674The extension may decide to alter or disallow the change, in the
675example above permessage-deflate restricts the size of his rx
676output buffer also considering the protocol's rx_buf_size member.
677
678
Andy Greencd0c6962016-03-28 10:12:37 +0800679New application lwsws
680---------------------
681
682A libwebsockets-based general webserver is built by default now, lwsws.
683
684It's configured by JSON, by default in
685
686 /etc/lwsws/conf
687
688which contains global lws context settings like this
689
690{
691 "global": {
692 "uid": "99",
693 "gid": "99",
694 "interface": "eth0",
695 "count-threads": "1"
696 }
697}
698
699 /etc/lwsws/conf.d/*
700
701which contains zero or more files describing vhosts, like this
702
703{
704 "vhosts": [
705 { "name": "warmcat.com",
706 "port": "443",
707 "host-ssl-key": "/etc/pki/tls/private/warmcat.com.key",
708 "host-ssl-cert": "/etc/pki/tls/certs/warmcat.com.crt",
709 "host-ssl-ca": "/etc/pki/tls/certs/warmcat.com.cer",
710 "mounts": [
711 { "/": [
712 { "home": "file:///var/www/warmcat.com" },
713 { "default": "index.html" }
714 ]
715 }
716 ]
717 }
718 ]
719}
720
721
Andy Green0c3cc2e2016-02-20 09:12:52 +0800722
Andy Green9c60ed92016-02-16 12:32:18 +0800723v1.7.0
724======
725
Andy Green67112662016-01-11 11:34:01 +0800726Extension Changes
727-----------------
728
7291) There is now a "permessage-deflate" / RFC7692 implementation. It's very
730similar to "deflate-frame" we have offered for a long while; deflate-frame is
731now provided as an alias of permessage-deflate.
732
733The main differences are that the new permessage-deflate implementation:
734
735 - properly performs streaming respecting input and output buffer limits. The
736 old deflate-frame implementation could only work on complete deflate input
737 and produce complete inflate output for each frame. The new implementation
738 only mallocs buffers at initialization.
739
740 - goes around the event loop after each input package is processed allowing
741 interleaved output processing. The RX flow control api can be used to
742 force compressed input processing to match the rate of compressed output
743 processing (test--echo shows an example of how to do this).
744
745 - when being "deflate-frame" for compatibility he uses the same default zlib
746 settings as the old "deflate-frame", but instead of exponentially increasing
747 malloc allocations until the whole output will fit, he observes the default
748 input and output chunking buffer sizes of "permessage-deflate", that's
749 1024 in and 1024 out at a time.
750
7512) deflate-stream has been disabled for many versions (for over a year) and is
752now removed. Browsers are now standardizing on "permessage-deflate" / RFC7692
753
7543) struct lws_extension is simplified, and lws extensions now have a public
755api (their callback) for use in user code to compose extensions and options
756the user code wants. lws_get_internal_exts() is deprecated but kept around
757as a NOP. The changes allow one extension implementation to go by different
758names and allows the user client code to control option offers per-ext.
759
760The test client and server are updated to use the new way. If you use
761the old way it should still work, but extensions will be disabled until you
762update your code.
763
764Extensions are now responsible for allocating and per-instance private struct
765at instance construction time and freeing it when the instance is destroyed.
766Not needing to know the size means the extension's struct can be opaque
767to user code.
768
769
Andy Green3df58002015-12-25 12:44:12 +0800770User api additions
771------------------
772
Andy Green200a6a22016-02-15 20:36:02 +08007731) The info struct gained three new members
Andy Green3df58002015-12-25 12:44:12 +0800774
775 - max_http_header_data: 0 for default (1024) or set the maximum amount of known
776 http header payload that lws can deal with. Payload in unknown http
777 headers is dropped silently. If for some reason you need to send huge
778 cookies or other HTTP-level headers, you can now increase this at context-
779 creation time.
780
781 - max_http_header_pool: 0 for default (16) or set the maximum amount of http
782 headers that can be tracked by lws in this context. For the server, if
783 the header pool is completely in use then accepts on the listen socket
784 are disabled until one becomes free. For the client, if you simultaneously
785 have pending connects for more than this number of client connections,
786 additional connects will fail until some of the pending connections timeout
787 or complete.
788
Andy Green200a6a22016-02-15 20:36:02 +0800789 - timeout_secs: 0 for default (currently 20s), or set the library's
790 network activity timeout to the given number of seconds
791
Andy Green3df58002015-12-25 12:44:12 +0800792HTTP header processing in lws only exists until just after the first main
793callback after the HTTP handshake... for ws connections that is ESTABLISHED and
794for HTTP connections the HTTP callback.
795
796So these settings are not related to the maximum number of simultaneous
Andy Green3246ebb2015-12-26 12:03:06 +0800797connections, but the number of HTTP handshakes that may be expected or ongoing,
Andy Green3df58002015-12-25 12:44:12 +0800798or have just completed, at one time. The reason it's useful is it changes the
799memory allocation for header processing to be one-time at context creation
800instead of every time there is a new connection, and gives you control over
801the peak allocation.
802
803Setting max_http_header_pool to 1 is fine it will just queue incoming
804connections before the accept as necessary, you can still have as many
Andy Green5b85e392015-12-26 13:23:11 +0800805simultaneous post-header connections as you like. Since the http header
806processing is completed and the allocation released after ESTABLISHED or the
807HTTP callback, even with a pool of 1 many connections can be handled rapidly.
808
Andy Green066a7a12015-12-26 15:47:06 +08008092) There is a new callback that allows the user code to get acccess to the
810optional close code + aux data that may have been sent by the peer.
811
812LWS_CALLBACK_WS_PEER_INITIATED_CLOSE:
813 The peer has sent an unsolicited Close WS packet. @in and
814 @len are the optional close code (first 2 bytes, network
815 order) and the optional additional information which is not
816 defined in the standard, and may be a string or non-human-
817 readble data.
818 If you return 0 lws will echo the close and then close the
Andy Green67112662016-01-11 11:34:01 +0800819 connection. If you return nonzero lws will just close the
820 connection.
Andy Green066a7a12015-12-26 15:47:06 +0800821
822As usual not handling it does the right thing, if you're not interested in it
823just ignore it.
824
825The test server has "open and close" testing buttons at the bottom, if you
826open and close that connection, on close it will send a close code 3000 decimal
827and the string "Bye!" as the aux data.
828
829The test server dumb-increment callback handles this callback reason and prints
830
831lwsts[15714]: LWS_CALLBACK_WS_PEER_INITIATED_CLOSE: len 6
832lwsts[15714]: 0: 0x0B
833lwsts[15714]: 1: 0xB8
834lwsts[15714]: 2: 0x42
835lwsts[15714]: 3: 0x79
836lwsts[15714]: 4: 0x65
837lwsts[15714]: 5: 0x21
838
Andy Green1fb95e82015-12-26 17:20:34 +08008393) There is a new API to allow the user code to control the content of the
840close frame sent when about to return nonzero from the user callback to
841indicate the connection should close.
842
843/**
844 * lws_close_reason - Set reason and aux data to send with Close packet
845 * If you are going to return nonzero from the callback
846 * requesting the connection to close, you can optionally
847 * call this to set the reason the peer will be told if
848 * possible.
849 *
850 * @wsi: The websocket connection to set the close reason on
851 * @status: A valid close status from websocket standard
852 * @buf: NULL or buffer containing up to 124 bytes of auxiliary data
853 * @len: Length of data in @buf to send
854 */
855LWS_VISIBLE LWS_EXTERN void
856lws_close_reason(struct lws *wsi, enum lws_close_status status,
857 unsigned char *buf, size_t len);
858
859An extra button is added to the "open and close" test server page that requests
860that the test server close the connection from his end.
861
862The test server code will do so by
863
864 lws_close_reason(wsi, LWS_CLOSE_STATUS_GOINGAWAY,
865 (unsigned char *)"seeya", 5);
866 return -1;
867
868The browser shows the close code and reason he received
869
870websocket connection CLOSED, code: 1001, reason: seeya
871
Vijay Khurdiyae73d4462016-01-15 16:21:51 +08008724) There's a new context creation time option flag
Andy Green44a7f652015-12-29 11:20:09 +0800873
874LWS_SERVER_OPTION_VALIDATE_UTF8
875
876if you set it in info->options, then TEXT and CLOSE frames will get checked to
877confirm that they contain valid UTF-8. If they don't, the connection will get
878closed by lws.
879
Vijay Khurdiyae73d4462016-01-15 16:21:51 +08008805) ECDH Certs are now supported. Enable the CMake option
881
882cmake .. -DLWS_SSL_SERVER_WITH_ECDH_CERT=1
883
884**and** the info->options flag
885
Andy Greenff696482016-02-17 07:46:27 +0800886LWS_SERVER_OPTION_SSL_ECDH
Vijay Khurdiyae73d4462016-01-15 16:21:51 +0800887
888to build in support and select it at runtime.
889
Andy Greenff696482016-02-17 07:46:27 +08008906) There's a new api lws_parse_uri() that simplifies chopping up
Andy Green809d69a2016-01-14 11:37:56 +0800891https://xxx:yyy/zzz uris into parts nicely. The test client now uses this
Andy Greend3a55052016-01-19 03:34:24 +0800892to allow proper uris as well as the old address style.
893
8947) SMP support is integrated into LWS without any internal threading. It's
895very simple to use, libwebsockets-test-server-pthread shows how to do it,
896use -j <n> argument there to control the number of service threads up to 32.
897
898Two new members are added to the info struct
899
900 unsigned int count_threads;
901 unsigned int fd_limit_per_thread;
902
903leave them at the default 0 to get the normal singlethreaded service loop.
904
905Set count_threads to n to tell lws you will have n simultaneous service threads
906operating on the context.
907
908There is still a single listen socket on one port, no matter how many
909service threads.
910
911When a connection is made, it is accepted by the service thread with the least
912connections active to perform load balancing.
913
914The user code is responsible for spawning n threads running the service loop
915associated to a specific tsi (Thread Service Index, 0 .. n - 1). See
916the libwebsockets-test-server-pthread for how to do.
917
918If you leave fd_limit_per_thread at 0, then the process limit of fds is shared
919between the service threads; if you process was allowed 1024 fds overall then
920each thread is limited to 1024 / n.
921
922You can set fd_limit_per_thread to a nonzero number to control this manually, eg
923the overall supported fd limit is less than the process allowance.
924
925You can control the context basic data allocation for multithreading from Cmake
926using -DLWS_MAX_SMP=, if not given it's set to 32. The serv_buf allocation
927for the threads (currently 4096) is made at runtime only for active threads.
928
929Because lws will limit the requested number of actual threads supported
930according to LWS_MAX_SMP, there is an api lws_get_count_threads(context) to
931discover how many threads were actually allowed when the context was created.
932
933It's required to implement locking in the user code in the same way that
934libwebsockets-test-server-pthread does it, for the FD locking callbacks.
935
Andy Greenba119e92016-01-26 21:40:32 +0800936If LWS_MAX_SMP=1, then there is no code related to pthreads compiled in the
937library. If more than 1, a small amount of pthread mutex code is built into
938the library.
Andy Green809d69a2016-01-14 11:37:56 +0800939
Andy Greenba119e92016-01-26 21:40:32 +08009408) New API
941
942LWS_VISIBLE struct lws *
943lws_adopt_socket(struct lws_context *context, lws_sockfd_type accept_fd)
944
945allows foreign sockets accepted by non-lws code to be adopted by lws as if they
946had just been accepted by lws' own listen socket.
Andy Green3df58002015-12-25 12:44:12 +0800947
Danomi Czaski4e9c7f32016-01-28 09:40:53 +08009489) X-Real-IP: header has been added as WSI_TOKEN_HTTP_X_REAL_IP
949
Andy Green86ed65f2016-02-14 09:27:41 +080095010) Libuv support is added, there are new related user apis
951
952typedef void (lws_uv_signal_cb_t)(uv_loop_t *l, uv_signal_t *w, int revents);
953
954LWS_VISIBLE LWS_EXTERN int
955lws_uv_sigint_cfg(struct lws_context *context, int use_uv_sigint,
956 lws_uv_signal_cb_t *cb);
957
958LWS_VISIBLE LWS_EXTERN int
959lws_uv_initloop(struct lws_context *context, uv_loop_t *loop, int tsi);
960
961LWS_VISIBLE void
962lws_uv_sigint_cb(uv_loop_t *loop, uv_signal_t *watcher, int revents);
963
964and CMAKE option
965
966LWS_WITH_LIBUV
967
Danomi Czaski4e9c7f32016-01-28 09:40:53 +0800968
Andy Green3246ebb2015-12-26 12:03:06 +0800969User api changes
970----------------
971
9721) LWS_SEND_BUFFER_POST_PADDING is now 0 and deprecated. You can remove it; if
973you still use it, obviously it does nothing. Old binary code with nonzero
974LWS_SEND_BUFFER_POST_PADDING is perfectly compatible, the old code just
Andy Green5b85e392015-12-26 13:23:11 +0800975allocated a buffer bigger than the library is going to use.
Andy Green3246ebb2015-12-26 12:03:06 +0800976
Andy Green5b85e392015-12-26 13:23:11 +0800977The example apps no longer use LWS_SEND_BUFFER_POST_PADDING.
Andy Green3246ebb2015-12-26 12:03:06 +0800978
Andy Green4e2ac762015-12-26 20:26:11 +0800979The only path who made use of it was sending with LWS_WRITE_CLOSE --->
Andy Green3246ebb2015-12-26 12:03:06 +0800980
Andy Green1fb95e82015-12-26 17:20:34 +08009812) Because of lws_close_reason() formalizing handling close frames,
982LWS_WRITE_CLOSE is removed from libwebsockets.h. It was only of use to send
983close frames...close frame content should be managed using lws_close_reason()
984now.
985
Andy Green44a7f652015-12-29 11:20:09 +08009863) We check for invalid CLOSE codes and complain about protocol violation in
987our close code. But it changes little since we were in the middle of closing
988anyway.
989
9904) zero-length RX frames and zero length TX frames are now allowed.
991
9925) Pings and close used to be limited to 124 bytes, the correct limit is 125
993so that is now also allowed.
Andy Green3246ebb2015-12-26 12:03:06 +0800994
Steffen Vogelf9267172016-02-09 07:19:15 +01009956) LWS_PRE is provided as a synonym for LWS_SEND_BUFFER_PRE_PADDING, either is
Andy Green67112662016-01-11 11:34:01 +0800996valid to use now.
997
9987) There's generic support for RFC7462 style extension options built into the
999library now. As a consequence, a field "options" is added to lws_extension.
1000It can be NULL if there are no options on the extension. Extension internal
1001info is part of the public abi because extensions may be implemented outside
1002the library.
1003
Danomi Czaski4e9c7f32016-01-28 09:40:53 +080010048) WSI_TOKEN_PROXY enum was accidentally defined to collide with another token
1005of value 73. That's now corrected and WSI_TOKEN_PROXY moved to his own place at
100677.
1007
Andy Green86ed65f2016-02-14 09:27:41 +080010089) With the addition of libuv support, libev is not the only event loop
1009library in town and his api names must be elaborated with _ev_
1010
1011 Callback typedef: lws_signal_cb ---> lws_ev_signal_cb_t
1012 lws_sigint_cfg --> lws_ev_sigint_cfg
1013 lws_initloop --> lws_ev_initloop
1014 lws_sigint_cb --> lws_ev_sigint_cb
1015
101610) Libev support is made compatible with multithreaded service,
1017lws_ev_initloop (was lws_initloop) gets an extra argument for the
1018thread service index (use 0 if you will just have 1 service thread).
1019
1020LWS_VISIBLE LWS_EXTERN int
Andy Greenff696482016-02-17 07:46:27 +08001021lws_ev_initloop(struct lws_context *context, ev_loop_t *loop, int tsi);
Andy Green86ed65f2016-02-14 09:27:41 +08001022
Andy Green3df58002015-12-25 12:44:12 +08001023
Andy Greenaef3dc42016-05-06 07:45:19 +08001024(for earlier changelogs, see the tagged releases)