Tianjie Xu | a5dcb7c | 2018-09-25 12:25:15 -0700 | [diff] [blame] | 1 | [DRAFT] |
| 2 | |
| 3 | Introduction: |
| 4 | |
| 5 | This document is a draft proposal for Shard Brotli dictionaries in the fetch spec |
| 6 | (https://fetch.spec.whatwg.org/). |
| 7 | |
| 8 | The goal is to add support for custom dictionaries for Brotli. A dictionary is used |
| 9 | to improve compression. A client can download a dictionary from a server and then |
| 10 | use it to decompress resources compressed with this dictionary. |
| 11 | |
| 12 | This document specifies how the client and server negotiate the dictionary over HTTP. |
| 13 | A high level overview is as follows: The server adds an HTTP header to the response |
| 14 | with a URL of the dictionary. The browser downloads the dictionary from the URL and |
| 15 | then caches it so it can be reused. The server also adds a checksum to an HTTP header |
| 16 | which the client uses to verify the dictionary. Caching, CORS, and other existing |
| 17 | mechanisms are used. A dictionary can be a pre-made static dictionary, but does not |
| 18 | have to be, for example a previous page loaded from this server, or an old version |
| 19 | of a page, can be used as well. |
| 20 | |
| 21 | Below are changes and additions to add Shared Brotli dictionaries to the fetch spec |
| 22 | at https://fetch.spec.whatwg.org/: |
| 23 | |
| 24 | Additions to `4.5. HTTP-network-or-cache fetch` |
| 25 | |
| 26 | Add to point `15. Modify httpRequest’s header list per HTTP.`: |
| 27 | |
| 28 | If the recursive-sbr flag is enabled, `Accept-Encoding` may not contain `sbr` |
| 29 | [NOTE-BOX] When sbr can be used, it is possible to add a header Available-Dict |
| 30 | with the URL and hash code of a cached resource. The server may then use it as |
| 31 | shared dictionary. |
| 32 | |
| 33 | Additions to `4.6. HTTP-network fetch` |
| 34 | |
| 35 | Add after point `10. Run these steps, but abort if the ongoing fetch is terminated`: |
| 36 | |
| 37 | 11. Let codings be the result of extracting header list values given |
| 38 | `Content-Encoding` and response’s header list. |
| 39 | 12. If codings contains `sbr` |
| 40 | 1. If the header list does not contain `Sbr-Dict`, return a network error |
| 41 | 2. Let dictionaryId be the result of extracting header list values given |
| 42 | `Sbr-Dict` and response’s header list. |
| 43 | |
| 44 | To point `12. Run these substeps in parallel:`, add new first sub-point: |
| 45 | |
| 46 | 1. If codings contains `sbr`, run these subsubsteps: |
| 47 | 1. Let dictionaryResponse be the result of performing a |
| 48 | Shared-Brotli-dictionary fetch given dictionaryId and request. |
| 49 | 2. If dictionaryResponse is a network error, return a network error. |
| 50 | |
| 51 | Change point `12.4. Set bytes to the result of handling content codings given codings and bytes.` to: |
| 52 | |
| 53 | 4. Set bytes to the result of handling content codings given codings, bytes |
| 54 | and, if codings contains `sbr`, also dictionaryResponse's body. |
| 55 | [NOTE-BOX] If the dictionary is still being fetched, which happens in |
| 56 | parallel, enqueue bytes in a compressed buffer and handle content coding |
| 57 | once the dictionary is fetched |
| 58 | |
| 59 | Additions to `2.2.4. Bodies` |
| 60 | |
| 61 | |
| 62 | Change last section `To handle content codings ...` to: |
| 63 | |
| 64 | To handle content codings given codings, bytes and optionally a dictionary, run these substeps: |
| 65 | 1. If codings are not supported, return bytes. |
| 66 | 2. If the codings has `sbr`, run these subsubsteps: |
| 67 | a. Return the result of decoding bytes and dictionary with the Shared |
| 68 | Brotli decoder. |
| 69 | [Shared Brotli Spec] [IANA Brotli](https://www.iana.org/assignments/http-parameters/http-parameters.xhtml) |
| 70 | 3. Else: |
| 71 | a. Return the result of decoding bytes with the given codings, as |
| 72 | explained in HTTP. [HTTP] [HTTP-SEMANTICS] [HTTP-COND] [HTTP-CACHING] |
| 73 | [HTTP-AUTH] |
| 74 | |
| 75 | New section `4.10. Shared-Brotli-dictionary fetch` |
| 76 | |
| 77 | To perform a Shared-Brotli-dictionary fetch using dictionaryId, and parentRequest, perform these steps: |
| 78 | |
| 79 | 1. Let dictionaryURL be the URL extracted from dictionaryId |
| 80 | 2. Let dictionaryHash be the hash id extracted from dictionaryId |
| 81 | 3. Let dictionaryRequest be a new request whose method is `GET`, url is |
| 82 | dictionaryURL, mode is "cors", and client is parentRequest's client. |
| 83 | 4. Let dictionaryResponse be the result of performing an |
| 84 | [HTTP-network-or-cache](https://fetch.spec.whatwg.org/#concept-http-network-or-cache-fetch) |
| 85 | fetch using dictionaryRequest with the recursive-sbr flag set to true. |
| 86 | [NOTE-BOX] For compression benefits, the dictionary should be reused to |
| 87 | decode multiple different responses. We rely on caching to achieve this. |
| 88 | It is suggested for servers to not add any "no-cache" or short "max-age" |
| 89 | Cache-Control directives, and it is suggested for the client to effectively |
| 90 | support caching it. |
| 91 | [NOTE-BOX] Since the same dictionary can be identified by a hash code, a |
| 92 | browser can avoid fetching a dictionary if it already has one with the same |
| 93 | hashed cached from a different source URL. |
| 94 | [NOTE-BOX] It is suggested that a server does not reuse the same URL |
| 95 | to host an updated or different dictionary. Instead the same dictionary URL |
| 96 | should contain a dictionary with the same content and same hash. |
| 97 | 5. If dictionaryResponse is a network error, return a network error. |
| 98 | 6. If dictionaryResponse's status is not an ok status, return a network error. |
| 99 | 7. Let tokens be the result of |
| 100 | [parsing metadata](https://w3c.github.io/webappsec-subresource-integrity/#parse-metadata) |
| 101 | given dictionaryHash. |
| 102 | [Subresource Integrity](https://w3c.github.io/webappsec-subresource-integrity/) |
| 103 | 8. If tokens is no metadata or the length of tokens is not 1, return a network |
| 104 | error |
| 105 | 9. Let algorithm be the alg component of tokens[0]. If alg is 'hw3', set |
| 106 | algorithm to 256-bit HighwayHash |
| 107 | 10. Let digest be the val component of tokens[1]. |
| 108 | 11. Let hashValue be the result of base64 decoding digest |
| 109 | [base64](https://tools.ietf.org/html/rfc4648) |
| 110 | 12. If hashValue is not a valid base64 encoding, return a network error |
| 111 | [NOTE-BOX] All of the supported hashing algorithms are cryptographically |
| 112 | secure. |
| 113 | 13. Compute the hash code of dictionaryResponse's body using algorithm and |
| 114 | compare this checksum for equality with hashValue. If the computed |
| 115 | checksum does not match hashValue, return a network error. |
| 116 | 14. Return dictionaryResponse. |