tree 3794868a5a226ecd88705c6df2f5c5a911cc23be
parent a56ab156e5887bebba51ba6de3335a670b69c2dd
author Wyatt Hepler <hepler@google.com> 1597347120 -0700
committer CQ Bot Account <commit-bot@chromium.org> 1597795630 +0000

pw_tokenizer: domain and database creation support

Support new argumens to the pw_tokenizer_database template.

- create: Create a new CSV or binary database instead of updating an
  existing file. This could be used, for example, to generate a binary
  version of an in-tree CSV database.
- domain: Extract tokens from the specified tokenization domain.

Change-Id: I0c91176f99ec07bc9f690d2a6c542eca606c88a3
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/15962
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Keir Mierle <keir@google.com>