Wyatt Hepler | f9fb90f | 2020-09-30 18:59:33 -0700 | [diff] [blame] | 1 | .. _module-pw_blob_store: |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 2 | |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 3 | ============= |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 4 | pw_blob_store |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 5 | ============= |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 6 | ``pw_blob_store`` is a storage container library for storing a single blob of |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 7 | data. ``BlobStore`` is a flash-backed persistent storage system with integrated |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 8 | data integrity checking that serves as a lightweight alternative to a file |
| 9 | system. |
| 10 | |
Ted Pudlik | 1b69a4e | 2021-11-13 00:30:12 +0000 | [diff] [blame] | 11 | ----- |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 12 | Usage |
Ted Pudlik | 1b69a4e | 2021-11-13 00:30:12 +0000 | [diff] [blame] | 13 | ----- |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 14 | Most operations on a ``BlobStore`` are done using ``BlobReader`` and |
| 15 | ``BlobWriter`` objects that have been constructed using a ``BlobStore``. Though |
| 16 | a ``BlobStore`` may have multiple open ``BlobReader`` objects, no other |
| 17 | readers/writers may be active if a ``BlobWriter`` is opened on a blob store. |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 18 | |
David Rogers | 2835cf6 | 2022-03-10 20:23:20 -0800 | [diff] [blame] | 19 | The data state of a blob can be checked using the ``HasData()`` method. |
| 20 | The method returns true if the blob is currenty valid and has at least one data |
| 21 | byte. This allows checking if a blob has stored data without needing to |
| 22 | instantiate and open a reader or writer. |
| 23 | |
David Rogers | 9657141 | 2021-11-05 03:30:35 -0700 | [diff] [blame] | 24 | Write buffer |
| 25 | ============ |
| 26 | |
| 27 | BlobStore uses a write buffer to allow writes smaller than and/or unaligned to |
| 28 | the flash write aligment. BlobStore also supports using the write buffer for |
| 29 | deferred writes that can be enqueued and written to flash at a later time or by |
| 30 | a different thread/context. |
| 31 | |
| 32 | BlobStore can be used with a zero-size write buffer to reduce memory |
| 33 | requirements. When using zero-size write buffer, the user is required to write |
| 34 | maintain write sizes that are a multiple of the flash write size the blob is |
| 35 | configured for. |
| 36 | |
| 37 | If a non-zero sized write buffer is used, the write buffer size must be a |
| 38 | multiple of the flash write size. |
| 39 | |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 40 | Writing to a BlobStore |
| 41 | ---------------------- |
| 42 | ``BlobWriter`` objects are ``pw::stream::Writer`` compatible, but do not support |
| 43 | reading any of the blob's contents. Opening a ``BlobWriter`` on a ``BlobStore`` |
| 44 | that contains data will discard any existing data if ``Discard()``, ``Write |
| 45 | ()``, or ``Erase()`` are called. There is currently no mechanism to allow |
| 46 | appending to existing data. |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 47 | |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 48 | .. code-block:: cpp |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 49 | |
Armando Montanez | 0e8aff8 | 2021-09-30 13:21:12 -0700 | [diff] [blame] | 50 | BlobStore::BlobWriterWithBuffer writer(my_blob_store); |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 51 | writer.Open(); |
| 52 | writer.Write(my_data); |
| 53 | |
| 54 | // ... |
| 55 | |
| 56 | // A close is implied when a BlobWriter is destroyed. Manually closing a |
| 57 | // BlobWriter enables error handling on Close() failure. |
| 58 | writer.Close(); |
| 59 | |
| 60 | Erasing a BlobStore |
| 61 | =================== |
| 62 | There are two distinctly different mechanisms to "erase" the contents of a BlobStore: |
| 63 | |
| 64 | #. ``Discard()``: Discards any ongoing writes and ensures ``BlobReader`` objects |
| 65 | see the ``BlobStore`` as empty. This is the fastest way to logically erase a |
| 66 | ``BlobStore``. |
| 67 | #. ``Erase()``: Performs an explicit flash erase of the ``BlobStore``'s |
| 68 | underlying partition. This is useful for manually controlling when a flash |
| 69 | erase is performed before a ``BlobWriter`` starts to write data (as flash |
| 70 | erase operations may be time-consuming). |
| 71 | |
| 72 | Naming a BlobStore's contents |
| 73 | ============================= |
| 74 | Data in a ``BlobStore`` May be named similarly to a file. This enables |
| 75 | identification of a BlobStore's contents in cases where different data may be |
| 76 | stored to a shared blob store. This requires an additional RAM buffer that can |
| 77 | be used to encode the BlobStore's KVS metadata entry. Calling |
| 78 | ``MaxFileNameLength()`` on a ``BlobWriter`` will provide the max file name |
| 79 | length based on the ``BlobWriter``'s metadata encode buffer size. |
| 80 | |
| 81 | ``SetFileName()`` performs a copy of the provided file name, meaning it's safe |
| 82 | for the ``std::string_view`` to be invalidated after the function returns. |
| 83 | |
| 84 | .. code-block:: cpp |
| 85 | |
Armando Montanez | 0e8aff8 | 2021-09-30 13:21:12 -0700 | [diff] [blame] | 86 | constexpr size_t kMaxFileNameLength = 48; |
| 87 | BlobStore::BlobWriterWithBuffer<kMaxFileNameLength> writer(my_blob_store); |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 88 | writer.Open(); |
| 89 | writer.SetFileName("stonks.jpg"); |
| 90 | writer.Write(my_data); |
| 91 | // ... |
| 92 | writer.Close(); |
| 93 | |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 94 | Reading from a BlobStore |
| 95 | ------------------------ |
David Rogers | 2835cf6 | 2022-03-10 20:23:20 -0800 | [diff] [blame] | 96 | A ``BlobStore`` may have multiple open ``BlobReader`` objects. No other |
| 97 | readers/writers may be open/active if a ``BlobWriter`` is opened on a blob |
| 98 | store. |
Armando Montanez | 28ad6f4 | 2021-08-30 16:23:57 -0700 | [diff] [blame] | 99 | |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 100 | 0) Create BlobReader instance |
| 101 | 1) BlobReader::Open(). |
| 102 | 2) Read data using BlobReader::Read() or |
Yecheng Zhao | 4a6deb4 | 2021-09-01 15:23:00 -0700 | [diff] [blame] | 103 | BlobReader::GetMemoryMappedBlob(). BlobReader is seekable. Use |
| 104 | BlobReader::Seek() to read from a desired offset. |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 105 | 3) BlobReader::Close(). |
| 106 | |
Ted Pudlik | 1b69a4e | 2021-11-13 00:30:12 +0000 | [diff] [blame] | 107 | -------------------------- |
Armando Montanez | 0aa452b | 2021-09-29 17:21:33 -0700 | [diff] [blame] | 108 | FileSystem RPC integration |
Ted Pudlik | 1b69a4e | 2021-11-13 00:30:12 +0000 | [diff] [blame] | 109 | -------------------------- |
Armando Montanez | 0aa452b | 2021-09-29 17:21:33 -0700 | [diff] [blame] | 110 | ``pw_blob_store`` provides an optional ``FileSystemEntry`` implementation for |
| 111 | use with ``pw_file``'s ``FlatFileSystemService``. This simplifies the process of |
| 112 | enumerating ``BlobStore`` objects as files via ``pw_file``'s ``FileSystem`` RPC |
| 113 | service. |
| 114 | |
Ted Pudlik | 1b69a4e | 2021-11-13 00:30:12 +0000 | [diff] [blame] | 115 | ----------- |
David Rogers | 17793d6 | 2021-02-05 03:29:02 -0800 | [diff] [blame] | 116 | Size report |
Ted Pudlik | 1b69a4e | 2021-11-13 00:30:12 +0000 | [diff] [blame] | 117 | ----------- |
David Rogers | 17793d6 | 2021-02-05 03:29:02 -0800 | [diff] [blame] | 118 | The following size report showcases the memory usage of the blob store. |
| 119 | |
| 120 | .. include:: blob_size |
| 121 | |
| 122 | |
David Rogers | 2d19502 | 2020-07-16 14:07:47 -0700 | [diff] [blame] | 123 | .. note:: |
| 124 | The documentation for this module is currently incomplete. |