| <html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>The C Code Generator</title><meta name="generator" content="DocBook XSL Stylesheets V1.71.0"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="article" lang="en"><div class="titlepage"><div><div><h2 class="title"><a name="id2425612"></a>The C Code Generator</h2></div></div><hr></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="#id2478318">Design</a></span></dt><dt><span class="section"><a href="#id2517105">The Generated Code</a></span></dt><dt><span class="section"><a href="#id2478728">The protobuf-c Library</a></span></dt><dt><span class="section"><a href="#id2478765">protobuf-c: the Descriptor structures</a></span></dt><dt><span class="section"><a href="#id2478802">protobuf-c: helper structures and typedefs</a></span></dt><dd><dl><dt><span class="section"><a href="#buffers">Buffers</a></span></dt></dl></dd><dt><span class="section"><a href="#id2479168">protobuf-c: packing and unpacking messages</a></span></dt><dt><span class="section"><a href="#id2479561">Author</a></span></dt></dl></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2478318"></a>Design</h2></div></div></div><p>The overall goal is to keep the code-generator as simple |
| as possible. Hopefully performance isn't sacrificed to that end!</p><p>Anyways, we generate very little code: we mostly generate |
| structure definitions (for example enums and structures |
| for messages) and some metadata which is basically |
| reflection-type data.</p><p>The serializing and deserializing is implemented in a library, |
| called libprotobuf-c rather than generated code.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2517105"></a>The Generated Code</h2></div></div></div><p> |
| For each enum, we generate a C enum. |
| For each message, we generate a C structure |
| which can be cast to a <span class="type">ProtobufCMessage</span>. |
| </p><p> |
| For each enum and message, we generate a descriptor |
| object that allows us to implement a kind of reflection |
| on the structures. |
| </p><p>First, some naming conventions: |
| </p><div class="itemizedlist"><ul type="disc"><li><p> |
| The name of the type for enums and messages and services |
| is camel case (meaning WordsAreCrammedTogether) |
| except that double-underscores are used to delimit |
| scopes. For example: |
| </p><pre class="programlisting"> |
| package foo.bar; |
| message BazBah { |
| int32 val; |
| } |
| </pre><p> |
| would generate a C type <span class="type">Foo__Bar__BazBah</span>.</p></li><li><p>Functions and globals are all lowercase, with camel-case |
| words separated by single underscores. |
| For example: |
| </p><pre class="programlisting"> |
| Foo__Bar__BazBah *foo__bar__baz_bah__unpack |
| (ProtobufCAllocator *allocator, |
| size_t length, |
| const unsigned char *data); |
| </pre><p> |
| </p></li><li><p>Enums values are all uppercase.</p></li><li><p> |
| Stuff we dd to your symbol names will also be |
| separated by a double-underscore. For example, |
| the unpack method above.</p></li></ul></div><p> |
| </p><p> |
| We also generate descriptor objects for messages |
| and enums. These are declared in the .h files: |
| </p><pre class="programlisting"> |
| extern const ProtobufCMessageDescriptor |
| foo__bar__baz_bah__descriptor; |
| </pre><p> |
| </p><p> |
| The message structures all begin with <span class="type">ProtobufCMessageDescriptor*</span> |
| which is sufficient to allow them to be cast to <span class="type">ProtobufCMessage</span>. |
| </p><p> |
| We generate some functions for each message: |
| </p><div class="itemizedlist"><ul type="disc"><li><p><code class="function">unpack()</code>. Unpack data for a particular |
| message-format: |
| </p><pre class="programlisting"> |
| Foo__Bar__BazBah * |
| foo__bar__baz_bah__unpack (ProtobufCAllocator *allocator, |
| size_t length, |
| const unsigned char *data); |
| </pre><p> |
| Note that <em class="parameter"><code>allocator</code></em> may be NULL. |
| </p></li><li><p><code class="function">free_unpacked()</code>. Free a message |
| that you obtained with the unpack method: |
| </p><pre class="programlisting"> |
| void |
| foo__bar__baz_bah__free_unpacked (Foo__Bar__BazBah *baz_bah, |
| ProtobufCAllocator *allocator); |
| </pre><p> |
| </p></li><li><p><code class="function">get_packed_size()</code>. Find how long |
| the serialized representation of the data will be: |
| message-format: |
| </p><pre class="programlisting"> |
| size_t |
| foo__bar__baz_bah__get_packed_size |
| (const Foo__Bar__BazBah *message); |
| </pre><p> |
| </p></li><li><p><code class="function">pack()</code>. Pack message |
| into buffer; assumes that buffer is long enough (use get_packed_size first!). |
| </p><pre class="programlisting"> |
| size_t |
| foo__bar__baz_bah__pack |
| (const Foo__Bar__BazBah *message, |
| unsigned char *packed_data_out); |
| </pre><p> |
| </p></li><li><p><code class="function">pack_to_buffer()</code>. Pack message |
| into virtualize buffer. |
| </p><pre class="programlisting"> |
| size_t |
| foo__bar__baz_bah__pack_to_buffer |
| (const Foo__Bar__BazBah *message, |
| ProtobufCBuffer *buffer); |
| </pre><p> |
| </p></li></ul></div><p> |
| </p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2478728"></a>The protobuf-c Library</h2></div></div></div><p>This library is used by the generated code; |
| it includes common structures and enums, |
| as well as functions that most users of the generated code |
| will want.</p><p> |
| There are three main components: |
| </p><div class="orderedlist"><ol type="1"><li><p>the Descriptor structures</p></li><li><p>helper structures and objects</p></li><li><p>packing and unpacking code</p></li></ol></div><p> |
| </p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2478765"></a>protobuf-c: the Descriptor structures</h2></div></div></div><p>For example, enums are described in terms of structures: |
| |
| </p><pre class="programlisting"> |
| struct _ProtobufCEnumValue |
| { |
| const char *name; |
| const char *c_name; |
| int value; |
| }; |
| |
| struct _ProtobufCEnumDescriptor |
| { |
| const char *name; |
| const char *short_name; |
| const char *package_name; |
| |
| /* sorted by value */ |
| unsigned n_values; |
| const ProtobufCEnumValue *values; |
| |
| /* sorted by name */ |
| unsigned n_value_names; |
| const ProtobufCEnumValue *values_by_name; |
| }; |
| </pre><p>Likewise, messages are described by: |
| |
| </p><pre class="programlisting"> |
| struct _ProtobufCFieldDescriptor |
| { |
| const char *name; |
| int id; |
| ProtobufCFieldLabel label; |
| ProtobufCFieldType type; |
| unsigned quantifier_offset; |
| unsigned offset; |
| void *descriptor; /* for MESSAGE and ENUM types */ |
| }; |
| struct _ProtobufCMessageDescriptor |
| { |
| const char *name; |
| const char *short_name; |
| const char *package_name; |
| |
| /* sorted by field-id */ |
| unsigned n_fields; |
| const ProtobufCFieldDescriptor *fields; |
| }; |
| </pre><p> |
| And finally services are described by: |
| |
| </p><pre class="programlisting"> |
| struct _ProtobufCMethodDescriptor |
| { |
| const char *name; |
| const ProtobufCMessageDescriptor *input; |
| const ProtobufCMessageDescriptor *output; |
| }; |
| struct _ProtobufCServiceDescriptor |
| { |
| const char *name; |
| unsigned n_methods; |
| ProtobufCMethodDescriptor *methods; // sorted by name |
| }; |
| </pre></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2478802"></a>protobuf-c: helper structures and typedefs</h2></div></div></div><p>We defined typedefs for a few types |
| which are used in .proto files but do not |
| have obvious standard C equivalents: |
| </p><div class="itemizedlist"><ul type="disc"><li><p>a boolean type (<span class="type">protobuf_c_boolean</span>)</p></li><li><p>a binary-data (bytes) type (<span class="type">ProtobufCBinaryData</span>)</p></li><li><p>the various int types (<span class="type">int32_t</span>, <span class="type">uint32_t</span>, <span class="type">int64_t</span>, <span class="type">uint64_t</span>) |
| are obtained by including <code class="filename">inttypes.h</code></p></li></ul></div><p> |
| </p><p>We also define a simple allocator object, ProtobufCAllocator |
| that let's you control how allocations are done. |
| This is predominately used for parsing.</p><p>There is a virtual buffer facility that |
| only has to implement a method to append binary-data |
| to the buffer. This can be used to serialize messages |
| to different targets (instead of a flat slab of data).</p><p>We define a base-type for all messages, |
| for code that handles messages generically. |
| All it has is the descriptor object.</p><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="buffers"></a>Buffers</h3></div></div></div><p>One important helper type is the <span class="type">ProtobufCBuffer</span> |
| which allows you to abstract the target of serialization. The only |
| thing that a buffer has is an <code class="function">append</code> method: |
| </p><pre class="programlisting"> |
| struct _ProtobufCBuffer |
| { |
| void (*append)(ProtobufCBuffer *buffer, |
| size_t len, |
| const unsigned char *data); |
| } |
| </pre><p> |
| ProtobufCBuffer subclasses are often defined on the stack. |
| </p><p> |
| For example, to write to a <span class="type">FILE</span> you could make: |
| </p><pre class="programlisting"> |
| typedef struct |
| { |
| ProtobufCBuffer base; |
| FILE *fp; |
| } BufferAppendToFile |
| static void my_buffer_file_append (ProtobufCBuffer *buffer, |
| unsigned len, |
| const unsigned char *data) |
| { |
| BufferAppendToFile *file_buf = (BufferAppendToFile *) buffer; |
| fwrite (data, len, 1, file_buf->fp); // XXX: no error handling! |
| } |
| </pre><p> |
| </p><p> |
| To use this new type of Buffer, you would do something like: |
| </p><pre class="programlisting"> |
| ... |
| BufferAppendToFile tmp; |
| tmp.base.append = my_buffer_file_append; |
| tmp.fp = fp; |
| protobuf_c_message_pack_to_buffer (&message, &tmp); |
| ... |
| </pre><p> |
| </p><p> |
| A commonly builtin subtype is the BufferSimple |
| which is declared on the stack and uses a scratch buffer provided by the user |
| for its initial allocation. It does exponential resizing. |
| To create a BufferSimple, use code like: |
| </p><pre class="programlisting"> |
| unsigned char pad[128]; |
| ProtobufCBufferSimple buf = PROTOBUF_C_BUFFER_SIMPLE_INIT (pad); |
| ProtobufCBuffer *buffer = (ProtobufCBuffer *) &simple; |
| protobuf_c_buffer_append (buffer, 6, (unsigned char *) "hi mom"); |
| </pre><p> |
| You can access the data as buf.len and buf.data. For example, |
| </p><pre class="programlisting"> |
| assert (buf.len == 6); |
| assert (memcmp (buf.data, "hi mom", 6) == 0); |
| </pre><p> |
| To finish up, use: |
| </p><pre class="programlisting"> |
| PROTOBUF_C_BUFFER_SIMPLE_CLEAR (&buf); |
| </pre><p> |
| </p></div></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2479168"></a>protobuf-c: packing and unpacking messages</h2></div></div></div><p> |
| To pack messages one first computes their packed size, |
| then provide a buffer to pack into. |
| </p><pre class="programlisting"> |
| size_t protobuf_c_message_get_packed_size |
| (ProtobufCMessage *message); |
| void protobuf_c_message_pack (ProtobufCMessage *message, |
| unsigned char *out); |
| </pre><p> |
| </p><p> |
| Or you can use the "streaming" approach: |
| </p><pre class="programlisting"> |
| void protobuf_c_message_pack_to_buffer |
| (ProtobufCMessage *message, |
| ProtobufCBuffer *buffer); |
| </pre><p> |
| where <span class="type">ProtobufCBuffer</span> is a base object with an append metod. |
| See <a href="#buffers" title="Buffers">the section called “Buffers”</a>. |
| </p><p> |
| To unpack messages, you should simple call |
| </p><pre class="programlisting"> |
| ProtobufCMessage * |
| protobuf_c_message_unpack (const ProtobufCMessageDescriptor *, |
| ProtobufCAllocator *allocator, |
| size_t len, |
| const unsigned char *data); |
| </pre><p> |
| If you pass NULL for <em class="parameter"><code>allocator</code></em>, then |
| the default allocator will be used. |
| </p><p> |
| You can cast the result to the type that matches |
| the descriptor. |
| </p><p> |
| The result of unpacking should be freed with protobuf_c_message_free(). |
| </p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2479561"></a>Author</h2></div></div></div><p>Dave Benson.</p></div></div></body></html> |