4.11.3 Payloads

The reserved fields discussed in the previous section are most often discrete units like words, double-words, and the like, they are usually of some fixed size, and they are used to delimit some space that is not to be used.

Another kind of padding happens when an entity contains space to be used to store some kind of payload whose contents are not determined. This would be such an example:

type Packet =
  struct
  {
    offset<uint<32>,B> payload_size;
    byte[payload_size] payload;
    int flags;
  };

In this example we are using a payload field which is an array of bytes. The size of the payload is determined by the packet header, and the contents are not determined. Of course this assumes that the payload sizes are divisible in whole bytes; a bit-oriented format may need to use an array of bits instead.

This approach of using a byte (or bit) array like in the example above has the advantage of providing a field with the bytes (or bits) to the user, for inspection and modification:

(poke) packet.payload
[23UB, ...]
(poke) packet.payload[0] = 0

The user can still map whatever payload structure in that space using the attributes of a mapped Packet. For example, if the packet contains an array of ULEB128 numbers, we could do:

(poke) var numbers = ULEB128[packet.payload'size] @ packet.payload'offset

But this approach has a disadvantage: every time the packet structure is mapped or written the entire payload array gets decoded and encoded. If the payloads are big enough (think about the data blocks of a file described by a file system i-node for example) this can be a big problem in terms of performance.

Another problem of using byte (or bit) arrays for payloads is that the printed representation of the struct values include the contents of the arrays, and most often the user won’t be interested in seeing that:

(poke) Packet { payload_size = 23#B }
Packet {
  payload_size=0x17U#B,
  payload=[0x0UB,0x0UB,0x0UB,0x0UB,0x0UB,...],
  flags=0x0
}

Another alternative is to implement the padding implied by a payload using field labels:

type Packet =
  struct
  {
    offset<uint<32>,B> payload_size;
    int flags @ OFFSET + payload_size;
  };

Note how a payload field no longer exists in the struct type, and the field flags is defined to start at offset OFFSET + payload_size. This way no explicit array is encoded/decoded when manipulating Packet values:

(poke) .set omaps yes
(poke) Packet { payload_size = 500#Mb }
Packet {
  payload_size=62500000U#B @ 0UL#b,
  flags=0 @ 4000000032UL#b
} @ 0UL#b

In this example we used the omaps option, which asks poke to print the offsets of the fields. The offset of flags is 4000000032 bits, or 500 megabytes:

(poke) 4000000032UL #b/#MB
500UL

Mapping this new Packet involves reading and decoding five bytes, for the payload_size and flags only. This is clearly much faster and avoids unneeded IO.

However you may be wondering, if there is no explicit payload field, how to access the payload space? A way is to define a method to the struct to provide the payload attributes:

type Packet =
  struct
  {
    offset<uint<32>,B> payload_size;:
    var payload_offset = OFFSET;
    int flags @ OFFSET + payload_size;

    method get_payload_offset = off64:
    {
      return payload_offset;
    }
  };

Note how we captured the offset of the payload using a variable in the strict type definition. Returning OFFSET in get_payload_offset wouldn’t work for obvious reasons: in the body of the method OFFSET evaluates to the end of flags in this case.

Using this method you can easily access the payload (again as an array of ULEB128 numbers) like this:

var numbers = ULEB128[packet.payload_size @ packet.get_payload_offset

Finally, using labels for this purpose makes the printed representation of the struct values more readable by not including the payload bytes in it:

(poke) Packet {}
Packet {
  payload_size=0x0U#B,
  flags=0x0
}