OverviewOpenWire is used to marshal objects to byte arrays and back. We will refer to the marshaled objects as commands. A TCP network connection would see multiple commands back to back on the stream. Commands are not delimited in anyway and are variable sized. +---------+ +---------+ +---------+ | command | | command | .... | command | +---------+ +---------+ +---------+ All data primitive types used in the encoded commands are encoded in big-endian/network byte order.
primitive types and encoding:
| | | | | |
+----------+ | +-----------+ | +-----------+ | +-----------+ | +-----------+ | +-----------+ | +-----------+
| byte | | | char | | | short | | | int | | | long | | | float | | | double |
+----------+ | +-----------+ | +-----------+ | +-----------+ | +-----------+ | +-----------+ | +-----------+
| 1 octect | | | 2 octects | | | 2 octects | | | 4 octects | | | 8 octects | | | 4 octects | | | 8 octects |
+----------+ | +-----------+ | +-----------+ | +-----------+ | +-----------+ | +-----------+ | +-----------+
| | | | | |
Wire Format NegotiationOpenWire is an extensible in that it supports adding new encoding options while still being backward compatible with previous versions of the protocol. Every OpenWire protocol session initially starts with all encoding options turned off and at version 1 of command marshalers. An initial WIREFORMAT_INFO command is exchanged between the two nodes so that additional encoding features can be enabled. If both sides request an encoding feature to be enabled then it will be enabled. The command marshalers used will be the highest version that they both support. +-------------------------+ +---------+ +---------+ +---------+ | WIREFORMAT_INFO command | | command | | command | .... | command | +-------------------------+ +---------+ +---------+ +---------+ Command EncodingEvery command is encoded as follows: command encoding: [=If SizePrefixDisabled =] [ option is not enabled. ] [ +------+ ] +------+-------------------------+ [ | size | ] | type | command-specific-fields | [ +------+ ] +------+-------------------------+ [ | int | ] | byte | (size-1) octects | [ +------+ ] +------+-------------------------+ [========================]
If a command type does not have any command specific content, then size would be 1. Prefixing the size on every command is done to aid in non-blocking IO processing so that receiver can easily determine when a whole command has been received and can be processed. But finding out the size of command takes additional buffering on the sender size and is not needed if the receiver is doing blocking IO. If the SizePrefixDisabled option is enabled after the exchange of WIREFORMAT_INFO packets, then every subsequent command will not have the size prefixed. Command Types
Command Field EncodingAll OpenWire commands use the same algorithm when encoding their fields. The algorithm restricts commands to use fields that are of the following type:
Notice that OpenWire commands can nest other OpenWire commands in it's fields. Care must be taken that only acyclic graphs of commands are marshaled. String Type EncodingStrings fields may be null. If it is null then it encodes to a single "0" byte.
string encoding:
[=If not-null is 1===========]
+----------+ [ +-------+----------------+ ]
| not-null | [ | size | encoded-string | ]
+----------+ [ +-------+----------------+ ]
| byte | [ | short | size octects | ]
+----------+ [ +-------+----------------+ ]
[============================]
Byte Array Type EncodingByte array fields may be null. If it is null then it encodes to a single "0" byte. byte-array encoding: [=If not-null is 1========] +----------+ [ +------+--------------+ ] | not-null | [ | size | data | ] +----------+ [ +------+--------------+ ] | byte | [ | int | size octects | ] +----------+ [ +------+--------------+ ] [=========================]
N Sized Byte Array Type EncodingFixed Size Byte array fields may NOT be null and their length must be N length. Used for byte arrays that are always a fixed size.
N-sized-byte-array encoding:
+-----------+
| data |
+-----------+
| N octects |
+-----------+
Throwable Type EncodingThrowable fields may be null. If it is null then it encodes to a single "0" byte.
throwable encoding:
[=If not-null is 1===========================================================================]
[ [=If StackTraceEnabled option is enabled.==================] ]
[ [ [=Repeated size times======================] ] ]
+----------+ [ +----------------+---------+ [ +-------+ [ +--------+--------+--------+-------------+ ] ] ]
| not-null | [ | exception-name | message | [ | size | [ | class | method | file | line-number | ] ] ]
+----------+ [ +----------------+---------+ [ +-------+ [ +--------+--------+--------+-------------+ ] ] ]
| byte | [ | string | string | [ | short | [ | string | string | string | int | ] ] ]
+----------+ [ +----------------+---------+ [ +-------+ [ +--------+--------+--------+-------------+ ] ] ]
[ [ [============================================] ] ]
[ [==========================================================] ]
[============================================================================================]
If the StackTraceEnabled encoding option is enabled after the exchange of WIREFORMAT_INFO packets, then every Throwable filed will have stack trace data appended to it. Nested Command Type EncodingNested command fields may be null. If it is null then it encodes to a single "0" byte.
nested-object encoding:
[=If not-null is 1===================]
+----------+ [ +------+-------------------------+ ]
| not-null | [ | type | command-specific-fields | ]
+----------+ [ +------+-------------------------+ ]
| byte | [ | byte | variable sized | ]
+----------+ [ +------+-------------------------+ ]
[====================================]
Cached Nested Command Type EncodingNested Command types can be cached so that subsequent marshaling operations of the same object result in a smaller on the wire size. By default the CacheEnabled option is not enabled and therefore standard nested-object encoding is used. cached-object-encoding: [=If CacheEnabled option is enabled=====================] [ [=If new-value is 1===========] ] [ +-----------+-------+ [ +-------------------------+ ] ] [ | new-value | key | [ | command-specific-fields | ] ] [ +-----------+-------+ [ +-------------------------+ ] ] [ | byte | short | [ | nested-object | ] ] [ +-----------+-------+ [ +-------------------------+ ] ] [ [=============================] ] [=====================================================] ] [=If CacheEnabled option is disabled =] [ +-------------------------+ ] [ | command-specific-fields | ] [ +-------------------------+ ] [ | nested-object | ] [ +-------------------------+ ] [=====================================]
Loose EncodingThe default encoding explained so far is called "loose encoding" and is the default encoding used when OpenWire if first initialized. Loose encoding is simple to implement does not add much CPU overhead to the marshaling/unmarshaling process. It is able to marshal an object graph in single pass of the object tree. Tight EncodingAnother encoding option is supported by OpenWire which is called TightEncoding. When tight encoding is used, it uses bit streams to marshal all the boolean values that would have taken up a byte in loose encoding to a bit in the bit stream. To build the bit stream 2 passes through the object graph are taken so this is more CPU intensive marshaling process but it produces a smaller on the wire size. |
