v2 Reference Profile

The v2 reference profile is the normative interoperability profile for Twilic v2. All conforming implementations must satisfy the requirements listed here.

v2 is a clean break from v1. A v2 decoder is not required to decode v1 payloads.

Required Tag Families

A v2 conforming encoder MUST support all first-byte tag families:

Range	Family
0x00..0x7F	positive fixint
0x80..0x9F	fixstr
0xA0..0xAF	fixarray
0xB0..0xBF	fixmap
0xE0..0xFF	negative fixint

And all extended tags from 0xC0 through 0xDF. See Wire Tags for the full table.

Required Kinds

Dynamic Profile (Required)

• MAP: fixmap, map16, map32 with key_ref support
• ARRAY: fixarray, array16, array32
• SCALAR: all fixint, u8..u64, i8..i64, f64, null, false, true
• STRING: fixstr, str8, str16, str32
• BINARY: bin8, bin16, bin32
• KEY INTERNING: key_ref (0xD8)
• STRING INTERNING: str_ref (0xD9)
• SHAPE INTERNING: shape_def (0xD6) + shape_ref (0xD7)
• TYPED VECTOR: typed_vec (0xDA)

Batch Profile (Required)

• row_batch (0xDB)
• col_batch (0xDC) with all required codecs (see below)

Stateful Profile (Optional)

Stateful forms are optional. Implementations that do not support stateful mode MUST handle stateless messages correctly and MUST reject stateful frames with a clear error.

• state_patch (0xDD)
• template_batch (0xDE)

Required Reset Operations

• RESET_STATE: invalidates all session state (bases, templates, dictionaries). Mandatory to implement; both sender and receiver must honor it.

After RESET_STATE:

• Both sides treat all old state IDs as invalid.
• Sender must emit stateless full frame or fresh registrations before resuming stateful references.

Required Column Codecs

All v2 implementations MUST support these codecs in typed_vec and col_batch:

Codec	Description
DIRECT_BITPACK	Direct bit packing
DELTA_BITPACK	Delta + bit packing
FOR_BITPACK	Frame-of-reference + bit packing
RLE	Run-length encoding
SIMPLE8B	Simple-8b: packs multiple small integers per 64-bit word
XOR_FLOAT	XOR of adjacent floats + bit packing

The following codecs are RECOMMENDED but not required:

Codec	Description
DELTA_FOR_BITPACK	Delta + FOR + bit packing
DELTA_DELTA_BITPACK	Second-order delta + bit packing
PATCHED_FOR	FOR with outlier patching

Default Deterministic Heuristics

All v2 implementations MUST use these deterministic encoding rules:

Integer width selection: smallest valid width (fixint first, then by range).

Key ID assignment: first-seen order within a message. Randomized assignment is forbidden.

String ID assignment: first-seen order within a message.

Shape ID assignment: first-seen order when shape_def appears.

Codec selection for typed_vec / col_batch: deterministic for equal input statistics and equal profile configuration. Randomized or non-deterministic selection is forbidden.

Recommended Metadata

Implementations SHOULD include:

• Format version in session handshake or frame header when operating in stateful mode.
• Profile indicator when mixing Dynamic and Bound profiles in the same stream.

Constraints for Stateful Operation

When using stateful mode:

• Ordered delivery MUST be guaranteed.
• Reset frames (RESET_STATE) MUST NOT be dropped.
• State ID spaces (base_id, template_id, dictionary IDs) MUST NOT be reused within the same session epoch.
• Unknown reference policy MUST be fixed at deployment time: either fail-fast or stateless retry.

Interoperability

An implementation is considered v2-interoperable when it:

1. Encodes all Dynamic Profile values deterministically.
2. Decodes all Dynamic, Batch, and stateless Bound messages correctly.
3. Resets all intern tables at each top-level message boundary.
4. Fails decode on unknown key_ref, str_ref, or shape_ref IDs.
5. Supports all required column codecs.
6. Honors RESET_STATE.

Extension Point

ext (0xDF) is available for application-defined extensions.

• Extension types 0x00..0x7F are reserved for future Twilic specification use.
• Extension types 0x80..0xFF are available for application-defined use.

Implementations MUST NOT silently discard unknown ext frames — they should either pass them through or fail with a clear error.