TezzLLM Weights Format (TEZW)
TezzLLM models are saved in a custom binary layout called TEZW (TezzLLM Weight format v2). This page details the layout and header structures.
📐 Binary Specifications
All multi-byte integers in the file are stored in little-endian byte order. Float values are stored as double-precision floating-point numbers (float64 / 8 bytes each).
File Structure Table:
| Offset (Bytes) | Size (Bytes) | Data Type | Field | Description |
|----------------|--------------|-----------|-------|-------------|
| 0 | 4 | Char | magic | Magic signature: 'T', 'Z', '2', 'W' |
| 4 | 4 | Int32 | version | File format version (currently 2) |
| 8 | 4 | Int32 | n_layer | Number of transformer layers (default 4) |
| 12 | 4 | Int32 | n_head | Number of attention heads (default 4) |
| 16 | 4 | Int32 | dim | Hidden embedding dimension (default 128) |
| 20 | 4 | Int32 | vocab_sz | Vocabulary size (default 256) |
| 24 | 4 | Int32 | max_seq | Maximum sequence context window (default 128) |
| 28 | 4 | Int32 | ffn_dim | Feed-forward network dimension (default 512) |
| 32 | 8 | Double | - | Reserved / Padding |
| 40+ | Variable | Double[] | weights | Flat array of parameter values (double-precision float) |
🧮 Parameter Ordering
The flat double array starting at byte offset 40 contains the model weights in the exact sequential order listed below:
- Token Embedding Table (
wte)
[vocab_sz, dim]
- Total values: 256 128 = 32,768
- Final RMSNorm scale (
rms_f)
[dim]
- Total values: 128
- Per-Layer Parameters
n_layer layers sequentially, the following blocks are concatenated:
- RMSNorm 1 scale (rms1): [dim] (128 values)
- Query Projection (wq): [dim, dim] (16,384 values)
- Key Projection (wk): [dim, dim] (16,384 values)
- Value Projection (wv): [dim, dim] (16,384 values)
- Output Projection (wo): [dim, dim] (16,384 values)
- RMSNorm 2 scale (rms2): [dim] (128 values)
- SwiGLU Gate Projection (wgate): [ffn_dim, dim] (65,536 values)
- SwiGLU Value Projection (wval): [ffn_dim, dim] (65,536 values)
- SwiGLU Projection Output (wproj): [dim, ffn_dim] (65,536 values)
Total Parameter Math (Tiny v1):
- Embedding:
32,768 - Final Normalization:
128 - Layer Block:
128 + (16,384 4) + 128 + (65,536 3) = 65,536 + 256 + 196,608 = 262,400per layer. - Total for 4 layers:
262,400 4 = 1,049,600 - Total parameters:
32,768 + 128 + 1,049,600 = 1,082,496 - Total file size:
40 bytes (header) + 1,082,496 8 bytes = 8,660,008 bytes (~8.66 MB)
🛠 Parsing Code (TezzNative)
This example shows how weight fields are loaded in the inference engine:
fn load_weights(path:str) -> int:
data:str = read_file(path)
if (data as int) == 0:
ret 0
// Verify magic signature
unsafe:
ptr:char = data as *char
if ptr[0] != 'T' || ptr[1] != 'Z' || ptr[2] != '2' || ptr[3] != 'W':
ret 0 // Magic error
n_layer:int = read_u32(data, 8)
n_head:int = read_u32(data, 12)
dim:int = read_u32(data, 16)
// Weights array starts at data pointer + 40 bytes
w_array:int = (data as int) + 40
ret w_array