Sia

Quick Start

Serialize and deserialize data, manage offsets, and work with complex types.
Want the easy path? If you have more than a couple of message types, start with the Schema Quick Start instead. Define your types in a .sia file and get generated encode/decode functions for every language — no manual serialization code needed.

Prerequisites

Before you begin, make sure you have Sia installed for your language. See the Installation Guide for details.

Your First Serialization

Create an instance

import { Sia } from "@timeleap/sia";

const sia = new Sia();
The default constructor uses a shared 32 MB global buffer (lazy-allocated on first use) for maximum performance. This is ideal for serializing data in hot paths where you process one message at a time. For isolated buffers, use Sia.alloc(size) or pass your own Uint8Array to the constructor.

::

import sia "github.com/TimeleapLabs/go-sia/v2/pkg"

s := sia.New()

from sia import Sia

s = Sia()

#include <sia/sia.hpp>

auto s = sia::New();

Serialize data

sia
  .addString8("Alice")
  .addUInt8(30)
  .addString16("alice@example.com")
  .addBool(true);

const bytes = sia.toUint8Array();

s.AddString8("Alice").
    AddUInt8(30).
    AddString16("alice@example.com").
    AddBool(true)

bytes := s.Bytes()

s.add_string8("Alice") \
 .add_uint8(30) \
 .add_string16("alice@example.com") \
 .add_bool(True)

raw_bytes = s.content

s->AddString8("Alice")
 ->AddUInt8(30)
 ->AddString16("alice@example.com")
 ->AddBool(true);

auto bytes = s->Bytes();
Every add* method returns the instance, so you can chain calls into a single fluent expression. This works in all four languages.

Deserialize data

Read fields back in the same order they were written:

const reader = new Sia(bytes);

const name = reader.readString8(); // "Alice"
const age = reader.readUInt8(); // 30
const email = reader.readString16(); // "alice@example.com"
const active = reader.readBool(); // true

reader := sia.NewFromBytes(bytes)

name := reader.ReadString8()    // "Alice"
age := reader.ReadUInt8()       // 30
email := reader.ReadString16()  // "alice@example.com"
active := reader.ReadBool()     // true

reader = Sia()
reader.set_content(raw_bytes)

name = reader.read_string8()     # "Alice"
age = reader.read_uint8()        # 30
email = reader.read_string16()   # "alice@example.com"
active = reader.read_bool()      # True

auto reader = sia::NewFromBytes(bytes);

auto name = reader->ReadString8();    // "Alice"
auto age = reader->ReadUInt8();       // 30
auto email = reader->ReadString16();  // "alice@example.com"
auto active = reader->ReadBool();     // true
You must read fields in the exact same order and with the same types as they were written. Sia is a sequential format with no field names or type markers: reading out of order will produce corrupted data or throw errors.

::

Understanding Offsets

Sia maintains an internal offset that advances automatically as you read or write data.

const sia = new Sia();

// offset starts at 0
sia.addUInt8(10); // writes 1 byte, offset is now 1
sia.addUInt32(1000); // writes 4 bytes, offset is now 5
sia.addString8("hi"); // writes 1 (length) + 2 (data) = 3 bytes, offset is now 8

// To read, reset the offset
sia.seek(0); // offset is back to 0
sia.readUInt8(); // reads 1 byte, offset is now 1
sia.readUInt32(); // reads 4 bytes, offset is now 5
sia.readString8(); // reads 3 bytes, offset is now 8

s := sia.New()

// offset starts at 0
s.AddUInt8(10)    // writes 1 byte
s.AddUInt32(1000) // writes 4 bytes
s.AddString8("hi") // writes 1 (length) + 2 (data) = 3 bytes

// To read, reset the offset
s.Seek(0)
s.ReadUInt8()    // reads 1 byte
s.ReadUInt32()   // reads 4 bytes
s.ReadString8()  // reads 3 bytes

s = Sia()

# offset starts at 0
s.add_uint8(10)     # writes 1 byte
s.add_uint32(1000)  # writes 4 bytes
s.add_string8("hi") # writes 1 (length) + 2 (data) = 3 bytes

# To read, reset the offset
s.seek(0)
s.read_uint8()     # reads 1 byte
s.read_uint32()    # reads 4 bytes
s.read_string8()   # reads 3 bytes

auto s = sia::New();

// offset starts at 0
s->AddUInt8(10);     // writes 1 byte
s->AddUInt32(1000);  // writes 4 bytes
s->AddString8("hi"); // writes 1 (length) + 2 (data) = 3 bytes

// To read, reset the offset
s->Seek(0);
s->ReadUInt8();     // reads 1 byte
s->ReadUInt32();    // reads 4 bytes
s->ReadString8();   // reads 3 bytes

Working with Arrays

Arrays serialize a list of elements with a length prefix. You provide a callback that defines how each element is serialized or deserialized:

const scores = [95, 87, 72, 100, 88];

const sia = new Sia();
sia.addArray8(scores, (s, score) => {
  s.addUInt8(score);
});

sia.seek(0);
const result = sia.readArray8((s) => s.readUInt8());
// [95, 87, 72, 100, 88]

scores := []uint8{95, 87, 72, 100, 88}

s := sia.NewSiaArray[uint8]()
s.AddArray8(scores, func(s *sia.ArraySia[uint8], score uint8) {
    s.AddUInt8(score)
})

reader := sia.NewArrayFromBytes[uint8](s.Bytes())
result := reader.ReadArray8(func(s *sia.ArraySia[uint8]) uint8 {
    return s.ReadUInt8()
})

scores = [95, 87, 72, 100, 88]

s = Sia()
s.add_array8(scores, lambda s, score: s.add_uint8(score))

s.seek(0)
result = s.read_array8(lambda s: s.read_uint8())
# [95, 87, 72, 100, 88]

#include <sia/array.hpp>

std::vector<int> scores = {95, 87, 72, 100, 88};

auto s = sia::New();
sia::AddArray8<int>(s, scores,
    [](auto self, const int& score) { self->AddUInt8(score); });

auto reader = sia::NewFromBytes(s->Bytes());
auto result = sia::ReadArray8<int>(reader,
    [](auto self) -> int { return self->ReadUInt8(); });

The suffix (8, 16, 32, 64) determines the maximum number of elements:

  • addArray8: up to 255 elements
  • addArray16: up to 65,535 elements
  • addArray32: up to ~4 billion elements

Serializing Objects

Sia does not have a built-in object type. Instead, define serializer and deserializer functions for your data structures:

interface User {
  name: string;
  age: number;
  email: string;
}

function serializeUser(sia: Sia, user: User): void {
  sia.addString8(user.name).addUInt8(user.age).addString16(user.email);
}

function deserializeUser(sia: Sia): User {
  return {
    name: sia.readString8(),
    age: sia.readUInt8(),
    email: sia.readString16(),
  };
}

const users: User[] = [
  { name: "Alice", age: 30, email: "alice@example.com" },
  { name: "Bob", age: 25, email: "bob@example.com" },
];

const sia = new Sia();
sia.addArray8(users, serializeUser);

sia.seek(0);
const result = sia.readArray8(deserializeUser);

type User struct {
    Name  string
    Age   uint8
    Email string
}

users := []User{
    {Name: "Alice", Age: 30, Email: "alice@example.com"},
    {Name: "Bob", Age: 25, Email: "bob@example.com"},
}

s := sia.NewSiaArray[User]()
s.AddArray8(users, func(s *sia.ArraySia[User], user User) {
    s.AddString8(user.Name).AddUInt8(user.Age).AddString16(user.Email)
})

reader := sia.NewArrayFromBytes[User](s.Bytes())
result := reader.ReadArray8(func(s *sia.ArraySia[User]) User {
    return User{
        Name:  s.ReadString8(),
        Age:   s.ReadUInt8(),
        Email: s.ReadString16(),
    }
})

from dataclasses import dataclass

@dataclass
class User:
    name: str
    age: int
    email: str

users = [
    User("Alice", 30, "alice@example.com"),
    User("Bob", 25, "bob@example.com"),
]

s = Sia()
s.add_array8(users, lambda s, u: (
    s.add_string8(u.name).add_uint8(u.age).add_string16(u.email)
))

s.seek(0)
result = s.read_array8(lambda s: User(
    s.read_string8(),
    s.read_uint8(),
    s.read_string16()
))

struct User {
    std::string name;
    uint8_t age;
    std::string email;
};

std::vector<User> users = {
    {"Alice", 30, "alice@example.com"},
    {"Bob", 25, "bob@example.com"},
};

auto s = sia::New();
sia::AddArray8<User>(s, users,
    [](auto self, const User& u) {
        self->AddString8(u.name)->AddUInt8(u.age)->AddString16(u.email);
    });

auto reader = sia::NewFromBytes(s->Bytes());
auto result = sia::ReadArray8<User>(reader,
    [](auto self) -> User {
        return {
            self->ReadString8(),
            self->ReadUInt8(),
            self->ReadString16()
        };
    });
If you're defining many message types, consider using the schema compiler to generate serialization code automatically.

Error Handling

Each language handles serialization errors differently:

TypeScript throws Error objects with descriptive messages:

const sia = new Sia(new Uint8Array(0));
try {
  sia.readUInt8();
} catch (e) {
  console.error(e.message); // "Not enough data to read uint8"
}

Go returns zero values silently on out-of-bounds reads. No errors or panics:

reader := sia.NewFromBytes([]byte{})
reader.ReadUInt8()    // returns 0
reader.ReadString8()  // returns ""
reader.ReadBool()     // returns false

Python raises ValueError with descriptive messages:

s = Sia()
try:
    s.read_uint8()
except ValueError as e:
    print(e)  # "Not enough data to read uint8"

C++ returns zero/empty values silently on out-of-bounds reads. No exceptions:

auto reader = sia::NewFromBytes({});
reader->ReadUInt8();     // returns 0
reader->ReadString8();   // returns ""
reader->ReadBool();      // returns false

Functional API (TypeScript only)

TypeScript also provides standalone functions that work directly on a Buffer. These are tree-shakeable: your bundler can drop any functions you don't use.

import {
  Buffer,
  addString8,
  addUInt8,
  readString8,
  readUInt8,
} from "@timeleap/sia";

const buf = Buffer.alloc(256);
addString8(buf, "Alice");
addUInt8(buf, 30);

const bytes = buf.toUint8Array();

const reader = new Buffer(bytes);
console.log(readString8(reader)); // "Alice"
console.log(readUInt8(reader)); // 30

The functional and class APIs are interchangeable: the Sia class methods are thin wrappers around these functions.

Best Practices

Match read and write order

Always read fields in the exact same order and types as they were written.

Choose the right size

Use the smallest integer and length prefix that fits your data. addUInt8 for values up to 255, addString8 for short strings, etc.

Reuse instances

In hot loops, reuse a single instance with seek(0) rather than creating new ones.

Schema Compiler

Define schemas in .sia files and generate code for all four languages.

Next Steps

Core Concepts

How Sia manages buffers, memory, and offsets internally.

API Reference

Complete reference for every method on the Sia class.

Schema Compiler

Define schemas in .sia files and generate serialization code for all four languages.

Installation

Install Sia for TypeScript, Go, Python, or C++ and verify your setup.

Buffer Management

Understand Sia's buffer system built on Uint8Array and DataView, allocation strategies, and memory safety.