Buffer Management
Overview
Sia's binary serialization is built on top of a lightweight Buffer class that wraps JavaScript's Uint8Array and DataView. This design provides fast, low-level byte operations while remaining compatible with both browser and Node.js environments.
Every Sia instance holds a contiguous byte buffer, a DataView for multi-byte reads/writes, and an offset that tracks the current position. Understanding how buffers are allocated, shared, and extracted is key to writing efficient Sia code.
The Buffer Class
The Sia class extends an internal Buffer class that manages the raw byte storage:
class Buffer {
public size: number; // Total capacity in bytes
public content: Uint8Array; // The underlying byte array
public offset: number; // Current read/write position
public dataView: DataView; // View for multi-byte operations
constructor(uint8Array: Uint8Array) {
/* ... */
}
static alloc(size: number): Buffer; // New zero-filled buffer
static allocUnsafe(size: number): Buffer; // Slice from shared pool
seek(offset: number): this; // Set position, returns this
skip(count: number): this; // Advance position, returns this
setContent(uint8Array: Uint8Array): this; // Replace backing buffer
}
The Buffer class is also exported directly, so you can use it with the standalone functions for tree-shakeable imports:
import { Buffer, addUInt8, readUInt8 } from "@timeleap/sia";
const buf = Buffer.alloc(64);
addUInt8(buf, 42);
buf.seek(0);
console.log(readUInt8(buf)); // 42
| Property | Type | Description |
|---|---|---|
size | number | Total capacity of the buffer in bytes |
content | Uint8Array | The underlying byte array holding serialized data |
offset | number | Current read/write position, advanced automatically |
dataView | DataView | Used internally for multi-byte integer operations |
The DataView is constructed with the correct byteOffset and byteLength, which ensures that subarrays (used by allocUnsafe) work correctly without writing outside their boundaries.
Buffer Allocation Strategies
Sia provides three ways to create instances, each with different memory and performance characteristics.
Default Constructor (Shared 32 MB Buffer)
import { Sia } from "@timeleap/sia";
const sia = new Sia();
When you create a Sia instance without arguments, it uses a global shared 32 MB buffer that is lazy-allocated on first use:
const GLOBAL_SHARED_UNSAFE_BUFFER = {
buffer: null as Uint8Array | null,
offset: 0,
};
function getSharedBuffer(): Uint8Array {
if (!GLOBAL_SHARED_UNSAFE_BUFFER.buffer) {
GLOBAL_SHARED_UNSAFE_BUFFER.buffer = new Uint8Array(32 * 1024 * 1024);
}
return GLOBAL_SHARED_UNSAFE_BUFFER.buffer;
}
This is the fastest option because it avoids per-call memory allocation. The buffer is created once on first use and reused across all default-constructed instances. Importing the module does not allocate the buffer until you actually create a Sia instance or call a function that needs it.
Sia instances
write to the same underlying memory. Always extract your data with
toUint8Array() before creating another default instance or before the data
leaves the current synchronous scope.Safe Allocation (Sia.alloc)
const sia = Sia.alloc(1024); // 1 KB dedicated buffer
Allocates a new, zero-initialized Uint8Array of the given size. The buffer is completely independent of other instances.
static alloc(size: number): Sia {
return new Sia(new Uint8Array(size));
}
Use Sia.alloc when you need:
- An isolated buffer that won't be affected by other Sia operations
- A buffer that will be held in memory beyond the current synchronous scope
- Concurrent serialization across async boundaries
Unsafe Allocation (Sia.allocUnsafe)
const sia = Sia.allocUnsafe(256);
Carves a subarray from the global shared buffer, advancing the shared offset. This avoids allocation overhead while giving you a bounded region to write into.
static allocUnsafe(size: number): Sia {
const shared = getSharedBuffer();
const begin =
GLOBAL_SHARED_UNSAFE_BUFFER.offset + size > shared.length
? 0
: GLOBAL_SHARED_UNSAFE_BUFFER.offset;
const subarray = shared.subarray(begin, begin + size);
GLOBAL_SHARED_UNSAFE_BUFFER.offset = begin + size;
return new Sia(subarray);
}
The offset wraps around to 0 when the shared buffer is full, so stale data from previous operations may be present. This is safe as long as you write before you read.
When to Use What
| Constructor | Allocation | Isolation | Best For |
|---|---|---|---|
new Sia() | None (shared) | None | Quick serialize + extract |
Sia.alloc(n) | New buffer | Full | Long-lived buffers, concurrent use |
Sia.allocUnsafe(n) | Shared slice | Partial | High-throughput, short-lived buffers |
Memory Operations
Writing Data
The Buffer class provides two low-level write methods used internally by Sia:
// Write a single byte
add(data: Uint8Array) {
if (this.offset + data.length > this.size) {
throw new Error("Buffer overflow");
}
this.content.set(data, this.offset);
this.offset += data.length;
}
// Write a Uint8Array
addOne(data: number) {
if (this.offset + 1 > this.size) {
throw new Error("Buffer overflow");
}
this.content[this.offset] = data;
this.offset++;
}
Both methods check for overflow before writing and advance the offset automatically.
Offset Management
Use seek to jump to an absolute position, or skip to advance by a relative number of bytes:
const sia = new Sia();
// Write some data
sia.addUInt32(42).addString8("hello");
// Jump back to the beginning to read
sia.seek(0);
const value = sia.readUInt32(); // 42
// Skip past known-length fields
sia.skip(6); // skip the string (1 byte length + 5 bytes data)
Both methods return this for chaining:
sia.seek(0).skip(4).readString8();
Extracting Data
After serializing, you need to extract the bytes from the buffer. Sia provides two methods with different trade-offs.
toUint8Array(): Safe Copy
const bytes = sia.toUint8Array();
Returns a new Uint8Array containing a copy of the written data (from offset 0 to the current offset). The returned array is independent of the Sia buffer: safe to store, send over the network, or pass to any API.
toUint8ArrayReference(): Zero Copy
const ref = sia.toUint8ArrayReference();
Returns a subarray view of the internal buffer. No data is copied, so there is no allocation. However, the returned array shares memory with the Sia instance.
toUint8ArrayReference(), the returned bytes will be overwritten if the buffer is reused. Always use toUint8Array() when the data must outlive the current operation.const sia = new Sia();
sia.addUInt8(42);
const ref = sia.toUint8ArrayReference();
console.log(ref[0]); // 42
sia.seek(0);
sia.addUInt8(99);
console.log(ref[0]); // 99 -- reference sees the change
Memory Safety
Overflow Protection
Every write operation checks that there is enough space in the buffer before writing. If a write would exceed the buffer capacity, a Buffer overflow error is thrown:
const sia = Sia.alloc(2);
sia.addUInt8(1); // OK
sia.addUInt8(2); // OK
sia.addUInt8(3); // throws "Buffer overflow"
Subarray Boundaries
When using Sia.allocUnsafe, the DataView is constructed with the correct byteOffset and byteLength from the subarray. This ensures that writes stay within the allocated region and do not corrupt adjacent memory:
const byteArray = new Uint8Array(8).fill(0x01);
const sub = byteArray.subarray(4);
const sia = new Sia(sub);
sia.addUInt32(1);
// Bytes 0-3 of the original array are untouched
const untouched = byteArray.slice(0, 4);
// [0x01, 0x01, 0x01, 0x01] -- unmodified
Best Practices
Copy Before Storing
toUint8Array() when data must outlive the current scope. Use
toUint8ArrayReference() only for immediate consumption.Right-Size Your Buffers
Sia.alloc(n) with a known size to avoid wasting memory. The default 32
MB buffer is convenient but oversized for small payloads.Complete Example
Here's a full example combining allocation, writing, extraction, and reading:
import { Sia } from "@timeleap/sia";
// 1. Allocate a dedicated buffer
const writer = Sia.alloc(256);
// 2. Serialize data
writer
.addUInt8(1) // version
.addString8("Alice") // name
.addUInt32(1000) // score
.addBool(true); // active
// 3. Extract a safe copy of the bytes
const bytes = writer.toUint8Array();
// 4. Create a reader from the bytes
const reader = new Sia(bytes);
const version = reader.readUInt8(); // 1
const name = reader.readString8(); // "Alice"
const score = reader.readUInt32(); // 1000
const active = reader.readBool(); // true
// 5. Or reuse the writer by seeking back
writer.seek(0);
const v = writer.readUInt8(); // 1
Performance Considerations
Choose the Right Allocation Strategy
For maximum throughput in request-response scenarios (e.g., WebSocket RPC), use Sia.allocUnsafe combined with toUint8Array(). This avoids allocation for the buffer itself while still producing a safe copy of the output:
function handleRequest(data: Uint8Array): Uint8Array {
const reader = new Sia(data);
const method = reader.readAscii8();
const writer = Sia.allocUnsafe(256);
writer.addString8(processRequest(method));
return writer.toUint8Array();
}
Use Zero-Copy Reads in Hot Paths
When reading byte arrays in performance-critical loops, pass asReference = true to avoid copying:
const chunk = sia.readByteArray32(true); // subarray, no copy
processChunk(chunk); // use immediately
// Don't store `chunk` -- it shares buffer memory
Reuse Sia Instances
Instead of creating new Sia instances in a loop, reuse one with setContent():
const sia = new Sia();
for (const message of messages) {
sia.setContent(message);
const type = sia.readUInt8();
const payload = sia.readString16();
// process...
}
Prefer ASCII for Known-ASCII Strings
addAscii8 / readAscii8 bypasses the TextEncoder / TextDecoder and uses a hand-optimized conversion loop. Use it when you know the string contains only ASCII characters (codes 0--127).