Advanced

Unsafe Buffer Allocation

How Sia's allocUnsafe works, when to use it, and how to avoid common pitfalls with shared buffer allocation.

Overview

Sia.allocUnsafe uses a shared global buffer. The returned Sia instance may contain stale data from previous operations. Only use it when you will overwrite all bytes before reading and consume the output before the buffer wraps around.

Sia provides allocUnsafe as a high-performance alternative to alloc. Instead of allocating a new Uint8Array and zeroing it out, allocUnsafe carves a subarray from a pre-allocated 32 MB shared buffer. This eliminates both the allocation cost and the zeroing cost, making it roughly 4x faster for short-lived serialization tasks.

Understanding `allocUnsafe`

How It Works

The shared buffer is a single Uint8Array of 32 MB, lazy-allocated on first use (not at import time). allocUnsafe returns a subarray starting at the current offset:

// Simplified internal implementation
const GLOBAL_SHARED_UNSAFE_BUFFER = {
  buffer: null as Uint8Array | null,
  offset: 0,
};

function getSharedBuffer(): Uint8Array {
  if (!GLOBAL_SHARED_UNSAFE_BUFFER.buffer) {
    GLOBAL_SHARED_UNSAFE_BUFFER.buffer = new Uint8Array(32 * 1024 * 1024);
  }
  return GLOBAL_SHARED_UNSAFE_BUFFER.buffer;
}

static allocUnsafe(size: number): Sia {
  const shared = getSharedBuffer();
  const begin =
    GLOBAL_SHARED_UNSAFE_BUFFER.offset + size > shared.length
      ? 0 // wrap around to the beginning
      : GLOBAL_SHARED_UNSAFE_BUFFER.offset;

  const subarray = shared.subarray(begin, begin + size);
  GLOBAL_SHARED_UNSAFE_BUFFER.offset = begin + size;
  return new Sia(subarray);
}

Key details:

The returned Sia instance operates on a subarray of the shared buffer, not a copy.
The offset advances with each call. When it would exceed the buffer length, it wraps around to 0.
The memory region is not zeroed: it may contain data from previous allocUnsafe calls.

Comparison with Other Constructors

Constructor	Allocation	Zeroing	Isolation	Speed
`new Sia()`	None (shared 32 MB)	No	None	Fastest
`Sia.allocUnsafe(n)`	None (shared slice)	No	Partial	Very fast
`Sia.alloc(n)`	New `Uint8Array(n)`	Yes	Full	Slower

Performance Benefits

Zero Allocation Cost

allocUnsafe does not call new Uint8Array(). The shared buffer is allocated once on first use and reused indefinitely.

// No allocation — returns a subarray of the shared buffer
function serializeEvent(type: number, data: string): Uint8Array {
  const sia = Sia.allocUnsafe(256);
  sia.addUInt8(type).addString8(data);
  return sia.toUint8Array(); // copies out the written portion
}
// The subarray is abandoned after toUint8Array — no cleanup needed

// Allocates and zeros 256 bytes every call
function serializeEvent(type: number, data: string): Uint8Array {
  const sia = Sia.alloc(256);
  sia.addUInt8(type).addString8(data);
  return sia.toUint8Array();
}
// Two allocations per call: the Sia buffer + the output copy

Reduced Garbage Collection Pressure

Since allocUnsafe does not create new Uint8Array objects, there is less for the garbage collector to track and sweep.

// 100,000 iterations: 100,000 allocations (output copies only)
for (let i = 0; i < 100_000; i++) {
  const sia = Sia.allocUnsafe(128);
  sia.addUInt32(i);
  results.push(sia.toUint8Array());
}

// 100,000 iterations: 200,000 allocations (buffer + output copy)
for (let i = 0; i < 100_000; i++) {
  const sia = Sia.alloc(128);
  sia.addUInt32(i);
  results.push(sia.toUint8Array());
}

Faster Instance Creation

No constructor overhead beyond computing the subarray bounds:

// Approximate benchmark results (ops/sec)
//
// Sia.allocUnsafe(256):  ~15,000,000 ops/sec
// Sia.alloc(256):        ~3,500,000 ops/sec
// new Sia():             ~18,000,000 ops/sec
//
// allocUnsafe is ~4x faster than alloc for instance creation

Safety Considerations

Uninitialized Memory

The returned buffer region may contain data from previous operations. This is safe only if you write before you read:

// Safe: all bytes are written before extraction
const sia = Sia.allocUnsafe(8);
sia.addUInt32(42).addUInt32(100);
const bytes = sia.toUint8Array(); // contains exactly what was written

// Dangerous: reading before writing
const sia2 = Sia.allocUnsafe(8);
const stale = sia2.toUint8Array(); // may contain stale data!

Never read from an allocUnsafe buffer before writing to it. The memory region may contain sensitive data from previous operations.

Buffer Wraparound

When the shared buffer offset would exceed 32 MB, it wraps around to 0. This means earlier allocUnsafe regions may be overwritten by newer calls:

// Earlier allocation
const a = Sia.allocUnsafe(1024);
a.addString16("important data");
const refA = a.toUint8ArrayReference(); // reference to shared buffer

// Many allocations later... the shared buffer wraps around
for (let i = 0; i < 100_000; i++) {
  Sia.allocUnsafe(512); // eventually wraps and overwrites 'a's region
}

// refA now contains garbage — the shared buffer was overwritten
console.log(refA); // unpredictable content

Always extract data with toUint8Array() (which copies) before the buffer can wrap.

Concurrent Usage

allocUnsafe is not safe for concurrent use across async boundaries. If two async operations call allocUnsafe and write interleaved data, they will corrupt each other's buffers.

// Unsafe: interleaved async writes to the shared buffer
async function handleRequest(req: Request): Promise<Uint8Array> {
  const sia = Sia.allocUnsafe(256);
  sia.addString8(req.method);
  await someAsyncOperation(); // another handler may call allocUnsafe here!
  sia.addString16(req.body); // shared buffer may have been modified
  return sia.toUint8Array();
}

// Safe: use alloc for concurrent scenarios
async function handleRequestSafe(req: Request): Promise<Uint8Array> {
  const sia = Sia.alloc(256); // dedicated buffer
  sia.addString8(req.method);
  await someAsyncOperation();
  sia.addString16(req.body);
  return sia.toUint8Array();
}

Best Practices

When to Use `allocUnsafe`

Use allocUnsafe when all of the following are true: serialization is synchronous (no await between writes), you write all bytes before reading or extracting, you extract data (toUint8Array()) before any other allocUnsafe call could overwrite the region, and performance is critical with measured allocation bottlenecks.

When to Avoid `allocUnsafe`

Avoid allocUnsafe when serialization spans async boundaries (await between writes), you need to hold a reference to the buffer across ticks or function calls, you are building a long-lived buffer (e.g., accumulating data over time), you are working in a multi-threaded environment (Web Workers with shared memory), or security is a concern and you cannot guarantee all bytes are overwritten.

Safe Usage Patterns

Pattern 1: Synchronous Serialize-and-Extract

The most common and safest pattern:

function serializeUser(user: User): Uint8Array {
  const sia = Sia.allocUnsafe(256);
  sia
    .addUInt32(user.id)
    .addString8(user.name)
    .addUInt8(user.age)
    .addBool(user.active);
  return sia.toUint8Array(); // copy out immediately
}

Pattern 2: Immediate Send

For WebSocket or network scenarios where data is sent immediately:

function sendMessage(ws: WebSocket, type: number, payload: string): void {
  const sia = Sia.allocUnsafe(512);
  sia.addUInt8(type).addString16(payload);
  // toUint8ArrayReference is safe here because send() copies internally
  ws.send(sia.toUint8ArrayReference());
}

WebSocket.send() copies the buffer data internally before returning. This makes toUint8ArrayReference() safe in this specific case: the reference is consumed synchronously by send().

Pattern 3: Batch Processing with Extraction

Process multiple items using the same allocUnsafe region:

function serializeBatch(users: User[]): Uint8Array[] {
  return users.map((user) => {
    const sia = Sia.allocUnsafe(256);
    sia.addUInt32(user.id).addString8(user.name).addUInt8(user.age);
    return sia.toUint8Array(); // safe: copies before next iteration
  });
}

Performance Comparison

// Sia.alloc: dedicated buffer per call
// - 2 allocations per operation (buffer + output)
// - Full isolation
// - ~3.5M ops/sec for creation

function serializeSafe(user: User): Uint8Array {
  const sia = Sia.alloc(256);
  sia.addUInt32(user.id).addString8(user.name).addUInt8(user.age);
  return sia.toUint8Array();
}

// Sia.allocUnsafe: shared buffer slice
// - 1 allocation per operation (output only)
// - Partial isolation (bounded subarray)
// - ~15M ops/sec for creation

function serializeUnsafe(user: User): Uint8Array {
  const sia = Sia.allocUnsafe(256);
  sia.addUInt32(user.id).addString8(user.name).addUInt8(user.age);
  return sia.toUint8Array();
}

// new Sia(): full shared buffer
// - 1 allocation per operation (output only)
// - No isolation (shared 32 MB buffer)
// - ~18M ops/sec for creation

const sharedSia = new Sia();

function serializeShared(user: User): Uint8Array {
  sharedSia.seek(0);
  sharedSia.addUInt32(user.id).addString8(user.name).addUInt8(user.age);
  return sharedSia.toUint8Array();
}

Summary

Pros

Zero allocation cost, reduced GC pressure, ~4x faster instance creation than alloc. Ideal for high-throughput synchronous serialization.

Cons

Uninitialized memory, buffer wraparound risk, unsafe across async boundaries. Requires disciplined usage patterns.

Alternative: new Sia()

The default constructor is even faster than allocUnsafe for single-instance reuse with seek(0). Prefer it when you can reuse one instance.

Alternative: Sia.alloc()

Use alloc when you need full isolation: async workflows, long-lived buffers, concurrent access, or security-sensitive contexts.

Edit this pageorReport an issue

Tree Shaking

Reduce bundle size by importing only the serialization functions you need.

Custom Serialization Patterns

Advanced patterns for building custom serializers, handling complex data structures, versioning, and performance optimization.