Proposal: A way for Wasm allocators to notify the host of memory growth

[castholm]

Summary

Due to how Wasm memory works in the browser and how buffers are invalidated whenever memory grows, JavaScript code needs to constantly check if array buffer views into Wasm memory have been detached via polling, which incurs overhead that might be undersirable in real-time graphics and/or games. If there was a way for WasmAllocator/WasmPageAllocator to synchronously notify the host whenever it resizes memory so that the JavaScript side can detect detached views and recreate them without polling, performance would be improved.

Background

In JavaScript, you use typed arrays like Uint32Array to read from/write into Wasm memory. One important quirk you need to be aware of and deal with is that when Wasm memory grows, arrays created from Wasm memory become detached and can no longer be used.

The easiest way to deal with this is to create short-lived arrays every time you access Wasm memory and to assume that the arrays may become detached any time you call into Wasm code again. Recreating arrays over and over every time you access memory yields lots of garbage for the GC to collect, but in non-performance-critical applications it is usually fine.

// imported by Wasm
function readInt32(ptr) {
    const value = new Int32Array(wasm.exports.memory.buffer, ptr)[0]
    console.log(value)
}

In applications like graphics or games where high performance is desired and where you want to avoid allocations, one common strategy is to have one global variable for each kind of typed array your application needs (one Int8Array, one Uint8Array and so on) and to reuse them everywhere. Several Web APIs like WebGL 2.0 have overloaded functions that take a typed array, an offset and a length specifically to support reusing arrays instead of needing to create new objects (e.g. via subarray()).

But because memory could grow at any time, we always need to check if the arrays have become detached before using them and recreate them if necessary. This doesn't generate garbage but it still incurs some minor overhead that adds up over the lifetime of the application, especially considering how rare memory growth is. (According to this comment, depending on the JS engine the overhead could be anywhere from 10% to 50% compared to a plain variable access.)

let int32View = new Int32Array(wasm.exports.memory.buffer)

function getInt32View() {
    if (int32View.buffer !== wasm.exports.memory.buffer) {
        int32View = new Int32Array(wasm.exports.memory.buffer)
    }
    return int32View
}

// imported by Wasm
function readInt32(ptr) {
    const value = getInt32View()[ptr >> 2]
    console.log(value)
}

The best option would be if the JavaScript side could be notified whenever memory growth happens and to only recreate the views there and then, so that consuming code doesn't need to poll the status of the arrays. There is an open WebAssembly issue discussing synchronous memory growth callbacks but there are some issues like re-entrancy that would need to be nailed down before tackling it and it doesn't seem to have had much traction recently.

There is a different open WebAssembly issue for integrating with the Stage 3 Resizable and Growable ArrayBuffers proposal that seems like it would enable memory growth without detaching buffers (making callbacks and recreating arrays unnecessary), but it's still only a proposal and it might take a long time before this gains support in browsers.

Proposal

Since there doesn't seem to be any plans to add synchronous memory growth callbacks to the WebAssembly Web API itself (and growable ArrayBuffers might be far away), maybe we could add support for having WasmAllocator/WasmPageAllocator call back to a user-specified function whenever @wasmMemoryGrow() is invoked?

I suggest adding a wasmAllocatorGrowthCallback option to std.options:

// std.options
pub const options = struct {
    // Called immediately after `WasmAllocator` or `WasmPageAllocator` increases the size of Wasm
    // memory.
    //
    // This is primarily meant to be used to notify the host (such as a browser running JavaScript)
    // that Wasm memory has been resized so that it can recreate detached resources if necessary.
    pub const wasmAllocatorGrowthCallback: fn (index: u32, delta: u32) void = if (@hasDecl(options_override, "wasmAllocatorGrowthCallback"))
        options_override.wasmAllocatorGrowthCallback
    else
        noOp;
};

Performance-critical web applications could then configure and use the callback like this:

extern fn onWasmMemoryGrowth(index: u32, delta: u32): void;

const std_options = struct {
    pub const wasmAllocatorGrowthCallback = onWasmMemoryGrowth;
}

let int32View = new Int32Array(wasm.exports.memory.buffer)

// imported by Wasm
function onWasmMemoryGrowth() {
    // We can ignore the parameters as we know the index is always 0 (for now).
    int32View = new Int32Array(wasm.exports.memory.buffer)
}

// imported by Wasm
function readInt32(ptr) {
    const value = int32View[ptr >> 2]
    console.log(value)
}

Alternatives

Currently, you could just implement your own custom allocator by copy/pasting the source code for one of the Wasm allocators and adding the growth callback you need yourself. This is a slight inconvenience but still fine, but it also means that you need to make sure that none of your code (including dependencies) uses standard library Wasm allocators. If you're providing some sort of framework (like Mach or zero-graphics) for people to use, your users also need to be made aware that they must use the framework's allocator in favor of the standard library ones.

[Luukdegram]

There are a few reasons why I disagree with the proposed solution:

• This 'safety' is only added for allocators within the standard library. Users may want to use other allocators for reasons, such as performance, safety, etc.
• It's trying to hide a very important detail.
• Any other allocator wanting this feature, must either re-implement it exactly or call one of the standard library's allocators also.

I believe a much more elegant solution would be to provide such a feature as a library:

const WasmSafetyAllocator = struct{
    backing_allocator: std.mem.Allocator,
    call_back: *const fn() void,

    fn init(backing_allocator: std.mem.Allocator, call_back: *const fn() void) WasmSafetyAllocator {
        return .{.backing_allocator = backing_allocator, .call_back = call_back};
    }

    pub fn allocator(allocator: *WasmLimitedAllocator) std.mem.Allocator {
        return .{
            .ptr = allocator,
            .vtable = &.{
                .alloc = alloc,
                .resize = resize,
                .free = free,
            },
        };
    }

    fn alloc(ctx: *anyopaque, len: usize, log2_ptr_align: u8, ret_addr: usize) ?[*]u8 {
        const allocator: *WasmSafetyAllocator = @ptrCast(@alignCast(ctx));
        const result = allocator.backing_allocator.rawAlloc(len, log2_ptr_align, ret_addr);
        if (result != null) allocator.call_back();
        return result;
    }
};

You can then achieve this feature while allowing any allocator by simply passing it as the backing allocator as such:

extern fn safety_callback() void;

pub fn main() void {
    var fast_allocator: SuperFastAllocator = .{};
    var safe_allocator = WasmSafetyAllocator.init(fast_allocator.allocator(), safety_callback);
    const main_allocator = safe_allocator.allocator();
}

Now we have an extensible allocator anyone can use with their favorite allocator and no silly globals that provide options to our program. It's now up to the user whether to use this allocator or not, but it's best to educate the user rather than trying to abstract away important details.

[castholm]

There are a few reasons why I disagree with the proposed solution:

• This 'safety' is only added for allocators within the standard library. Users may want to use other allocators for reasons, such as performance, safety, etc.
• It's trying to hide a very important detail.
• Any other allocator wanting this feature, must either re-implement it exactly or call one of the standard library's allocators also.

Could you elaborate on what you mean with it hiding an important detail?

The option would specifically be for the only two things in the standard library that use the @wasmMemoryGrow() builtin. It not affecting external Wasm allocators doesn't seem any stranger to me than the logFn or cryptoRandomSeed options not affecting external logging or crypto libraries that don't directly use the Zig standard library.

An alternative that would work for both stdlib and external Wasm allocators would be being able to hook into the @wasmMemoryGrow() builtin itself (like how @panic() calls builtin.panic() which in turn might delegate to root.panic()/root.os.panic()). If the memory growth callback was considered to be an OS feature and exposed via the root.os BYOOS API that could also work.

I'm suggesting exposing the callback as a static option because Wasm allocators are not parameterized like GeneralPurposeAllocator. If the callback could be passed as an option to a function returning the allocator type like with GPA it would be a cleaner design, but because the static std.heap.page_allocator (which is the default backing allocator used by GPA) uses WasmPageAllocator when targeting Wasm it would require some rethinking around those parts.

Your wrapping allocator is nice and cleaner than anything I've come up with so far, so I'll steal it for use in my projects for now. It currently calls back on every allocation instead of just allocations that resulted in memory growth, but that could be fixed by comparing the new memory size after allocating against the old size:

fn alloc(ctx: *anyopaque, len: usize, log2_ptr_align: u8, ret_addr: usize) ?[*]u8 {
    const allocator: *WasmSafetyAllocator = @ptrCast(@alignCast(ctx));
    const oldMemorySize = @wasmMemorySize(0);
    const result = allocator.backing_allocator.rawAlloc(len, log2_ptr_align, ret_addr);
    if (@wasmMemorySize(0) != oldMemorySize) allocator.call_back();
    return result;
}

[castholm]

There is a different open WebAssembly issue for integrating with the Stage 3 Resizable and Growable ArrayBuffers proposal that seems like it would enable memory growth without detaching buffers (making callbacks and recreating arrays unnecessary), but it's still only a proposal and it might take a long time before this gains support in browsers.

This proposal was just merged into the WebAssembly JavaScript Interface spec, which (to my understanding) will let the JS side obtain a resizable array buffer backed by wasm memory via memory.toResizableBuffer(), which won't get detached and invalidate views if Wasm memory grows. Support for this will likely get implemented into major browsers within a few release cycles, so I'm closing this proposal as no longer needed or relevant. Applications that need to handle detached array buffers today can either allocate all memory the application will ever need upfront or use a custom allocator as a workaround.