How to pass rich data types between a TypeScript and a Rust WebAssembly module

by Arend van Beelen on Feb 18, 2022

Ever wondered how to pass rich data types such as structs between a TypeScript and Rust WebAssembly module? Our Principal Software Engineer Arend van Beelen has all the insights.

For starters, you need to be able to pass fat pointers between the TypeScript and Rust WebAssembly module.

A fat pointer can be a u64 that indicates both the 32-bit offset where the data starts as well as the total length of the data. In TypeScript, it can be represented using a BigInt.

#[doc(hidden)]
pub fn to_fat_ptr(ptr: *const u8, len: u32) -> FatPtr {
    (ptr as FatPtr) << 32 | (len as FatPtr)
}

#[doc(hidden)]
pub fn from_fat_ptr(ptr: FatPtr) -> (*const u8, u32) {
    ((ptr >> 32) as *const u8, (ptr & 0xffffffff) as u32)
}

Unfortunately, you can’t point this fat pointer directly to your data in Rust. That causes trouble with vectors and other indirection. You want to serialize your data with something like msgpack.org, and then pass a pointer to the serialized data. That works great going from Rust to TypeScript, but what about the other way around?

function fromFatPtr(fatPtr: FatPtr): [ptr: number, len: number] {
  return [
    Number.parseInt((fatPtr >> 32n).toString()),
    Number.parseInt((fatPtr & 0xffff_ffffn).toString()),
  ];
}

function toFatPtr(ptr: number, len: number): FatPtr {
  return (BigInt(ptr) << 32n) | BigInt(len);
}

The fat pointer can only point to memory belonging to the WebAssembly module, but when we pass from TypeScript to Rust, how does TypeScript know which memory to use?

#[no_mangle]
pub fn __fp_malloc(len: u32) -> FatPtr {
    let ptr = unsafe {
        std::alloc::alloc(
            Layout::from_size_align(len as usize, MALLOC_ALIGNMENT)
                .expect("Allocation failed unexpectedly, check requested allocation size"),
        )
    };
    to_fat_ptr(ptr, len)
}

To solve this, the WebAssembly needs to expose memory allocation functions, such as malloc() and free(). These are well-known from the world of C, but unlike their C counterparts, these will work with fat pointers.

#[doc(hidden)]
pub unsafe fn import_value_from_host<'de, T: Deserialize<'de>>(fat_ptr: FatPtr) -> T {
    let (ptr, len) = from_fat_ptr(fat_ptr);
    if len & 0xff000000 != 0 {
        panic!("Unknown extension bits");
    }

    let slice = std::slice::from_raw_parts(ptr, len as usize);
    let mut deserializer = Deserializer::new(slice).with_human_readable();
    let value = T::deserialize(&mut deserializer).unwrap();

    __fp_free(fat_ptr);

    value
}

Now, whenever TypeScript needs to pass complex data to the Rust WebAssembly, it:serializes the DataSource.calls malloc() to get the memorycopies the datapasses the fat pointer to a Rust functionAnd finally, someone calls free().Found this interesting?For more updates follow us on Twitter.