Skip to content

Add AI Capabilities to SpacetimeDB #4910

@clockwork-labs-bot

Description

@clockwork-labs-bot

This issue tracks reimplementing the work from stale PR #4589, which has been closed because it is too far out of date to merge directly.

## Original pull request
- PR #4589: [[Draft] Add AI Capabilities to SpacetimeDB](https://github.com/clockworklabs/SpacetimeDB/pull/4589)
- Original author: @definenoob

## What the original PR was trying to do
With this PR, STDB becomes the first AI-enabled database. **Seeking feedback from CWL on this**.

Summary

  • Adds native ONNX inference as a host-side capability exposed to WASM modules via the SpacetimeDB ABI (spacetime_10.5)
  • Models are loaded by name from {server_data_dir}/models/{name}.onnx on the host filesystem and cached after first use — model bytes never enter WASM memory, only input/output
    tensor data crosses the boundary
  • Uses tract-onnx (pure Rust) for inference, no C++ dependencies or WASI required

Design

Single ABI call (spacetime_10.5):

  • onnx_run(name_ptr, name_len, input_ptr, input_len, out) -> errno — loads/caches model by name, runs inference, returns output tensors via BytesSource

Crate changes:

  • spacetimedb-lib — shared Tensor type (Vec shape + Vec data) with BSATN serialization
  • spacetimedb-primitives — ONNX_ERROR errno code
  • spacetimedb-bindings-sys — raw ABI declaration + safe wrapper
  • spacetimedb-bindings — high-level OnnxClient with single run(name, inputs) method, exposed via ReducerContext::onnx and ProcedureContext::onnx
  • spacetimedb-core — host-side OnnxModel using tract-onnx, per-instance model cache (HashMap<String, OnnxModel>) in WasmInstanceEnv, path traversal validation, models_dir wired
    from HostController.data_dir

Usage from a module:

let input = vec![Tensor { shape: vec![1, 10], data: vec![0.0; 10] }];
let output = ctx.onnx.run("bot_brain", &input)?;

Test plan

  • Place a .onnx model file in {data_dir}/models/ and verify ctx.onnx.run("model_name", &input) succeeds
  • Verify inference produces correct output tensors for known model inputs
  • Verify second call to same model uses cache (no re-parse)
  • Verify invalid model names (path traversal, empty) return errors
  • Verify loading a nonexistent model returns a clear error
  • Verify compilation on both native and wasm32-unknown-unknown targets

The onnx feature flag works across all three crates:

  • spacetimedb (bindings): features = ["onnx"] — enables ctx.onnx.run(...) API
  • spacetimedb-bindings-sys: features = ["onnx"] — enables raw ABI declaration
  • spacetimedb-core: features = ["onnx"] — enables host-side inference + tract-onnx dependency

Without the feature, tract-onnx isn't compiled, no ONNX code exists in the binary, and the ABI function isn't registered.

This is ACID-compliant AI inference.

run the model inside the reducer to decide what to write. The inference result feeds directly into the transactional state change — no round-trip to an external service, no eventual consistency, just one atomic unit of "think and act."

## Closure context
- Closed from the `close-stale-pr-create-issue` stale-PR sweep.

## Reimplementation notes
- Reimplement this work on top of current `master` in a new PR.
- Keep the original PR linked as historical context and as a source of useful implementation ideas where still relevant.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions