This issue tracks reimplementing the work from stale PR #4589, which has been closed because it is too far out of date to merge directly.
## Original pull request
- PR #4589: [[Draft] Add AI Capabilities to SpacetimeDB](https://github.com/clockworklabs/SpacetimeDB/pull/4589)
- Original author: @definenoob
## What the original PR was trying to do
With this PR, STDB becomes the first AI-enabled database. **Seeking feedback from CWL on this**.
Summary
- Adds native ONNX inference as a host-side capability exposed to WASM modules via the SpacetimeDB ABI (spacetime_10.5)
- Models are loaded by name from {server_data_dir}/models/{name}.onnx on the host filesystem and cached after first use — model bytes never enter WASM memory, only input/output
tensor data crosses the boundary
- Uses tract-onnx (pure Rust) for inference, no C++ dependencies or WASI required
Design
Single ABI call (spacetime_10.5):
- onnx_run(name_ptr, name_len, input_ptr, input_len, out) -> errno — loads/caches model by name, runs inference, returns output tensors via BytesSource
Crate changes:
- spacetimedb-lib — shared Tensor type (Vec shape + Vec data) with BSATN serialization
- spacetimedb-primitives — ONNX_ERROR errno code
- spacetimedb-bindings-sys — raw ABI declaration + safe wrapper
- spacetimedb-bindings — high-level OnnxClient with single run(name, inputs) method, exposed via ReducerContext::onnx and ProcedureContext::onnx
- spacetimedb-core — host-side OnnxModel using tract-onnx, per-instance model cache (HashMap<String, OnnxModel>) in WasmInstanceEnv, path traversal validation, models_dir wired
from HostController.data_dir
Usage from a module:
let input = vec![Tensor { shape: vec![1, 10], data: vec![0.0; 10] }];
let output = ctx.onnx.run("bot_brain", &input)?;
Test plan
- Place a .onnx model file in {data_dir}/models/ and verify ctx.onnx.run("model_name", &input) succeeds
- Verify inference produces correct output tensors for known model inputs
- Verify second call to same model uses cache (no re-parse)
- Verify invalid model names (path traversal, empty) return errors
- Verify loading a nonexistent model returns a clear error
- Verify compilation on both native and wasm32-unknown-unknown targets
The onnx feature flag works across all three crates:
- spacetimedb (bindings): features = ["onnx"] — enables ctx.onnx.run(...) API
- spacetimedb-bindings-sys: features = ["onnx"] — enables raw ABI declaration
- spacetimedb-core: features = ["onnx"] — enables host-side inference + tract-onnx dependency
Without the feature, tract-onnx isn't compiled, no ONNX code exists in the binary, and the ABI function isn't registered.
This is ACID-compliant AI inference.
run the model inside the reducer to decide what to write. The inference result feeds directly into the transactional state change — no round-trip to an external service, no eventual consistency, just one atomic unit of "think and act."
## Closure context
- Closed from the `close-stale-pr-create-issue` stale-PR sweep.
## Reimplementation notes
- Reimplement this work on top of current `master` in a new PR.
- Keep the original PR linked as historical context and as a source of useful implementation ideas where still relevant.
This issue tracks reimplementing the work from stale PR #4589, which has been closed because it is too far out of date to merge directly.
Summary
tensor data crosses the boundary
Design
Single ABI call (spacetime_10.5):
Crate changes:
from HostController.data_dir
Usage from a module:
let input = vec![Tensor { shape: vec![1, 10], data: vec![0.0; 10] }];
let output = ctx.onnx.run("bot_brain", &input)?;
Test plan
The onnx feature flag works across all three crates:
Without the feature, tract-onnx isn't compiled, no ONNX code exists in the binary, and the ABI function isn't registered.
This is ACID-compliant AI inference.
run the model inside the reducer to decide what to write. The inference result feeds directly into the transactional state change — no round-trip to an external service, no eventual consistency, just one atomic unit of "think and act."