arrow-core

Foundation layer of the layered Arrow stack for Pylon. Exports the Arrow type system, schema/field/array/record-batch resources, and the Arrow C Data interface boundary. Nothing else.

See docs/arrow-layered-architecture-plan.md in the python-wasm repo for the framing and docs/arrow-implementation-plan.md for the phased work breakdown.

Status

Scaffolding only. P1 Week 1 of the implementation plan: WIT contract landed, Rust crate scaffolded, all method bodies return NotImplemented. P1 Week 2 fills the bodies.

Layout

wit/
  types.wit         pure data-type variant + supporting enums + arrow-error
  schema.wit        Schema / Field / SchemaBuilder resources
  array.wit         Array resource + typed scalar accessors + bulk constructors
  record-batch.wit  RecordBatch resource
  c-data.wit        Arrow C Data interface (pointer + copy modes, P0 picks one)
  world.wit         arrow-core-component (exporter), arrow-core-import (consumer)
src/
  lib.rs            wit-bindgen entry point
  types.rs          type system conversions
  schema.rs         schema/field/builder impls
  array.rs          array impl + typed accessors
  record_batch.rs   record-batch impl
  c_data.rs         C Data interface impl
Cargo.toml          arrow-array + arrow-buffer + arrow-data + arrow-schema only

Explicit non-dependencies

This crate does not depend on:

• arrow-arith / arrow-cast / arrow-compute / arrow-ord / arrow-select / arrow-string → those live in arrow-compute
• arrow-csv / arrow-json / arrow-ipc → those live in arrow-ipc
• parquet → arrow-parquet
• tokio / async runtimes → not appropriate at this layer

Keep it that way. The whole point of the layering is that consumers can take arrow-core without inheriting the rest of the arrow-rs ecosystem.

Build

Not yet wired. scripts/build.sh is the P1 Week 2 deliverable. Once ready it will:

1. cargo build --target wasm32-wasip2 --release
2. wasm-tools component new target/wasm32-wasip2/release/arrow_core_component.wasm \ --adapt wasi_snapshot_preview1.reactor.wasm \ -o build/arrow-core.component.wasm
3. wasm-tools validate build/arrow-core.component.wasm

Versioning

WIT package: arrow:core@0.1.0. Stays at 0.1.x until the C Data ABI question is resolved and one external consumer ships against the interface. Bumps to 1.0.0 only after that consumer signs off.

License

Apache-2.0 (matches arrow-rs).