Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added — per-link record codecs

- **Issue #178:** one `Linkable` record type can now select JSON, bounded
Postcard, or a custom codec independently on each inbound/outbound connector
route through `linked_*_with(url, codec)` or builder-level
`with_link_codec(codec)`. Selection is fused at registration time; bounded
Postcard routes retain the reusable scratch/owned-overflow behavior from
issue #177 without a per-message registry, lock, or lookup. See
[Design 045](docs/design/045-per-link-codec-selection.md). Re-selecting a
codec replaces both the owned and scratch serializers, so changing a route
from bounded postcard to owned JSON cannot retain stale postcard output.

### Changed — Design 038 simplification pass (breaking)

Implementation of the accepted items of [design 038](docs/design/038-technical-debt-and-simplification-review.md)
Expand Down
4 changes: 4 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 12 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,13 @@ help:
build:
@printf "$(GREEN)Building AimDB (all valid combinations)...$(NC)\n"
@printf "$(YELLOW) → Building aimdb-data-contracts (std)$(NC)\n"
cargo build --package aimdb-data-contracts --features "std,simulatable,migratable,observable,linkable-json"
cargo build --package aimdb-data-contracts --features "std,simulatable,migratable,observable,linkable-json,linkable-postcard"
@printf "$(YELLOW) → Building aimdb-data-contracts (no_std)$(NC)\n"
cargo build --package aimdb-data-contracts --no-default-features --features alloc
@printf "$(YELLOW) → Building aimdb-data-contracts (no_std + format-neutral linkable)$(NC)\n"
cargo build --package aimdb-data-contracts --no-default-features --features alloc,linkable
@printf "$(YELLOW) → Building aimdb-data-contracts (no_std + linkable-postcard)$(NC)\n"
cargo build --package aimdb-data-contracts --no-default-features --features alloc,linkable-postcard
@printf "$(YELLOW) → Building aimdb-data-contracts (no_std + linkable-json + migratable)$(NC)\n"
cargo build --package aimdb-data-contracts --no-default-features --features alloc,linkable-json,migratable
@printf "$(YELLOW) → Building aimdb-core (no_std + alloc)$(NC)\n"
Expand Down Expand Up @@ -114,9 +116,11 @@ build:
test:
@printf "$(GREEN)Running all tests (valid combinations)...$(NC)\n"
@printf "$(YELLOW) → Testing aimdb-data-contracts (std)$(NC)\n"
cargo test --package aimdb-data-contracts --features "std,simulatable,migratable,observable,linkable-json"
cargo test --package aimdb-data-contracts --features "std,simulatable,migratable,observable,linkable-json,linkable-postcard"
@printf "$(YELLOW) → Testing aimdb-data-contracts (no_std + alloc + format-neutral linkable)$(NC)\n"
cargo test --package aimdb-data-contracts --no-default-features --features alloc,linkable
@printf "$(YELLOW) → Testing aimdb-data-contracts (no_std + alloc + linkable-postcard)$(NC)\n"
cargo test --package aimdb-data-contracts --no-default-features --features alloc,linkable-postcard
@printf "$(YELLOW) → Testing aimdb-data-contracts (no_std + alloc + linkable-json + migratable)$(NC)\n"
cargo test --package aimdb-data-contracts --no-default-features --features alloc,linkable-json,migratable
@printf "$(YELLOW) → Testing aimdb-core (no_std + alloc)$(NC)\n"
Expand Down Expand Up @@ -211,11 +215,13 @@ clippy:
@printf "$(YELLOW) → Clippy on aimdb-derive$(NC)\n"
cargo clippy --package aimdb-derive --all-targets -- -D warnings
@printf "$(YELLOW) → Clippy on aimdb-data-contracts (std)$(NC)\n"
cargo clippy --package aimdb-data-contracts --features "std,simulatable,migratable,observable,linkable-json" --all-targets -- -D warnings
cargo clippy --package aimdb-data-contracts --features "std,simulatable,migratable,observable,linkable-json,linkable-postcard" --all-targets -- -D warnings
@printf "$(YELLOW) → Clippy on aimdb-data-contracts (no_std + alloc)$(NC)\n"
cargo clippy --package aimdb-data-contracts --no-default-features --features alloc -- -D warnings
@printf "$(YELLOW) → Clippy on aimdb-data-contracts (no_std + alloc + format-neutral linkable)$(NC)\n"
cargo clippy --package aimdb-data-contracts --no-default-features --features alloc,linkable --all-targets -- -D warnings
@printf "$(YELLOW) → Clippy on aimdb-data-contracts (no_std + alloc + linkable-postcard)$(NC)\n"
cargo clippy --package aimdb-data-contracts --no-default-features --features alloc,linkable-postcard --all-targets -- -D warnings
@printf "$(YELLOW) → Clippy on aimdb-data-contracts (no_std + alloc + linkable-json + migratable)$(NC)\n"
cargo clippy --package aimdb-data-contracts --no-default-features --features alloc,linkable-json,migratable --all-targets -- -D warnings
@printf "$(YELLOW) → Clippy on aimdb-core (no_std + alloc)$(NC)\n"
Expand Down Expand Up @@ -295,7 +301,7 @@ doc:
@mkdir -p target/doc-final/cloud
@mkdir -p target/doc-final/embedded
@printf "$(YELLOW) → Building cloud/edge documentation$(NC)\n"
cargo doc --package aimdb-data-contracts --features "std,simulatable,migratable,observable,linkable-json" --no-deps
cargo doc --package aimdb-data-contracts --features "std,simulatable,migratable,observable,linkable-json,linkable-postcard" --no-deps
cargo doc --package aimdb-core --features "std,tracing,observability" --no-deps
cargo doc --package aimdb-tokio-adapter --features "tokio-runtime,tracing,observability" --no-deps
cargo doc --package aimdb-sync --no-deps
Expand Down Expand Up @@ -344,6 +350,8 @@ test-embedded:
cargo check --package aimdb-data-contracts --target thumbv7em-none-eabihf --target-dir $(EMBEDDED_CHECK_TARGET_DIR) --no-default-features --features alloc
@printf "$(YELLOW) → Checking aimdb-data-contracts (no_std + alloc + format-neutral linkable) on thumbv7em-none-eabihf target$(NC)\n"
cargo check --package aimdb-data-contracts --target thumbv7em-none-eabihf --target-dir $(EMBEDDED_CHECK_TARGET_DIR) --no-default-features --features alloc,linkable
@printf "$(YELLOW) → Checking aimdb-data-contracts (no_std + alloc + linkable-postcard) on thumbv7em-none-eabihf target$(NC)\n"
cargo check --package aimdb-data-contracts --target thumbv7em-none-eabihf --target-dir $(EMBEDDED_CHECK_TARGET_DIR) --no-default-features --features alloc,linkable-postcard
@printf "$(YELLOW) → Checking aimdb-data-contracts (no_std + alloc + linkable-json + migratable) on thumbv7em-none-eabihf target$(NC)\n"
cargo check --package aimdb-data-contracts --target thumbv7em-none-eabihf --target-dir $(EMBEDDED_CHECK_TARGET_DIR) --no-default-features --features alloc,linkable-json,migratable
@printf "$(YELLOW) → Checking weather-mesh-common (no_std migratable, real TemperatureV1ToV2 chain, no direct serde_json dep) on thumbv7em-none-eabihf target$(NC)\n"
Expand Down
19 changes: 18 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,13 +151,30 @@ Every capability is an opt-in trait on your schema type: implement it and **exac

| Contract | Implement when… | Verb it unlocks | Tier |
| --- | --- | --- | --- |
| [`Linkable`](https://aimdb.dev/blog/connectors-where-aimdb-meets-the-real-world) | the record is mirrored to/from an endpoint (MQTT, KNX, serial, UDS…) | `.linked_from(url)` / `.linked_to(url)` (`linkable-json` provides the JSON derive; codegen supports Postcard) | wire (prod) |
| [`Linkable`](https://aimdb.dev/blog/connectors-where-aimdb-meets-the-real-world) | the record is mirrored to/from an endpoint (MQTT, KNX, serial, UDS…) | `.linked_from(url)` / `.linked_to(url)` by default; `*_with(url, codec)` selects JSON/Postcard/custom per link | wire (prod) |
| [`Streamable`](https://aimdb.dev/blog/streamable-crossing-boundaries) | the record streams to browsers as schema-named JSON | ws-connector `.register::<T>()` | wire (prod) |
| [`Migratable`](https://aimdb.dev/blog/schema-migration-without-ceremony) | the schema evolved across versions | `migration_chain!` | wire (prod) |
| `Settable` | sync code outside AimDB sets the value | `SyncProducer::set_value(v)` | wire (prod) |
| `Observable` | the value is worth watching in production | `.observe()` → live last/min/max/mean on `record.list`/`record.get` | introspection (prod, optional) |
| `Simulatable` | the type can generate realistic synthetic data | `.simulate(profile, rng)` | **dev-only — never ships in prod** |

One record type can use different wire formats without runtime codec lookup:

```rust,ignore
use aimdb_data_contracts::{
link_codecs::{Json, Postcard},
LinkCodecRegistrarExt,
};

reg.linked_to_with("serial://mcu/temperature", Postcard::<128>);
reg.linked_to_with("mqtt://cloud/temperature", Json);
```

Enable `linkable-json` and/or `linkable-postcard`; the existing `.linked_*`
methods still use the record's `Linkable` implementation. See
[Design 045](docs/design/045-per-link-codec-selection.md) for the bounded
scratch/fallback model.

`Simulatable` is the odd one out: it lives behind the `simulatable` feature (never a default) and switching from simulated to real data is one `#[cfg]` in your app:

```rust
Expand Down
1 change: 1 addition & 0 deletions aimdb-bench/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ serde_json = { workspace = true, optional = true }
aimdb-data-contracts = { path = "../aimdb-data-contracts", default-features = false, features = [
"alloc",
"linkable",
"linkable-postcard",
"migratable",
], optional = true }

Expand Down
77 changes: 67 additions & 10 deletions aimdb-bench/benches/b0_alloc_linkable.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
//! B0-Linkable — allocation gate for the Postcard `Linkable::encode_into` path.
//! B0-Linkable — allocation gate for direct and per-link Postcard encoding.
//!
//! This deliberately measures the codec seam, not a complete connector. The
//! core pump still uses a boxed `SerializedReader` future and connector adapters
Expand All @@ -9,11 +9,12 @@ use std::hint::black_box;

use aimdb_bench::alloc::{reset, snapshot};
use aimdb_core::connector::SerializeError;
use aimdb_data_contracts::{Linkable, SchemaType};
use aimdb_data_contracts::{link_codecs, LinkCodec, Linkable, SchemaType};
use serde::{Deserialize, Serialize};

const WARMUP_ITERS: usize = 1_000;
const MEASURE_ITERS: usize = 10_000;
const ALLOCATOR_PROBE_BYTES: usize = 64;

#[global_allocator]
static GLOBAL: aimdb_bench::alloc::CountingAllocator<std::alloc::System> =
Expand Down Expand Up @@ -58,6 +59,40 @@ fn encode_batch(reading: &PostcardReading, out: &mut [u8; 256], iterations: usiz
}
}

fn encode_per_link_codec_batch(reading: &PostcardReading, out: &mut [u8; 256], iterations: usize) {
let codec = link_codecs::Postcard::<256>;
for _ in 0..iterations {
let written = codec
.encode_into(black_box(reading), black_box(out.as_mut_slice()))
.expect("per-link Postcard encode_into should fit the fixed scratch buffer");
black_box(&out[..written]);
}
}

/// Prove that this binary's global allocator is actually wired through the
/// counters. Run after the measured windows so the deliberate allocation
/// cannot contaminate either zero-allocation result.
fn allocation_counter_positive_control() -> (u64, u64) {
reset();

let mut probe = Vec::<u8>::with_capacity(black_box(ALLOCATOR_PROBE_BYTES));
probe.push(black_box(0xA5));
black_box(probe.as_slice());

let measured = snapshot();
assert!(
measured.0 >= 1,
"allocation counter missed the deliberate Vec allocation"
);
assert!(
measured.1 >= ALLOCATOR_PROBE_BYTES as u64,
"byte counter missed the deliberate Vec capacity"
);

drop(probe);
measured
}

fn main() {
let reading = PostcardReading {
value: 23.75,
Expand All @@ -68,19 +103,41 @@ fn main() {
encode_batch(&reading, &mut out, WARMUP_ITERS);
reset();
encode_batch(&reading, &mut out, MEASURE_ITERS);
let (allocations, bytes) = snapshot();
let (direct_allocations, direct_bytes) = snapshot();

encode_per_link_codec_batch(&reading, &mut out, WARMUP_ITERS);
reset();
encode_per_link_codec_batch(&reading, &mut out, MEASURE_ITERS);
let (codec_allocations, codec_bytes) = snapshot();

println!("=== B0 Linkable Postcard encode_into ===");
// This resets the global counters only after both result tuples have been
// captured, then deliberately allocates to reject a disconnected or broken
// counting allocator that would otherwise report a misleading zero.
let (probe_allocations, probe_bytes) = allocation_counter_positive_control();

println!("=== B0 Postcard encode_into ===");
println!(" Iterations : {MEASURE_ITERS}");
println!(" Allocations : {allocations}");
println!(" Bytes : {bytes}");
println!(" Allocator probe allocations : {probe_allocations}");
println!(" Allocator probe bytes : {probe_bytes}");
println!(" Direct Linkable allocations : {direct_allocations}");
println!(" Direct Linkable bytes : {direct_bytes}");
println!(" Per-link codec allocations : {codec_allocations}");
println!(" Per-link codec bytes : {codec_bytes}");

assert_eq!(
allocations, 0,
"encode_into allocated in the measured window"
direct_allocations, 0,
"direct Linkable::encode_into allocated in the measured window"
);
assert_eq!(
direct_bytes, 0,
"direct Linkable::encode_into allocated bytes in the measured window"
);
assert_eq!(
codec_allocations, 0,
"per-link Postcard codec allocated in the measured window"
);
assert_eq!(
bytes, 0,
"encode_into allocated bytes in the measured window"
codec_bytes, 0,
"per-link Postcard codec allocated bytes in the measured window"
);
}
6 changes: 6 additions & 0 deletions aimdb-bench/src/alloc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ pub struct CountingAllocator<A>(pub A);
// the only side-effect is the atomic counter updates.
unsafe impl<A: GlobalAlloc> GlobalAlloc for CountingAllocator<A> {
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
// These are independent observational counters: they publish no data
// and protect no other memory, so cross-variable ordering is irrelevant
// and Relaxed is sufficient even when allocator calls come from more
// than one thread.
ALLOC_COUNT.fetch_add(1, Ordering::Relaxed);
ALLOC_BYTES.fetch_add(layout.size() as u64, Ordering::Relaxed);
// SAFETY: caller guarantees `layout` is valid; delegated to inner.
Expand All @@ -48,6 +52,8 @@ unsafe impl<A: GlobalAlloc> GlobalAlloc for CountingAllocator<A> {
/// Reset both counters to zero.
///
/// Call once after the warmup phase, immediately before the measured window.
/// For deterministic attribution, the caller must ensure unrelated threads are
/// not allocating concurrently with `reset` or the measured window.
#[inline]
pub fn reset() {
ALLOC_COUNT.store(0, Ordering::Relaxed);
Expand Down
4 changes: 4 additions & 0 deletions aimdb-core/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
implementations, and `pump_client` keep their owned-`Vec` behavior. This
removes the codec allocation when an into-slice encoder is installed;
connector adapters may still copy the borrowed payload.
- **Scratch serializer reset hook.**
`OutboundConnectorBuilder::clear_serializer_into` lets higher-level builder
extensions replace a bounded serialization strategy with an owned-only one
without retaining the previous route-local scratch callback.
- **`RecordRegistrar::signal_gauge(name, unit) -> SignalGaugeHandle`** (design 041 §3.2) — the core hook behind `aimdb-data-contracts`'s `Observable::observe()`. Feeding values via `SignalGaugeHandle::update(f64)` folds them into per-record `SignalStats` (last/min/max/mean), surfaced through `RecordMetadata::signal_stats` on `record.list`/`record.get`. Mirrors the `with_name` precedent: always callable, an inert no-op handle when the `observability` feature is off, so callers never `#[cfg]` on core's features. New public types: `SignalGaugeHandle` (always available), `SignalStats`/`SignalGauge`/`SignalStatsInfo` (feature `observability`).

### Fixed
Expand Down
11 changes: 11 additions & 0 deletions aimdb-core/src/typed_api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -830,6 +830,17 @@ where
self
}

/// Removes a previously installed into-slice serializer.
///
/// Higher-level builder extensions use this when replacing a bounded
/// serialization strategy with an owned-only one. Clearing the fast path
/// prevents the old scratch serializer from emitting a different wire
/// format beside the replacement owned serializer.
pub fn clear_serializer_into(mut self) -> Self {
self.context_serializer_into = None;
self
}

/// Sets the operation timeout in milliseconds (the connector interprets
/// it; passed as the `timeout_ms` option — see
/// `ConnectorConfig::from_query`)
Expand Down
10 changes: 2 additions & 8 deletions aimdb-data-contracts/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- **Issue #177 — `Linkable::encode_into(&mut [u8])`.** Existing implementations
remain source-compatible through a checked `to_bytes`-and-copy default. A new
`ENCODE_BUFFER_CAPACITY` associated constant lets implementations advertise a
real allocation-free override; `linked_to` then installs the core's reusable
scratch-buffer fast path while retaining `to_bytes` for oversized values.
The new method reuses `aimdb_core::connector::SerializeError`; no additional
codec error type is introduced. The older `from_bytes`/`to_bytes` `String`
errors remain unchanged under the compatibility-first decision.
- **Issue #178 — per-link codec selection.** `LinkCodec<T>`, `LinkCodecBuilderExt::with_link_codec` and `LinkCodecRegistrarExt::{linked_from_with, linked_to_with}` let the same record type use different formats on different connector routes without a runtime registry. Built-in `link_codecs::Json` and bounded `link_codecs::Postcard<N>` markers are feature-gated by `linkable-json` and the new `linkable-postcard`; Postcard reuses the route-local scratch and owned overflow fallback added for issue #177. Existing `.linked_from` and `.linked_to` behavior is unchanged. Repeated `with_link_codec` calls use complete last-selection-wins semantics: an owned-only replacement clears a scratch encoder left by an earlier bounded codec.
- **Issue #177 — `Linkable::encode_into(&mut [u8])`.** Existing implementations remain source-compatible through a checked `to_bytes`-and-copy default. A new `ENCODE_BUFFER_CAPACITY` associated constant lets implementations advertise a real allocation-free override; `linked_to` then installs the core's reusable scratch-buffer fast path while retaining `to_bytes` for oversized values. The new method reuses `aimdb_core::connector::SerializeError`; no additional codec error type is introduced. The older `from_bytes`/`to_bytes` `String` errors remain unchanged under the compatibility-first decision.
- `SimulatableRegistrarExt::simulate(profile, rng)` — installs a `.source()` loop emitting `T::simulate(...)` on a timer.
- `ObservableRegistrarExt::observe()` — feeds `T::signal()` into a core signal gauge (last/min/max/mean), surfaced on `record.list`/`record.get`/stage profiling; `ObservableRegistrarExt::log(node_id)` for the console-logging path (replaces the old `format_log`).
- `LinkableRegistrarExt::linked_from(url)` / `linked_to(url)` — one-line `.link_from()`/`.link_to()` wiring defaulted to `T::from_bytes`/`T::to_bytes`. `linkable` now also requires `aimdb-core`.
Expand Down
Loading