Console based on TanStack DB

diagnostics/README.md

@@ -0,0 +1,75 @@
+# diagnostics
+
+A diagnostics crate for differential / timely dataflow programs.
+Captures live operator, channel, and arrangement state into a DD
+computation, exposes it over a WebSocket, and ships two browser
+frontends.
+
+## Wiring it into a program
+
+```rust
+use diagnostics::{logging, server::Server};
+
+timely::execute_from_args(std::env::args(), move |worker| {
+    let state = logging::register(worker, /* log_logging */ false);
+    // Worker 0 owns the WebSocket server; others drop their sink so
+    // the dataflow's input frontiers can advance.
+    let _server = if worker.index() == 0 {
+        Some(Server::start(51371, state.sink))
+    } else {
+        drop(state.sink);
+        None
+    };
+
+    // ... your computation ...
+})
+```
+
+See `examples/scc-bench.rs` for a complete instrumented program.
+
+## Viewing diagnostics
+
+The server prints a hint on startup. The two options:
+
+### Single-file (no build step)
+
+```
+cd diagnostics
+python3 -m http.server 8000
+```
+
+Open `http://localhost:8000/index.html`, click Connect.
+
+This is the path of least resistance — `index.html` is one file you
+can scp anywhere. No tooling required.
+
+### Console (React + TanStack DB)
+
+```
+cd diagnostics/console
+npm install      # first time
+npm run dev
+```
+
+Open the URL Vite prints, click Connect. The console adds incremental
+filtering, a per-frame transactional commit boundary, and a
+forward-looking architecture; see `console/README.md` for details.
+The default port `5173` of Vite means simultaneous use with the
+single-file UI on port 8000 is fine.
+
+Both UIs consume the same wire format and can connect to the same
+running server side by side.
+
+## Wire format
+
+Each WebSocket message is one `Frame` envelope:
+
+```json
+{ "type": "Frame", "ts_us": <u64>, "updates": [...] }
+```
+
+The server emits a Frame only after the dataflow's frontier has
+advanced past `ts_us`, so each Frame is a transactionally complete
+view at one closed logical timestamp. Update variants inside
+`updates` are tagged unions for `Operator`, `Channel`, and `Stat`;
+see `src/server.rs::JsonUpdate` for the schema.

diagnostics/console/.gitignore

@@ -0,0 +1,24 @@
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+lerna-debug.log*
+
+node_modules
+dist
+dist-ssr
+*.local
+
+# Editor directories and files
+.vscode/*
+!.vscode/extensions.json
+.idea
+.DS_Store
+*.suo
+*.ntvs*
+*.njsproj
+*.sln
+*.sw?

diagnostics/console/README.md

@@ -0,0 +1,54 @@
+# Diagnostics console
+
+A React frontend for the diagnostics WebSocket server defined in this
+crate. An alternative to the single-file `diagnostics/index.html`; both
+consume the same wire format (`Frame` envelopes, one per closed
+timestamp).
+
+## Running
+
+Start a diagnostics-instrumented program (e.g.
+`cargo run --release --example scc-bench -p diagnostics -- -w4`),
+then in this directory:
+
+```
+npm install
+npm run dev
+```
+
+Open the URL Vite prints, click Connect (defaults to
+`ws://localhost:51371`).
+
+For a production build:
+
+```
+npm run build
+```
+
+The output lands in `dist/` and can be served by any static file
+server.
+
+## Layout
+
+```
+src/
+  collections.ts   TanStack DB local-only collections (operators, channels, statCounters)
+  ws.ts            WebSocket bridge — applies one Frame as one transaction across collections
+  derive.ts        recursive aggregates (transitive messages, descendants, sumElapsed)
+  graph/
+    buildScopeGraph.ts  pure: derived state + scope id → renderable graph
+    layout.ts           async: chain detection + ELK layered layout
+  components/
+    Overview.tsx   sortable root-dataflow table
+    Detail.tsx     scope graph hosting + breadcrumb + filter affordances
+    Graph.tsx      effect-driven layout, JSX SVG render
+  App.tsx          top-level shell: connect form, tabs, filter, scope stack
+```
+
+## Watermark contract
+
+Each WebSocket message is one `Frame` (`{ type, ts_us, updates }`),
+emitted by the server only after the dataflow's frontier has advanced
+past `ts_us`. The bridge applies each Frame as one TanStack DB
+`createTransaction`, so every `useLiveQuery` observer sees a sequence
+of consistent closed-timestamp views — never a half-applied frame.

diagnostics/console/eslint.config.js

@@ -0,0 +1,22 @@
+import js from '@eslint/js'
+import globals from 'globals'
+import reactHooks from 'eslint-plugin-react-hooks'
+import reactRefresh from 'eslint-plugin-react-refresh'
+import tseslint from 'typescript-eslint'
+import { defineConfig, globalIgnores } from 'eslint/config'
+
+export default defineConfig([
+  globalIgnores(['dist']),
+  {
+    files: ['**/*.{ts,tsx}'],
+    extends: [
+      js.configs.recommended,
+      tseslint.configs.recommended,
+      reactHooks.configs.flat.recommended,
+      reactRefresh.configs.vite,
+    ],
+    languageOptions: {
+      globals: globals.browser,
+    },
+  },
+])

diagnostics/console/index.html

@@ -0,0 +1,13 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Differential Dataflow Diagnostics</title>
+  </head>
+  <body>
+    <div id="root"></div>
+    <script type="module" src="/src/main.tsx"></script>
+  </body>
+</html>