diff --git a/.claude/skills/implement-resp-command/SKILL.md b/.claude/skills/implement-resp-command/SKILL.md new file mode 100644 index 000000000..89558763f --- /dev/null +++ b/.claude/skills/implement-resp-command/SKILL.md @@ -0,0 +1,116 @@ +--- +name: implement-resp-command +description: Add a new Redis/RESP command (or overload) to StackExchange.Redis end-to-end — enum, interfaces, RedisDatabase implementation, ResultProcessor, public-API tracking, and the ResultProcessor + RoundTrip unit tests. Use when asked to "add/implement/support a Redis command", wire up a new RESP command, expose a server feature on IDatabase/IDatabaseAsync, or add a result processor. +--- + +# Implement a new RESP command + +This walks through adding a command to **StackExchange.Redis** (the `src/StackExchange.Redis` client). Read `AGENTS.md` first — especially **Public API tracking → Backwards compatibility is paramount** and **Architecture**. Do every step; the build and the API analyzer will fail loudly if you skip the wiring, but the *tests* are what prove the command actually works. + +Use an existing, similarly-shaped command as your template (e.g. `StringGet`/`GET` for a simple key command, `StreamAutoClaim`/`XAUTOCLAIM` for a structured aggregate reply). Grep `RedisDatabase.cs` for one and mirror it. + +## Source the command's spec first + +Before writing anything, get the command's exact argument order and reply shape — you need it for the `Message` (request bytes) and the `ResultProcessor` (reply parsing), and the round-trip test asserts both precisely. + +- **Existing / released commands** are described in two authoritative places (substitute the command name, lower-case): + - **Server source, JSON spec** — e.g. `https://github.com/redis/redis/blob/unstable/src/commands/xdelex.json`. This is the most precise: argument tokens/order/optionality, `arity`, key specs, and the **`write`/`readonly` command flags** (which directly tell you the `IsPrimaryOnly` classification) plus, often, a `reply_schema`. + - **HTML docs** — e.g. `https://redis.io/docs/latest/commands/xdelex/`. More readable, with reply examples. + - (For non-Redis targets the equivalents are the Valkey/Garnet/etc. source and docs — but the wire command is usually identical.) +- **Module commands** (RediSearch `FT.*`, RedisJSON `JSON.*`, RedisTimeSeries `TS.*`, RedisBloom, …) live in each module's own repo, usually as a single aggregated `commands.json` (e.g. RediSearch: `https://github.com/RediSearch/RediSearch/blob/master/commands.json`) rather than core Redis's one-file-per-command layout. Use it the same way for argument/reply shape. **But module commands are generally handled by separate companion libraries (e.g. [NRedisStack](https://github.com/redis/NRedisStack)), not core StackExchange.Redis** — so usually you won't add them here at all; ad-hoc use goes through the generic `Execute`/`ExecuteAsync(string command, …)` → `RedisResult` API. If you *do* wire one as first-class, note the wire token is dotted (`FT.SEARCH`) and a C# enum member name can't contain `.`; the token for a member whose name isn't a valid identifier is supplied via the `[AsciiHash("FT.SEARCH")]` override — see `eng/StackExchange.Redis.Build/AsciiHash.md`. Confirm that a first-class typed binding is actually intended before following the enum steps below. +- **New / unreleased commands** may not be in either yet. In that case **ask the user for the spec** — the exact argument order and a concrete sample request/reply (RESP bytes if possible) — rather than guessing; the round-trip and ResultProcessor tests are only as correct as that sample. +- **RESP2 vs RESP3:** the reply (and occasionally argument handling) can differ subtly between protocols — e.g. a map/`%` vs a flat `*` array, a double/`,` vs a bulk-string number, or added attributes. The JSON `reply_schema` sometimes distinguishes them. Capture **both** forms and handle them in the `ResultProcessor` (and cover both in the unit tests). + +## Steps + +1. **Add the command name to the `RedisCommand` enum** — `src/StackExchange.Redis/Enums/RedisCommand.cs`. The enum member name *is* the wire token (`CommandMap` serializes it via `command.ToString()`), so name it exactly as Redis expects (e.g. `GETEX`, `XAUTOCLAIM`). Keep the existing alphabetical grouping. + - **Then classify it in `IsPrimaryOnly`** (the `switch` in the same file). That switch is **exhaustive** — its `default` *throws* `ArgumentOutOfRangeException` (*"Every RedisCommand must be defined in Message.IsPrimaryOnly…"*) at runtime for any unlisted command, so this is not optional. Put writes/mutations in the primary-only list; pure reads fall through to the replica-eligible branch. Getting it wrong mis-routes the command (e.g. a write sent to a replica). + +2. **Declare the method on the interfaces** — `src/StackExchange.Redis/Interfaces/IDatabase.cs` *and* `IDatabaseAsync.cs` (or the `.Arrays.cs` / `.VectorSets.cs` partials when relevant). Always provide both sync and async. + - **Back-compat:** never add an optional parameter to an existing shipped method (binary break → `MissingMethodException`). Add a new **overload** instead; see `AGENTS.md`. + - **Implement the new member on every `IDatabase`/`IDatabaseAsync` implementor**, or the build breaks. Chiefly `KeyspaceIsolation/KeyPrefixedDatabase.cs` — and there it must prefix keys via `ToInner(key)`; a stub that forwards without prefixing compiles but **silently breaks keyspace isolation** for the new command. If the command should also be usable in batches/transactions, add it to `IBatch`/`ITransaction` and their implementations (`RedisBatch`/`RedisTransaction`/`KeyPrefixedBatch`) too. + +3. **Implement in `RedisDatabase.cs`** (next to the template you picked). The standard shape: + ```csharp + public RedisValue StringGet(RedisKey key, CommandFlags flags = CommandFlags.None) + { + var msg = Message.Create(Database, flags, RedisCommand.GET, key); + return ExecuteSync(msg, ResultProcessor.RedisValue); + } + public Task StringGetAsync(RedisKey key, CommandFlags flags = CommandFlags.None) + { + var msg = Message.Create(Database, flags, RedisCommand.GET, key); + return ExecuteAsync(msg, ResultProcessor.RedisValue); + } + ``` + For argument shapes `Message.Create` doesn't cover (optional tokens, variadic args, multiple round-trips), write a private `Message` subclass overriding `WriteImpl` (search `RedisDatabase.cs` for `: Message` and `GetStringGetExMessage` for examples), or an `IMultiMessage`. + +4. **Pick or write the `ResultProcessor`** — `src/StackExchange.Redis/ResultProcessor.cs`. Reuse an existing one if the reply shape matches (`RedisValue`, `RedisValueArray`, `Int64`, `Boolean`, `Lease`, …). Otherwise add a nested `internal sealed class XProcessor : ResultProcessor` overriding `SetResult(PhysicalConnection, Message, ref RespReader)` to parse the reply with the `RespReader`, and expose it as a `public static readonly` field. Handle RESP2 vs RESP3 and older-server reply variants here. + +5. **New result types** go in `src/StackExchange.Redis/APITypes/` (mirror `StreamAutoClaimResult` etc.). + +6. **Update public-API tracking** — add every new public member to `PublicAPI.Unshipped.txt` (and the `net6.0/` subfolder if the API only exists on newer TFMs). The build error tells you the exact line. See `AGENTS.md`. + +7. **Write the two unit-test layers** (below). These run with **no external server**, so they're the fast, reliable proof of correctness — write them even if you also add live integration tests. + +8. **Gate pre-release server features** behind `[Experimental(Experiments.Server_8_x)]` when appropriate (see `src/RESPite/Shared/Experiments.cs`). + +## Tests — the two layers that matter + +### ResultProcessor unit test (parsing in isolation) + +Proves your `ResultProcessor` turns raw RESP bytes into the right typed value. Add a class under `tests/StackExchange.Redis.Tests/ResultProcessorUnitTests/` deriving `ResultProcessorUnitTest`; feed handcrafted RESP wire strings to `Execute(resp, ResultProcessor.X)` and assert on the result; use `ExecuteUnexpected(resp, ...)` for replies that must fail. Model it on `ResultProcessorUnitTests/StreamAutoClaim.cs`: + +```csharp +public class MyCommand(ITestOutputHelper log) : ResultProcessorUnitTest(log) +{ + [Fact] + public void Basic_Success() + { + var resp = "*2\r\n$3\r\n0-0\r\n*0\r\n"; // hand-built RESP reply + var result = Execute(resp, ResultProcessor.MyCommand); + Assert.Equal("0-0", result.NextStartId.ToString()); + } + + [Fact] + public void WrongShape_Failure() => ExecuteUnexpected("$5\r\nhello\r\n", ResultProcessor.MyCommand); +} +``` +Cover the cases that actually bite: RESP2 **and** RESP3 forms, empty arrays, null (`$-1`/`*-1`), older-server reply shapes (e.g. a 2-element vs 3-element reply across versions), and at least one malformed reply via `ExecuteUnexpected`. + +### RoundTrip unit test (full write + read, still no server) + +Proves the command **serializes to the exact bytes** Redis expects *and* parses back correctly, exercising `Message.WriteTo` + the command-map. Add to `tests/StackExchange.Redis.Tests/RoundTripUnitTests/` using `TestConnection.ExecuteAsync(message, processor, requestResp, responseResp, ...)`, which asserts the outbound RESP equals `requestResp` and then feeds `responseResp` back through the processor. See `RoundTripUnitTests/AdhocMessageRoundTrip.cs`: + +```csharp +[Theory(Timeout = 1000)] +[InlineData("hello", "*2\r\n$4\r\nECHO\r\n$5\r\nhello\r\n")] +public async Task MyCommand_RoundTrips(string payload, string requestResp) +{ + var msg = /* build the Message exactly as RedisDatabase does */; + var result = await TestConnection.ExecuteAsync(msg, ResultProcessor.MyCommand, requestResp, ":5\r\n", log: log); + Assert.Equal(5, result.AsInt32()); +} +``` +Verify the precise outbound bytes (length prefixes included), and ideally that command-map **rename** and **disable** behave (the `MapMode` pattern in that file). + +### Optional: live integration test + +Only if you need to prove behavior against a real server — these need the docker Redis topology (see `AGENTS.md → Testing topology`). An **absent** server is skipped automatically by the test infrastructure, so you don't write code for that. + +What you *do* need to handle for a new command is **server version**: most new commands are new server features, and the test must skip as inconclusive on servers too old to support them. Use the `require:` parameter when creating the connection — it connects and auto-skips when the live server is below the threshold: + +```csharp +await using var conn = Create(require: RedisFeatures.v7_4_0_rc1); +var db = conn.GetDatabase(); +// ... exercise the command ... +``` +Pick the `RedisFeatures.vX_Y_Z` constant matching the version that introduced the command (see `HashFieldTests.cs` / `CopyTests.cs` for the pattern). If your command needs a version threshold that doesn't exist yet, add the constant to `RedisFeatures`. This keeps the suite green across the range of server versions CI and contributors run against. + +The in-process managed server (`toys/StackExchange.Redis.Server`) may also need a handler if integration tests run against it. + +## Before finishing + +- `dotnet build Build.csproj -c Release /p:CI=true` — analyzers + `TreatWarningsAsErrors` must pass (this catches a missing `PublicAPI.Unshipped.txt` entry). +- `dotnet test tests/StackExchange.Redis.Tests/StackExchange.Redis.Tests.csproj -f net10.0 --filter "FullyQualifiedName~MyCommand"` — runs your new unit tests without any server. +- Double-check no shipped signature changed (back-compat). diff --git a/.claude/skills/summarize-database/SKILL.md b/.claude/skills/summarize-database/SKILL.md new file mode 100644 index 000000000..9727f7e06 --- /dev/null +++ b/.claude/skills/summarize-database/SKILL.md @@ -0,0 +1,111 @@ +--- +name: summarize-database +description: Profile/summarize a live Redis (RESP) database by sampling — discover key patterns, where data lives by count and by size, and what the values actually are (counters, JSON, XML, blobs, text). Use when asked to summarize, profile, analyze, audit, or characterize a Redis/Valkey/RESP database or its keyspace, understand key naming patterns, or find where memory/data is concentrated. +--- + +# Summarize a database + +Builds a statistical profile of a **live, often production** RESP database by sampling a fraction of the keyspace. The output answers three questions: + +- **(a) Key patterns** — tokenize keys so `users:1243:orders` and `users:543:orders` collapse to `users:{id}:orders`, and report the patterns by volume. +- **(b) Where the data is** — which patterns/types hold the bulk, reported **separately by count and by size** (they often disagree: many tiny keys vs. a few huge ones). +- **(c) What the values are** — numeric counters, JSON, XML, base64/binary blobs, or plain text. + +This is a **read-only, sampling** analysis. You are pointed at someone's real data; correctness and not harming the server matter more than completeness. + +## Hard safety rules (do not violate) + +- **Read-only only.** Never run a command that writes, deletes, expires, or reconfigures. No `FLUSHALL`/`FLUSHDB`, `DEL`, `SET`, `CONFIG SET`, `DEBUG RELOAD`, etc. +- **Never `KEYS`** (or `KEYS *`) — it blocks the server for the full scan on a large keyspace. Sample with `RANDOMKEY`; only use `SCAN` (cursored, with `COUNT`) if you deliberately need coverage, never `KEYS`. +- **Prefer a replica for the analysis load.** Run `INFO replication` to read the topology, then: + - If connected to a **primary that has replicas** (`role:master`, `connected_slaves > 0`, with `slaveN:ip=…,port=…` lines), **suggest the user re-point at one of those replicas** — `SCAN`/`MEMORY USAGE`/value reads then land on a replica instead of the production primary. + - If **already on a replica** (`role:slave`/`replica`), **stay there — do *not* suggest switching to its primary/master**, even though `master_host`/`master_port` are visible. Moving "up" to the primary would push the load onto production, the opposite of the goal. (A replica may lag slightly, so note the profile reflects near-current, not necessarily live, state.) +- **Never blindly read a value of unknown size.** Check the size command first; peek with `GETRANGE key 0 N`, never a bare `GET` on an unknown key. +- **Bound the expensive calls** — `MEMORY USAGE key SAMPLES `, and respect the sample cap below. +- **Confirm the target before connecting** — echo back host/port/db (and whether it's a replica) so the user can stop you hitting the wrong server. Also confirm you're authorized to query this (possibly production) system at all, not just that the address is right. +- **Value contents are confidential — get explicit consent before reading them.** Key *names* are read by necessity — they're the basis of the pattern analysis, so they're treated as probative and acceptable to surface. **Values are not:** the content-classification phase (step 4) pulls real data into the agent's context, which leaves the machine (sent to the model, possibly logged) and may contain PII, secrets, or tokens. Ask before running step 4. If the user declines, produce a **structure-only** summary (steps 0–3 + 5: patterns, types, sizes, TTL) and skip value sampling. Even with consent, don't sample obviously-sensitive namespaces (keys matching `*token*`/`*secret*`/`*password*`/`*session*`) unless the user opts in for those specifically. + +## Tooling + +Prefer `resp-cli` (the user's tool; transparent about the wire protocol); fall back to `redis-cli` if `resp-cli` is missing or lacks a command — and say so rather than silently switching. Key facts: + +- `resp-cli [cmd args] -h -p [-a ] [-n ] [--tls] [-3]` runs **one** command per process. `-r N` repeats a command N times — so `resp-cli -r 200 randomkey` is a clean random sampler. +- `resp-cli` has **no** `--scan`/`--bigkeys`/`--memkeys` mode. `redis-cli` does, and supports stdin **pipelining**, which matters for the probe phase (see Performance). +- **Never put the password on the command line.** `-a ` leaks via the process list (`ps`, visible to other local users), shell history, **and this transcript**. Use the auth env vars instead — `RESPCLI_AUTH` (resp-cli) / `REDISCLI_AUTH` (redis-cli) — and `--user ` for an ACL username. Use `--tls` for in-transit encryption; never disable certificate validation (`--insecure`/`--trust`) just to make a connection succeed. + +## Server compatibility (Garnet & other non-Redis RESP servers) + +The target may be a RESP-compatible server rather than stock Redis (this library explicitly supports Garnet, Valkey, etc.). **Garnet in particular advertises a Redis version via `HELLO` (e.g. `redis_version:7.4.3`) but exposes a `garnet_version` field and does *not* implement every command.** Detect the real server early with `HELLO`, and don't trust the `redis_version` string alone — `redis_mode`/version can look like Redis while commands are missing. + +Observed on Garnet 2.0.1 (adapt rather than fail): + +- **`RANDOMKEY` is unavailable** (`ERR unknown command`) — so the default sampler doesn't work. Fall back to **`SCAN`** (cursored, `COUNT`) for sampling. For a small keyspace, a **full `SCAN` enumeration is exact, cheap, and better than sampling** — do that and report 100% coverage instead of extrapolating. On a **large** keyspace where `RANDOMKEY` is missing *and* full enumeration is impractical there is no cheap unbiased sampler: a partial `SCAN` returns keys in hash-bucket order, so stopping early **biases** the sample. Either run `SCAN` across the full cursor (capping the per-key *probing*, not the scan itself) and sample from what you collect, or accept a partial-`SCAN` sample and **state the bias** in the report. +- **`OBJECT ENCODING` (and `OBJECT HELP`) are unavailable** — you can't report internal encoding; omit that column and note it as unavailable. +- **`INFO keyspace` may be empty** even when `DBSIZE` > 0 — rely on `DBSIZE` for the key count. +- Still available and used as normal: `SCAN`, `TYPE`, `MEMORY USAGE`, `TTL`, the O(1) size commands (`HLEN`/`SCARD`/…), and `HGETALL`/`HRANDFIELD`/`LRANGE`/etc. for content. + +The same "detect, then degrade gracefully and say which probes were unavailable" approach applies to any RESP server that under-implements the command set. + +## Procedure + +### 0. Scope & connect + +`HELLO` (identify the real server — watch for `garnet_version` or other non-Redis markers; see *Server compatibility*), then `INFO server`, `INFO keyspace`, `INFO memory`, `INFO replication`, and `DBSIZE`. Note the server/RESP version, total key count, used memory, and whether a replica is available. + +**Cluster:** if `INFO`/`CLUSTER INFO` shows cluster mode, `DBSIZE`/`RANDOMKEY`/`SCAN` are **per-node**. Either iterate each master node (sample and sum), or clearly state the summary is for the single node you connected to. Don't present one node's numbers as the whole cluster. + +### 1. Choose the sample size + +Default sample = **min(5% of `DBSIZE`, cap)**, where the cap defaults to **~2,000 keys**. Both the **fraction and the cap are user-overridable** — accept e.g. "sample 10%" or "up to 20000 keys" or "sample exactly 500". Report the actual sample size and what fraction of the DB it represents, since all extrapolations depend on it. + +Draw keys with `resp-cli -r randomkey`. `RANDOMKEY` samples **with replacement**, so de-duplicate the drawn keys (and note that a high duplicate rate implies a small or skewed keyspace). + +### 2. Probe each sampled key + +For each distinct sampled key collect: + +- `TYPE key` — string / hash / list / set / zset / stream. +- `OBJECT ENCODING key` — internal representation (`int`, `embstr`, `raw`, `listpack`, `intset`, `hashtable`, `skiplist`, `quicklist`, `stream`, …); reveals small-vs-large internal layout. +- **Element size**, O(1) per type: `STRLEN` (string), `HLEN` (hash), `LLEN` (list), `SCARD` (set), `ZCARD` (zset), `XLEN` (stream). +- **Memory**: `MEMORY USAGE key SAMPLES 5` — true bytes incl. overhead (additive to the element counts; the two answer different questions). Fall back gracefully if the command is disabled. +- `TTL key` — to report the volatile vs. persistent split. + +**Performance:** one `resp-cli` process per command is fine for small samples but slow for thousands. When the sample is large, build the probe commands and **pipeline them through `redis-cli`** (`printf '...\n' | redis-cli ...`), keeping `resp-cli` for the scoping/interactive steps. The sample cap is what keeps this bounded — honor it. + +### 3. Tokenize key patterns + +Split each key into segments on `:` (Redis convention) and `/`. Replace **variable** segments with a typed placeholder, then group keys by their tokenized pattern and count members. + +Collapse a segment when it looks like an identifier. In rough priority (most likely first), but cast a **wider net than just these**: + +- integers → `{int}` +- UUIDs / GUIDs → `{uuid}` +- hex strings (e.g. ≥8 hex chars) → `{hex}` +- ULIDs / base62 / base64-ish tokens, long random-looking strings → `{id}` +- epoch timestamps / date-like segments → `{ts}` / `{date}` +- **high-cardinality fallback**: a segment position that takes many distinct values across the sample (relative to its siblings) is almost certainly an id even if it doesn't match a shape above — collapse it to `{var}`. + +Keep low-cardinality, stable segments literal (`users`, `orders`, `cache`, `session`). Report patterns with estimated full-DB counts (sample count ÷ sample fraction) and flag that they're estimates. + +### 4. Classify content + +**This step reads real values — only run it with user consent (see Hard safety rules). Without consent, skip it and deliver a structure-only summary.** + +- **Strings**: use `STRLEN`, then `GETRANGE key 0 200` to peek (never a bare `GET` on a large/unknown value). Classify: integer counter (`^-?\d+$` and/or `OBJECT ENCODING` = `int`), JSON (`{`/`[` and parses), XML (`<…>`), base64/binary blob, or plain text. Note approximate value-size distribution. +- **Aggregates**: sample a few members rather than reading the whole structure. **Avoid `HGETALL`/`SMEMBERS`/`LRANGE key 0 -1` on unknown-size aggregates** — they're unbounded, so they're slow on large keys and pull far more data into context than needed. Size-gate first (`HLEN`/`SCARD`/`LLEN`), then use the random/ranged samplers: `HRANDFIELD key 5 WITHVALUES`, `SRANDMEMBER key 5`, `ZRANDMEMBER key 5 WITHSCORES`, `LRANGE key 0 4`, `XRANGE key - + COUNT 5` — and classify field names / element contents the same way. + +### 5. Report + +Produce a concise summary: + +- **Connection & scope**: target (host/port/db, replica?), `DBSIZE`, used memory, RESP version, cluster note; sample size and fraction. +- **Key patterns**: table of tokenized patterns → estimated count, dominant type, dominant encoding, example raw key. +- **Where the data is**: top patterns/types **by count** and, separately, **by estimated total size** (per-key `MEMORY USAGE` × extrapolation). Call out the count-vs-size divergence. +- **Content**: per major pattern, what the values are (counter / JSON / XML / blob / text) with an example shape (redact obvious secrets/PII in examples). +- **Caveats**: sampling error, with-replacement duplicates, per-node vs. cluster-wide, any commands that were unavailable. + +## Notes + +- Everything here is an **estimate** from a sample — say so, and give the sample fraction alongside extrapolated numbers. +- `redis-cli --bigkeys` / `--memkeys` are complementary built-ins for the size question, but they don't do pattern tokenization or content classification — this skill's value is the patterns + content, so use the built-ins only as a cross-check. +- See `AGENTS.md` for the `resp-cli`-over-`redis-cli` preference and the broader project context. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 000000000..18b82e8ba --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,6 @@ + + +## Checklist + +- [ ] I fully and freely contribute this code in accordance with the [project license](../LICENSE) (and am legally able to do so) +- [ ] I take responsibility for this contribution's quality and correctness, including any portions produced with AI assistance (see [CONTRIBUTING.md](../CONTRIBUTING.md)). diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..32eb43267 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,114 @@ +# AGENTS.md + +Guidance for AI coding agents working in this repository. Claude Code reads this via a `CLAUDE.md` that imports it. + +## What this is + +StackExchange.Redis — a high-performance .NET client for RESP servers (Redis, Valkey, Garnet, Azure Managed Redis, AWS ElastiCache, etc.). This is the **v3** line (`VersionPrefix` 3.0, current branch work on `marc/agents`), whose defining change is that the low-level IO core has been extracted into a separate **RESPite** library that StackExchange.Redis now sits on top of. + +## Solution layout + +- `src/StackExchange.Redis/` — the client library (the NuGet package). Public surface: `ConnectionMultiplexer`, `IDatabase`/`IDatabaseAsync`, `IServer`, `ISubscriber`, `ITransaction`/`IBatch`, `ConfigurationOptions`, `RedisValue`/`RedisKey`/`RedisResult`. +- `src/RESPite/` — standalone low-level RESP protocol library (separate package, `Marc Gravell` copyright). Owns wire-level parsing/writing: `RespReader`, `RespFrameScanner`, `RespPrefix`, buffer pooling (`CycleBuffer`, `MemoryTrackedPool`). StackExchange.Redis depends on it via `ProjectReference`. RESPite has no dependency on StackExchange.Redis. +- `eng/StackExchange.Redis.Build/` — a Roslyn analyzer/source-generator project that every other project references `OutputItemType="Analyzer"`. It generates code (e.g. `AsciiHashGenerator`) and enforces project-specific rules. Not shipped. +- `tests/` — `StackExchange.Redis.Tests` (xUnit v3, the main integration suite), `RESPite.Tests`, `*.Benchmarks` (BenchmarkDotNet), and `RedisConfigs` (server configs + docker compose, see Testing). +- `toys/` — runnable samples and an in-process RESP server (`StackExchange.Redis.Server`, used by tests as a managed fake server), Kestrel-hosted server, console tools. +- `docs/` — published documentation site source (markdown). `docs/ReleaseNotes.md` is the changelog (frozen at 3.0 — from v3 onward release notes live in GitHub Releases). + +`Build.csproj` is a `Microsoft.Build.Traversal` project that references everything under `eng/src/tests/toys` — build/test/pack it to act on the whole repo. `StackExchange.Redis.slnx` is the IDE solution (XML SLNX format, not classic `.sln`). + +## Build, test, pack + +```bash +# Build everything (CI uses Release) +dotnet build Build.csproj -c Release /p:CI=true + +# Full local build + start servers + run tests (Windows-oriented, also build.cmd) +pwsh ./build.ps1 -StartServers + +# Start the Redis test servers in the background (preferred; see "Testing topology") +docker compose --file tests/RedisConfigs/docker-compose.yml up -d --wait + +# Run the main test suite against one target framework (fastest inner loop) +dotnet test tests/StackExchange.Redis.Tests/StackExchange.Redis.Tests.csproj -c Release -f net10.0 + +# Run a single test class / method (xUnit v3 + Microsoft.Testing.Platform) +dotnet test tests/StackExchange.Redis.Tests/StackExchange.Redis.Tests.csproj -f net10.0 --filter "FullyQualifiedName~ClassName.MethodName" + +# Pack the library +dotnet pack src/StackExchange.Redis/StackExchange.Redis.csproj --no-build -c Release /p:Packing=true +``` + +- SDK is pinned (`global.json`, `allowPrerelease: false`); CI installs the 6/8/10 runtimes. `LangVersion` is 14. +- `TreatWarningsAsErrors=true` everywhere and `Features=strict` — warnings fail the build. Analyzers (StyleCop + the custom `eng` analyzer + PublicApiAnalyzers) run as part of the build. +- Library multi-targets `net461;netstandard2.0;net472;net6.0;net8.0;net10.0`. Conditional compile symbols: `VECTOR_SAFE` (all but net461), `UNIX_SOCKET` (net6.0+). The test project targets `net481;net8.0;net10.0`; `BUILD_CURRENT` is defined on the newest TFM (disables some parallelism for brittle tests). + +## Public API tracking (important — easy to trip over) + +Both shipped libraries use `Microsoft.CodeAnalysis.PublicApiAnalyzers`. Any change to the public surface fails the build until you update the API text files: + +- `src/StackExchange.Redis/PublicAPI/PublicAPI.{Shipped,Unshipped}.txt` (and the `net6.0/` subfolder for APIs that only exist on newer TFMs — each folder is effectively `NET_X_Y_OR_GREATER`). +- `src/RESPite/PublicAPI/...` likewise (with `net8.0/`). + +Add new members to `PublicAPI.Unshipped.txt`. The build error message tells you the exact line to add. + +### Backwards compatibility is paramount + +This library is heavily used and referenced across the .NET ecosystem, so **hard breaks to shipped public APIs are extremely discouraged** — especially binary breaks that surface as `MissingMethodException`/`MissingFieldException` at runtime for callers compiled against an older version. Note that source-compatible changes can still be binary breaks: **adding an optional parameter to an existing method changes its signature** and breaks already-compiled callers, so do not do it. The same applies to changing parameter/return types, renaming members, or removing them. + +Prefer additive, non-breaking patterns instead: +- **Add a new overload** rather than modifying an existing method's signature; use `[OverloadResolutionPriority(...)]` to steer the compiler toward the preferred overload when several would otherwise be ambiguous. +- **Deprecate, don't delete**: mark the old member `[Obsolete(...)]` (keeping it functional) and point callers at the replacement. +- When unsure whether a change is breaking, treat it as breaking and reach for an overload — or raise it for human review. + +## Experimental APIs + +Newer features (especially pre-release server APIs) are typically gated behind `[Experimental(...)]` diagnostic IDs defined in `src/RESPite/Shared/Experiments.cs` (`SER001`–`SER006`, e.g. `Respite = "SER004"`, version-gated server features `Server_8_4/8_6/8_8`). These IDs are in the root `NoWarn` list so consuming them internally doesn't error; docs live under `docs/exp/`. + +## Architecture (the big picture) + +Request flow, roughly outer → inner: + +1. **`ConnectionMultiplexer`** (split across many `ConnectionMultiplexer.*.cs` partials) is the root object — one per logical Redis deployment, meant to be shared/long-lived. It owns endpoints, configuration, pub/sub, sentinel logic, and server selection. +2. **`IDatabase` / `RedisDatabase`** (`RedisDatabase.cs`, ~6k lines) is the command surface. Each command builds a **`Message`** and hands it to the multiplexer with a **`ResultProcessor`** that knows how to parse the reply into the typed result. `Message.cs` and `ResultProcessor.cs` are the two hubs to understand command implementation — to add/modify a command, you create the message + pick/extend a result processor. +3. **`ServerEndPoint`** represents one physical server; **`PhysicalBridge`** manages the queue/backlog and connection lifecycle for a server; **`PhysicalConnection`** is the actual socket + read/write loop. This is where pipelining and the backlog policy live (see `docs/PipelinesMultiplexers.md`). +4. **RESPite** does the byte-level RESP framing beneath `PhysicalConnection` — scanning frames off the buffer (`RespFrameScanner`), reading values (`RespReader`, a `ref struct` with many `.cs` partials), and pooled buffers. + +Cross-cutting: `CommandMap` (command renaming/disabling per server type), `ServerType`/cluster slot routing (`ClusterConfiguration`, `ServerSelectionStrategy`), `CommandFlags` (sync/async, fire-and-forget, replica preference), keyspace isolation (`KeyspaceIsolation/`), profiling (`Profiling/`), and maintenance events (`Maintenance/`). RESP3 push/attribute support is reflected in both the reader and result processors. + +Partial-class file naming is heavily used: `Foo.cs` + `Foo.Bar.cs` are one type (the csproj wires `DependentUpon`). When editing a type, check for sibling `Foo.*.cs` files. + +## Testing topology + +Many tests are pure unit tests, or run against the in-process managed test server (`toys/StackExchange.Redis.Server`) and need no external Redis at all. The rest are **integration** tests that talk to a real server. + +The integration suite needs a **full local Redis topology**, not a single server. Bring it up with docker compose: + +```bash +cd tests/RedisConfigs && docker compose up -d --wait +``` + +Expected servers (defaults in `tests/StackExchange.Redis.Tests/Helpers/TestConfig.cs`): +- `6379` primary, `6380` replica (standalone tests use `6379,6380`) +- `6382`/`6383` failover pair, `6381`/`6384` secure/TLS +- `7000`-`7005` cluster nodes +- `7010`/`7011` + `26379`-`26381` sentinel + +Tests skip as *inconclusive* when their required server is absent (e.g. cluster tests skip with "Unable to connect to server"). Override hosts/ports for local runs with a `tests/StackExchange.Redis.Tests/TestConfig.json` (gitignored). A stray container squatting on `6379` is a common failure: it makes the primary reachable but leaves no replica/cluster, so replica/cluster tests fail or skip — clear it before bringing the compose up. + +To probe these servers ad hoc, the local user may have `resp-cli` installed — a `dotnet` global tool that is functionally similar to `redis-cli` (same basic flags: `-p`, `-a`, `-n`, `--tls`, `-3`). Prefer `resp-cli` when it's available; fall back to `redis-cli` otherwise. + +## Agent skills + +Repo-specific [Agent Skills](https://agentskills.io/home) (the portable `SKILL.md` open standard) live under `.claude/skills/`: + +- `implement-resp-command` — add a new RESP command to StackExchange.Redis end-to-end (enum, interfaces, `RedisDatabase`, `ResultProcessor`, public-API tracking, and the ResultProcessor + RoundTrip unit tests). +- `summarize-database` — profile a live (often production) RESP database by sampling: discover key patterns, where data lives by count and size, and what the values are. Read-only. + +That path is where Claude Code discovers them; the files themselves are tool-agnostic, so if your agent reads skills from a different directory (Codex uses `.agents/skills/`, etc.), point it at this folder or copy the skill across. + +## Conventions + +- Code style is enforced via `.editorconfig` + `Shared.ruleset` + StyleCop; 4-space indent, BOM + final newline on `.cs`, `System.*` usings first, no redundant `this.`. Build will fail on violations. +- `InternalsVisibleTo` exposes internals to the test/benchmark/server projects, so tests reach into internal types directly. +- `docs/` markdown is the user-facing documentation; update it for user-visible behavior changes. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..43c994c2d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 000000000..eeee9b380 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,11 @@ +# Contributing + +Thanks for contributing to StackExchange.Redis! For build, test, and architecture details, see [AGENTS.md](AGENTS.md). + +## Use of AI agents + +Contributions developed with the assistance of AI coding agents are accepted. Whether or not an agent was involved, **the contributing human is fully responsible for the correctness, quality, licensing, and maintainability of the code they submit** — review it as your own work before opening a PR. Reviewers evaluate the contribution, not the tooling behind it. + +## Submitting a pull request + +By submitting a pull request, you affirm that you fully and freely contribute the code under the [project license](LICENSE), that you are legally able to do so, and that you take responsibility for its quality and correctness (including any portions produced with AI assistance). diff --git a/src/StackExchange.Redis/Enums/RedisCommand.cs b/src/StackExchange.Redis/Enums/RedisCommand.cs index 863904f93..a355c02e6 100644 --- a/src/StackExchange.Redis/Enums/RedisCommand.cs +++ b/src/StackExchange.Redis/Enums/RedisCommand.cs @@ -59,7 +59,9 @@ internal enum RedisCommand ECHO, EVAL, EVALSHA, + [AsciiHash("EVAL_RO")] EVAL_RO, + [AsciiHash("EVALSHA_RO")] EVALSHA_RO, EXEC, EXISTS, @@ -208,6 +210,7 @@ internal enum RedisCommand SMISMEMBER, SMOVE, SORT, + [AsciiHash("SORT_RO")] SORT_RO, SPOP, SPUBLISH, diff --git a/tests/StackExchange.Redis.Tests/CommandMapUnitTests.cs b/tests/StackExchange.Redis.Tests/CommandMapUnitTests.cs new file mode 100644 index 000000000..9ec876c01 --- /dev/null +++ b/tests/StackExchange.Redis.Tests/CommandMapUnitTests.cs @@ -0,0 +1,50 @@ +using System; +using System.Text; +using Xunit; + +namespace StackExchange.Redis.Tests; + +/// +/// Pure unit tests (no server) over the default , pinning the exact RESP +/// bulk-string chunk that would be written to the wire for a given . +/// +public class CommandMapUnitTests +{ + [Theory] + // a vanilla command, for baseline + [InlineData(RedisCommand.GET, "$3\r\nGET\r\n")] + [InlineData(RedisCommand.ZREMRANGEBYSCORE, "$16\r\nZREMRANGEBYSCORE\r\n")] + // the read-only variants: the wire name uses an UNDERSCORE (these are the real Redis command + // names: EVAL_RO / EVALSHA_RO / SORT_RO), which is what command.ToString() yields today. + [InlineData(RedisCommand.EVAL_RO, "$7\r\nEVAL_RO\r\n")] + [InlineData(RedisCommand.EVALSHA_RO, "$10\r\nEVALSHA_RO\r\n")] + [InlineData(RedisCommand.SORT_RO, "$7\r\nSORT_RO\r\n")] + public void DefaultCommandMap_GetResp_ProducesExpectedWireBytes(object command, string expectedResp) + { + // command is boxed as object because RedisCommand is internal (less accessible than this public method) + ReadOnlySpan resp = CommandMap.Default.GetResp((RedisCommand)command); + Assert.Equal(expectedResp, Encoding.ASCII.GetString(resp)); + } + + [Theory] + // vanilla command, for baseline + [InlineData("GET", RedisCommand.GET)] + [InlineData("ZREMRANGEBYSCORE", RedisCommand.ZREMRANGEBYSCORE)] + // the underscore variants: parsing the real Redis wire name (with an underscore) MUST round-trip + // back to the matching enum value. This guards against the AsciiHash code-gen inferring '_' -> '-' + // (which would only recognise "EVAL-RO" and fail to parse the actual "EVAL_RO"). + [InlineData("EVAL_RO", RedisCommand.EVAL_RO)] + [InlineData("EVALSHA_RO", RedisCommand.EVALSHA_RO)] + [InlineData("SORT_RO", RedisCommand.SORT_RO)] + public void TryParseCI_ParsesRealWireName(string name, object expected) + { + var expectedCommand = (RedisCommand)expected; + + Assert.True(RedisCommandMetadata.TryParseCI(name.AsSpan(), out var fromChars), $"char parse failed for '{name}'"); + Assert.Equal(expectedCommand, fromChars); + + ReadOnlySpan bytes = Encoding.ASCII.GetBytes(name); + Assert.True(RedisCommandMetadata.TryParseCI(bytes, out var fromBytes), $"byte parse failed for '{name}'"); + Assert.Equal(expectedCommand, fromBytes); + } +}