docs(lexical): Allign docs and clean redaction.

Author: dylan-sutton-chavez

docs/pages/getting-started/what-it-is.md

@@ -5,31 +5,31 @@ description: "A subset of Python, compiled to bytecode and run on a sandboxed VM
 
 A sandboxed Python subset with classes, async/await, structural pattern matching, and `packages.json` imports, compiled to bytecode and run on a stack VM with adaptive inline caching and pure-function memoisation. See [Design](/implementation/design) for internals.
 
-Reads like Python (parses Python syntax). Runs differently, where what it executes is curated.
+Reads like Python (parses Python syntax). Runs differently, what it executes is curated.
 
 ## What it supports
 
-* **First-class functions**: pass them, return them, store them in lists and dicts. Decorators apply to both `def` and `class`.
-* **Lambdas with closures**: full lexical capture by value-snapshot at `MakeFunction`, currying, partial application.
-* **Generators and coroutines**: `yield`, `yield from`, `async def`, `await`. Generator expressions are eagerly materialised to lists; use a `def` with `yield` for true laziness.
-* **Comprehensions**: list, dict, and set, with multiple `for` clauses and `if` guards.
-* **Pattern matching**: `match` / `case` with literals, captures, OR-patterns, guards, and sequence patterns (one star permitted).
-* **Exceptions**: `try` / `except` / `else` / `finally`, named handlers, `raise X from Y` (chain info discarded but `X` is what propagates), and subclass-aware matching (`except Exception` catches `RuntimeError`).
-* **Context managers**: `with` and `async with` invoke `__enter__` / `__exit__` on the context-manager value; a truthy return from `__exit__` suppresses the raised exception.
-* **Protocol dunders**: operator overloading, indexing, iteration, hashing, and `repr` / `str` / `format` dispatch through user-defined dunders; see [Dunders](/language/dunders) for the full matrix.
-* **Numbers**: bounded integers (fast inline path, auto-promoting to wide; see [Integer width](/reference/limits-and-errors#integer-width)) and full IEEE-754 floats. No `complex`, `Decimal`, or `Fraction`.
-* **Sequences**: lists, tuples, dicts (insertion-ordered), sets, frozensets, ranges, strings (UTF-8, codepoint-indexed), and bytes.
-* **f-strings**: full grammar, embedded expressions, `{expr=}` self-doc, `!r` / `!s` / `!a` conversions, and format specs covering `s d b o x X f F e E g G n % c` plus fill / align / sign / `#` / `0` / width / `,` / precision.
-* **Walrus operator**: `:=` in expressions (Name target only).
-* **Type annotations**: parsed and discarded, no runtime `__annotations__`, no enforcement.
-* **Module identity**: `__name__` is bound to `"__main__"` in the entry chunk and to the module's spec inside imported modules, so the canonical `if __name__ == "__main__":` guard works as expected.
-* **Modules**: `import`, `from <spec> import names`, and `from <spec> import *` resolve at parse time through a host-injected resolver, with optional `#sha256-<hex>` integrity on URL specs. Two flavors: `.py` source modules and native modules; see [Imports](/reference/imports) for resolution semantics, [Writing modules](/reference/writing-modules) for the three delivery paths, and [Official packages](/reference/packages) for the ready-made modules (`json`, `dom`, `network`, `storage`, and more).
+- **First-class functions**: pass them, return them, store them in lists and dicts. Decorators apply to both `def` and `class`.
+- **Lambdas with closures**: full lexical capture by value-snapshot at `MakeFunction`, currying, partial application.
+- **Generators and coroutines**: `yield`, `yield from`, `async def`, `await`. Generator expressions are eagerly materialised to lists; use a `def` with `yield` for true laziness.
+- **Comprehensions**: list, dict, and set, with multiple `for` clauses and `if` guards.
+- **Pattern matching**: `match` / `case` with literals, captures, OR-patterns, guards, and sequence patterns (one star permitted).
+- **Exceptions**: `try` / `except` / `else` / `finally`, named handlers, `raise X from Y` (chain info discarded but `X` is what propagates), and subclass-aware matching (`except Exception` catches `RuntimeError`).
+- **Context managers**: `with` and `async with` invoke `__enter__` / `__exit__` on the context-manager value; a truthy return from `__exit__` suppresses the raised exception.
+- **Protocol dunders**: operator overloading, indexing, iteration, hashing, and `repr` / `str` / `format` dispatch through user-defined dunders; see [Dunders](/language/dunders) for the full matrix.
+- **Numbers**: bounded integers (fast inline path, auto-promoting to wide; see [Integer width](/reference/limits-and-errors#integer-width)) and full IEEE-754 floats. No `complex`, `Decimal`, or `Fraction`.
+- **Sequences**: lists, tuples, dicts (insertion-ordered), sets, frozensets, ranges, strings (UTF-8, codepoint-indexed), and bytes.
+- **f-strings**: full grammar, embedded expressions, `{expr=}` self-doc, `!r` / `!s` / `!a` conversions, and format specs covering `s d b o x X f F e E g G n % c` plus fill / align / sign / `#` / `0` / width / `,` / precision.
+- **Walrus operator**: `:=` in expressions (Name target only).
+- **Type annotations**: parsed and discarded, no runtime `__annotations__`, no enforcement.
+- **Module identity**: `__name__` is bound to `"__main__"` in the entry chunk and to the module's spec inside imported modules, so the canonical `if __name__ == "__main__":` guard works as expected.
+- **Modules**: `import`, `from <spec> import names`, and `from <spec> import *` resolve at parse time through a host-injected resolver, with optional `#sha256-<hex>` integrity on URL specs. Two flavors: `.py` source modules and native modules; see [Imports](/reference/imports) for resolution semantics, [Writing modules](/reference/writing-modules) for the three delivery paths, and [Official packages](/reference/packages) for the ready-made modules (`json`, `dom`, `network`, `storage`, and more).
 
 ## What it doesn't support
 
 These parse for syntactic compatibility but raise at runtime, or simply don't exist:
 
-- **Standard library**: no bundled stdlib; every module is external (see **Modules** above).
+- **Standard library**: no bundled stdlib; every module is external (see **Modules*- above).
 - **I/O**: `input()` reads from a host-provided buffer. There is no file system, no network, no `os`, no `sys`: these surface only when the host runtime registers them as [host capabilities](/reference/writing-modules#path-b-host-capability) (the same mechanism behind `print` and `input` themselves).
 - **Async surface**: `async def` creates real coroutines and the VM runs a cooperative scheduler, but there is no `asyncio` module: primitives are top-level builtins ([Async](/language/async)). Coroutines do not expose `.send()` / `.throw()` / `.close()`.
 - **Metaclasses, descriptor protocol, `__slots__`**: not modeled.

docs/pages/implementation/lexical.md

@@ -5,11 +5,11 @@ description: "Tokenization, indentation, f-strings, and source-level limits."
 
 ## Overview
 
-Hand-written LUT-driven scanner: walks source as raw bytes, produces `Token { kind, line, start, end }`. Offset-based, tokens carry byte indices, never text copies. The parser slices lazily for identifier and string content.
+Hand-written LUT-driven scanner: walks source as raw bytes, produces `Token { kind, line, start, end }`. Offset-based: tokens carry byte indices, never text copies. The parser slices lazily for identifier and string content.
 
 Linear time *O(n)*, branchless per-byte dispatch through two lookup tables. Lex-time diagnostics (unterminated strings, bad indent, unknown bytes, malformed underscores, oversized f-string nesting) collect in `Vec<LexError>` returned alongside the token stream; the parser folds them in for a single coherent report.
 
-A leading UTF-8 BOM (`EF BB BF`) is stripped before tokenisation so the first identifier doesn't fuse with the marker.
+A leading UTF-8 BOM (`EF BB BF`) is stripped before tokenization so the first identifier doesn't fuse with the marker.
 
 ## Token kinds
 
@@ -19,8 +19,8 @@ The token set tracks Python 3.13.12 closely. Categories implemented:
 - **Soft keywords**: `type`, `match`, and `case` demote to `Name` when followed by `(`, `:`, `=`, `,`, `)`, `]`, `Newline`, or `EOF`, so `type()`, `match(...)`, and identifiers named like them stay usable; at statement start (`match x:`) they keep keyword force.
 - **Wildcard**: Underscore (`_`) gets its own `Underscore` token; the parser distinguishes wildcard from name use.
 - **Operators**: 1-, 2-, and 3-character operator forms (`+`, `==`, `**=`, `//=`, etc.).
-- **Delimiters**: `( ) [ ] { } :, ; .`.
-- **Literals**: `Name`, `Int`, `Float`, `String`, `Bytes`. There is no `Complex` token, a trailing `j` / `J` is **not** lexed as a complex suffix; `1j` tokenises as `Int(1)` followed by `Name("j")`.
+- **Delimiters**: `( ) [ ] { } : , ; .`.
+- **Literals**: `Name`, `Int`, `Float`, `String`, `Bytes`. There is no `Complex` token: a trailing `j` / `J` is **not** lexed as a complex suffix; `1j` tokenises as `Int(1)` followed by `Name("j")`.
 - **F-string segments**: `FstringStart`, `FstringMiddle`, `FstringEnd`.
 - **Whitespace and structure**: `Comment`, `Newline`, `Indent`, `Dedent`, `Nl`, `Endmarker`.
 
@@ -54,7 +54,7 @@ Identifiers, digits, and whitespace use a `scan_while(pred)` driver looping over
 
 The scanner handles base prefixes (`0x` / `0o` / `0b`, case-insensitive), underscore separators, optional exponents, and leading-dot form. `Int` and `Float` are the only numeric token kinds.
 
-Underscores must sit between digits, leading/trailing/doubled raise `invalid '_' in numeric literal` or `consecutive '_' in numeric literal`. Empty radix body (`0x`, `0o`, `0b`) raises `missing digits in numeric literal`. Trailing dot (`5.`) is valid; empty exponent body (`1e`) is left to the float parser to avoid false-positives in format specs.
+Underscores must sit between digits; leading/trailing/doubled raise `invalid '_' in numeric literal` or `consecutive '_' in numeric literal`. Empty radix body (`0x`, `0o`, `0b`) raises `missing digits in numeric literal`. Trailing dot (`5.`) is valid; empty exponent body (`1e`) is left to the float parser to avoid false-positives in format specs.
 
 Complex literals unsupported: `1j` lexes as `Int(1)` + `Name("j")`.
 
@@ -72,7 +72,7 @@ fr'raw fstring' # raw f-string
 """triple""" # triple-quoted, single or double
 ```
 
-A leading prefix is recognised before the opening quote by the identifier scanner, verified against `is_string_prefix` / `is_fstring_prefix` / `is_bytes_prefix`. Triple-quoted strings span newlines (bumping `line` per `\n`). Backslash escapes are consumed at lex time but decoded by the parser. Recognised escapes: `\n \t \r \a \b \f \v \\ \' \" \xHH \uHHHH \UHHHHHHHH` plus 1,3 digit octal (`\012` -> `\n`, `\101` -> `A`). `\N{NAME}` is unimplemented, the 200 KB Unicode-name database is too costly for the WASM artifact.
+A leading prefix is recognised before the opening quote by the identifier scanner, verified against `is_string_prefix` / `is_fstring_prefix` / `is_bytes_prefix`. Triple-quoted strings span newlines (bumping `line` per `\n`). Backslash escapes are consumed at lex time but decoded by the parser. Recognised escapes: `\n \t \r \a \b \f \v \\ \' \" \xHH \uHHHH \UHHHHHHHH` plus 1–3 digit octal (`\012` -> `\n`, `\101` -> `A`). `\N{NAME}` is unimplemented — the 200 KB Unicode-name database is too costly for the WASM artifact.
 
 Errors anchor on the opening quote so the `^` marker points at the offender, not at end-of-line:
 
@@ -100,7 +100,7 @@ FstringMiddle("!")
 FstringEnd
 ```
 
-Expression tokens between `{` and `}` are emitted by the main lexer, not the f-string scanner, full expression grammar inside interpolations without special casing.
+Expression tokens between `{` and `}` are emitted by the main lexer, not the f-string scanner: full expression grammar inside interpolations, without special casing.
 
 `{{` and `}}` are escaped literal braces, no `Lbrace` / `Rbrace`; they survive into `FstringMiddle` text and are unescaped by the parser.
 
@@ -120,9 +120,9 @@ Edge Python uses an INDENT/DEDENT model. The scanner tracks a stack of column co
 | Dedent doesn't match an outer level | diagnostic `unindent does not match any outer indentation level` |
 | Mixed tabs and spaces in indent | `Endmarker` (lex halt) + diagnostic |
 
-The `nesting` counter is bumped by `(`, `[`, `{` and decremented by `)`, `]`. While `nesting > 0` line breaks emit `Nl` and the indent stack is frozen, multi-line expressions inside brackets without spurious INDENT/DEDENT.
+The `nesting` counter is bumped by `(`, `[`, `{` and decremented by `)`, `]`, `}`. While `nesting > 0`, line breaks emit `Nl` and the indent stack is frozen — multi-line expressions inside brackets without spurious INDENT/DEDENT.
 
-At EOF the lexer drains remaining levels off `indent_stack` for clean block closure, then emits `Endmarker`. No support for backslash line continuation (`\` + `\n`) outside brackets, wrap in parens. ASCII bytes with no operator slot (`$`, `?`, `` ` ``, stray `\`) raise `unexpected character` and are skipped.
+At EOF the lexer drains remaining levels off `indent_stack` for clean block closure, then emits `Endmarker`. No support for backslash line continuation (`\` + `\n`) outside brackets; wrap in parens. ASCII bytes with no operator slot (`$`, `?`, `` ` ``, stray `\`) raise `unexpected character` and are skipped.
 
 ## Soft-keyword disambiguation