feat: schema generation#1178
Open
gennaroprota wants to merge 18 commits into
Open
Conversation
✨ Highlights
🧾 Changes by Scope
🔝 Top Files
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1178 +/- ##
===========================================
- Coverage 82.12% 82.03% -0.09%
===========================================
Files 33 32 -1
Lines 3149 3090 -59
Branches 734 714 -20
===========================================
- Hits 2586 2535 -51
+ Misses 387 385 -2
+ Partials 176 170 -6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
An automated preview of the documentation is available at https://1178.mrdocs.prtest2.cppalliance.org/index.html If more commits are pushed to the pull request, the docs will rebuild at the same URL. 2026-05-14 07:44:08 UTC |
gennaroprota
force-pushed
the
feat/schema_generation
branch
6 times, most recently
from
cdda298 to
b9fac7c
Compare
Collaborator
:) |
Collaborator
Author
|
Path coverage is OK, isn't it? I added the PR description. |
Collaborator
alandefreitas
left a comment
alandefreitas
left a comment
There was a problem hiding this comment.
What about documentation? How do I use the feature? What about the schema in the documentation? Do we have no schema files in the repository? How do we check if the files are up to date in CI? Some of the changes in the golden files are also kind of weird (like ids changing and things like that and some groups are just empty, which could probably be removed in the schema - I'm not sure).
The PR is great though. The only reason I left many comment is because the PR is huge. 😅
| // ------------------------------------------------------- | ||
| // Header | ||
| // ------------------------------------------------------- | ||
| line("#"); |
Collaborator
There was a problem hiding this comment.
This main function is kind of hard for a human to read. Is this one of these cases where instead of using reflection to improve the schema, we make the function very complex to match the bad pattern we used to have before?
Collaborator
Author
There was a problem hiding this comment.
Indeed. The function is long because the schema is currently describing what XMLWriter happens to emit. That's bad.
Collaborator
There was a problem hiding this comment.
So has this one been resolved?
Collaborator
Author
There was a problem hiding this comment.
RncSchemaWriter.hpp no longer exists, because we emit RNG directly now. RngSchemaWriter.cpp should be relatively easy to read.
| properties.set("class", withDesc(std::move(classSchema), "class")); | ||
|
|
||
| dom::Object boolSchema; | ||
| boolSchema.set("type", "boolean"); |
Collaborator
Author
There was a problem hiding this comment.
Unfortunately, no. Those aren't C++ members of Symbol. They are synthesized at serialization time by tag_invoke:
io.map("class", std::string("symbol"));
io.map("isRegular", I.Extraction == ExtractionMode::Regular);
...
Collaborator
There was a problem hiding this comment.
Oh... That’s exactly the kind of thing reflection has to fix. Reflection is supposed to make the schema consistent. Not to include workarounds to keep the inconsistencies we created by accident because there was no reflection.
In the case of booleans for enums then maybe it's worth providing this as a convenience. Maybe. But if that's the case, reflection should handle this by generalizing it across all variables of the same type booleans/enums.
gennaroprota
force-pushed
the
feat/schema_generation
branch
7 times, most recently
from
73561af to
f9d5f63
Compare
gennaroprota
force-pushed
the
feat/schema_generation
branch
from
f9d5f63 to
fb9c795
Compare
gennaroprota
added a commit
to gennaroprota/mrdocs_fork
that referenced
this pull request
May 13, 2026
`Symbol`'s `tag_invoke` overload added four convenience booleans --
`isRegular`, `isSeeBelow`, `isImplementationDefined`, `isDependency` --
outside the described struct. Because reflection couldn't see them, the
JSON schema writer had to mirror the same hardcoded list.
This removes those booleans and lets templates compare the described
enum directly; e.g.:
{{#if isRegular}} -> {{#if (eq extraction "regular")}}
For the two `filter_by` / `any_of_by` sites that previously keyed on the
booleans, the helper family gains variadic siblings `filter_by_eq` and
`any_of_by_eq` -- signature `(container, key, value1, value2, ...)`.
This addresses review feedback on PR cppalliance#1178.
gennaroprota
force-pushed
the
feat/schema_generation
branch
from
fb9c795 to
0ae30ec
Compare
gennaroprota
added a commit
to gennaroprota/mrdocs_fork
that referenced
this pull request
May 14, 2026
`Symbol`'s `tag_invoke` overload added four convenience booleans --
`isRegular`, `isSeeBelow`, `isImplementationDefined`, `isDependency` --
outside the described struct. Because reflection couldn't see them, the
JSON schema writer had to mirror the same hardcoded list.
This removes those booleans and lets templates compare the described
enum directly; e.g.:
{{#if isRegular}} -> {{#if (eq extraction "regular")}}
For the two `filter_by` / `any_of_by` sites that previously keyed on the
booleans, the helper family gains variadic siblings `filter_by_eq` and
`any_of_by_eq` -- signature `(container, key, value1, value2, ...)`.
This addresses review feedback on PR cppalliance#1178.
gennaroprota
force-pushed
the
feat/schema_generation
branch
from
0ae30ec to
903033b
Compare
Add a --schemas[=<dir>] option that writes a JSON Schema file (mrdocs-dom-schema.json) describing every object and field available to Handlebars templates. The schema is derived from the same compile-time reflection metadata used by MapReflectedType.hpp, so it stays in sync with the code automatically. The option requires no config file or source files — it writes the schema and exits immediately.
OperatorKind was the only enum serialized as a raw integer. All other enums serialize as human-readable strings. Change tag_invoke to use getOperatorName, consistent with the rest.
--schemas now writes both mrdocs-dom-schema.json (Handlebars DOM) and mrdocs.rnc (XML output). The XML schema mirrors XMLWriter.cpp's serialization.
…ClassKind
Replace manual toString and tag_invoke overloads with
MRDOCS_DESCRIBE_ENUM for four enums whose kebab-case names match the
existing string representations.
The XML writer now emits these fields (e.g. <access>public</access>,
<constexpr>constexpr</constexpr>) where they were previously silently
skipped. None/none sentinel values are suppressed via a generic
has_none_enumerator check.
TypeKind stays manual because toKebabCase("LValueReference") produces
"l-value-reference", not the established "lvalue-reference".
… --schemas This guarantees the RELAX NG schema stays in sync with the C++ type definitions. Every CI run now validates all golden test XML files against a schema derived from the same reflection metadata that produces the XML.
This adds a small lookup table keyed by (typeName, memberName) carrying hand-written descriptions for the DOM seen by Handlebars templates which become "description" fields in the JSON schema.
The newly-added JsonEmitter.hpp duplicated functionality already provided by `dom::JSON::stringify`. This drops the new header and uses the existing `stringify`.
The new function name is more descriptive. Contextually, this expands the doc-comment to spell out what the function returns.
The schema headers are only used by ToolMain and the unit tests. They don't need to live under include/mrdocs/.
output XMLWriter silently dropped three kinds of data: - `SpecializationName::TemplateArgs`: `Polymorphic<Name>` fell into `writePolymorphic`'s generic branch with `T` deduced to the base Name (`Polymorphic::operator*` returns a base reference), so only `Name`'s own described members were emitted. Template specializations rendered as `<name>SmallVector</name>` rather than `<specialization-name>SmallVector<...></specialization-name>`. - `NoexceptInfo`: no `MRDOCS_DESCRIBE_STRUCT` - it serializes to a string via `tag_invoke`, which `writeElement` had no path for. Functions with noexcept-specifications produced no `<noexcept>` element. - `ExplicitInfo`: same pattern. explicit constructors and conversion operators produced no `<explicit>` element. This adds a `NameKind` branch in `writePolymorphic` (using a "-name" suffix on the kind tag to disambiguate from the `Name::Identifier` field), and adds a `NoexceptInfo`/`ExplicitInfo` branch in `writeElement` that emits the `toString()` value as text, skipping the empty case. Also update `RncSchemaWriter` to match: `Polymorphic<Name>` -> `AnyName`, drop `NoexceptInfo`/`ExplicitInfo` from the omit list. Most XML golden fixtures regenerate to include the now-emitted elements.
The XMLTags helper in src/lib/Gen/Xml/ contained two pieces of machinery that aren't specific to XML doc generation: an XML escaper (xmlEscape) and a tag/indent stream emitter. A RELAX NG schema writer, which will be introduced with the next commit, also needs them. So, we factored them out.
The --schemas option now writes a RELAX NG XML document directly. This gets rid of the trang RNC->RNG conversion step. Which, in turn, means we no longer need Java. The bootstrap script dependency on Java will be removed with the next commit.
The bootstrap script checked for Java because the build needed it to run trang.jar, which converted the RNC schema to RNG. But trang.jar is no longer used (the --schemas option directly emits a .rng), so Java is no longer needed.
The schema writer emits a `description` field for every type and every described member it touches, looking it up from DomDescriptions.hpp. When the lookup failed, it silently returned an empty string, so forgetting a description caused an undocumented `$defs` entry to be silently emitted. This adds an assert that fires when the lookup of a type or a member finds no entry. This allowed finding many missing entries, which have been added. Any future described type added to the schema trips the assert at build time until descriptions for it and its members are provided.
…ation targets Both schema files are now committed under docs/, parallel to the existing docs/mrdocs.schema.json (the YAML config schema) and are exposed to the Antora docs site as downloadable attachments. Two new CTest targets, `rng-schema-check` and `dom-schema-check`, run `cmake -E compare_files` between the freshly-generated schemas in the build tree and the checked-in copies; drift fails the test. The schemas custom_command is lifted out of the LibXml2 conditional so the freshness checks run independently of whether libxml2 is available. .gitattributes pins the two schema files to LF line endings, because --schemas emits LF line endings and we do a byte-for-byte comparison.
This replaces the hand-written DOM reference with one generated from mrdocs-dom-schema.json. A new Antora extension walks the file's `$defs` in source order and emits one section per type.
`Symbol`'s `tag_invoke` overload added four convenience booleans --
`isRegular`, `isSeeBelow`, `isImplementationDefined`, `isDependency` --
outside the described struct. Because reflection couldn't see them, the
JSON schema writer had to mirror the same hardcoded list.
This removes those booleans and lets templates compare the described
enum directly; e.g.:
{{#if isRegular}} -> {{#if (eq extraction "regular")}}
For the two `filter_by` / `any_of_by` sites that previously keyed on the
booleans, the helper family gains variadic siblings `filter_by_eq` and
`any_of_by_eq` -- signature `(container, key, value1, value2, ...)`.
This addresses review feedback on PR cppalliance#1178.
gennaroprota
force-pushed
the
feat/schema_generation
branch
from
903033b to
019d019
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR addes a
--schemas[=<dir>]option that generates a mrdocs-dom-schema.json (Handlebars DOM) and mrdocs.rnc (XML output). Both schemas are generated from the reflection metadata. The old, hand-written, mrdocs.rnc is replaced entirely in CI.Adds a
--schemas[=<dir>]option that emits two schemas derived from the reflection metadata in the source tree:mrdocs-dom-schema.jsondescribes the Handlebars DOM produced by the generator, andmrdocs.rng(RelaxNG) describes the XML output format. The hand-writtenmrdocs.rncis removed and replaced by the generated.rngeverywhere CI used it, eliminating drift between the schema and the actual output.The schemas are produced by walking the reflection types and member descriptions in the source tree, so they stay in sync with the code automatically as new metadata types are added. The DOM reference documentation is switched from a hand-maintained page to one generated by an Antora extension that reads the JSON schema.
A meaningful chunk of the diff is supporting refactor: a generic XML emitter is factored out of the existing
XMLTags/XMLWritercode intosrc/lib/Support/Xml.{cpp,hpp}, and several reflection types (AccessKind,ConstexprKind,OperatorKind,StorageClassKind,ParamDirection,TypeKind) are made describable through newMapReflectedType/MergeReflectedType/TypeTraitsheaders ininclude/mrdocs/Support/. Three small fixes ride along:OperatorKindis serialized as a string in the DOM (not the underlying enum), the XML emitter no longer drops template args /noexcept/explicitspecifiers, andasserts fire when a reflection type or member lacks a description.Changes
src/lib/Schemas/withDomDescriptions.hpp,DomSchemaWriter.hpp,RngSchemaWriter.{cpp,hpp}. Generic XML emitter extracted intosrc/lib/Support/Xml.{cpp,hpp}and consumed byGen/xml/XMLTagsandXMLWriter(which shrink correspondingly).--schemaswired throughtool/ToolArgs.{cpp,hpp}andtool/ToolMain.cpp. Several specifier headers ininclude/mrdocs/Metadata/refactored alongside newMapReflectedType,MergeReflectedType, andTypeTraitsheaders ininclude/mrdocs/Support/so reflection metadata is describable from the schema writers. Schema-writer headers live insrc/lib/Schemas/(deliberately kept out of the public API).src/test/Schemas/DomSchemaWriter.cppcovering the JSON schema writer end-to-end.noexcept,explicit) that the old XML writer was silently dropping. These are intentional output corrections, not blind regenerations.docs/modules/ROOT/pages/schemas.adocdocuments--schemas.docs/extensions/dom-reference.jsrenders the DOM reference page from the JSON schema; the corresponding hand-written section was removed fromgenerators.adoc.mrdocs-dom-schema.jsonandmrdocs.rngare also linked as downloadable attachments.mrdocs.rncremoved;CMakeLists.txtupdated somrdocs.rngandmrdocs-dom-schema.jsonare emitted and verified during the build. The Java prerequisite (previously needed to process.rnc) is dropped..rnginstead of.rnc— downstream tooling that consumes the schema file by name will need to update. The generated XML output now includes template args,noexcept, andexplicitattributes that were silently omitted before; consumers parsing the XML and not expecting those attributes may need to adjust.Testing
src/test/Schemas/DomSchemaWriter.cppexercises the JSON schema writer against a fixed set of reflection types and asserts the emitted schema matches expectations.mrdocs.rngandmrdocs-dom-schema.jsonfiles are paired with two CI verification steps that regenerate them from the current source and fail if the on-disk version drifts from the regenerated one. This keeps the schemas in sync with the code on every build rather than relying on a contributor remembering to regenerate.noexcept/explicitwill cause golden tests to fail.assertfires when a reflection type or member lacks a description, so any future addition to the reflection metadata that ships without schema metadata trips immediately in CI rather than silently producing an under-specified schema.Documentation
Two pieces:
docs/modules/ROOT/pages/schemas.adocdocuments the new--schemasoption, its outputs, and how to consume them.docs/extensions/dom-reference.jsis an Antora extension that renders the DOM reference frommrdocs-dom-schema.json; the hand-written DOM section ingenerators.adocwas removed in favor of this generated page, so DOM-reference docs now stay in sync with the code automatically.