Skip to content

Comments

hrw4u: native C++ parser and visitor for header_rewrite#12825

Open
zwoop wants to merge 8 commits intoapache:masterfrom
zwoop:Hrw4uCPP
Open

hrw4u: native C++ parser and visitor for header_rewrite#12825
zwoop wants to merge 8 commits intoapache:masterfrom
zwoop:Hrw4uCPP

Conversation

@zwoop
Copy link
Contributor

@zwoop zwoop commented Jan 23, 2026

This essentially converted the existing Python parser / visitor / emitters to C++ and natively calling the various factory functions to create the object structure. In addition, it adds a new testing tool, hrw_confcmp, which compares an old vs a hrw4u configuration, and make sure they both create the same set of objects. As such, the header_rewrite core also added some support to identify the various Operators and Conditions. Usage:

leif@hel ~/BUILDS/ats-dbg » ./tools/hrw_confcmp/run_tests.sh  ~/apache/trafficserver/tools/hrw4u/tests/data/{hooks,conds,ops,vars,examples}

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Testing directory: hooks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  ✓ hooks/read_response:             44 ms
  ✓ hooks/remap:                     39 ms
.
.
.
  ✓ examples/uuid:                   41 ms
  ✓ examples/x-debug:                41 ms

  Directory Summary: 18 passed, 0 failed (total: 18)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Overall Summary
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Total tests:  66
  Passed:       66
  Failed:       0

  Total time:   2721 ms
  Avg per test: 41 ms

✓ All tests passed!

@zwoop zwoop added this to the 11.0.0 milestone Jan 23, 2026
@zwoop zwoop self-assigned this Jan 23, 2026
@zwoop zwoop added header_rewrite header_rewrite plugin hrw4u labels Jan 23, 2026
@zwoop zwoop marked this pull request as draft January 23, 2026 17:02
@zwoop
Copy link
Contributor Author

zwoop commented Jan 23, 2026

I'm making this a draft for now, I think it needs to go into ATS v11, since we're getting close to 10.2 branching and release.

@zwoop zwoop changed the title Hrw4u cpp hrw4u: native C++ parser and visitor for header_rewrite Jan 24, 2026
@zwoop
Copy link
Contributor Author

zwoop commented Jan 31, 2026

[approve ci debian]

@zwoop zwoop force-pushed the Hrw4uCPP branch 2 times, most recently from 0580351 to ae1579b Compare February 9, 2026 02:03
@zwoop
Copy link
Contributor Author

zwoop commented Feb 9, 2026

[approve ci rocky]

@bryancall bryancall requested a review from bneradt February 9, 2026 23:41
@bryancall bryancall modified the milestones: 11.0.0, 10.2.0 Feb 9, 2026
@zwoop zwoop marked this pull request as ready for review February 9, 2026 23:42
@zwoop zwoop force-pushed the Hrw4uCPP branch 4 times, most recently from c2479e2 to 453a6ab Compare February 17, 2026 18:00
@bneradt
Copy link
Contributor

bneradt commented Feb 18, 2026

[approve ci]

Copy link
Contributor

@bneradt bneradt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've verified that with your latest updates I can build this with our internal Yahoo tree in my dev box. However we will need to update our build infrastructure to contain antlr before I can test this in a production box.

I can deal with that over the next few weeks. In the meantime, let's land this work and we can steer the ship if need be.

zwoop and others added 6 commits February 19, 2026 14:53
This also adds a new tool, and infrastructure, to allow us to compare
and verify that an old and new (hrw4u) configuration generates the same
structure of Statements and Operators (functional equivalence).

That tool could also be used in an automation process migrating
existing configurations to hrw4u.

u4wrh: adds an option to disable optimizations
Fixes parser problems in u4wrh with implicit hooks.
This also fixes a few issues in the tool chains
The hrw4u code was developed on macOS with Clang, which is lenient about
C++20 designated initializer ordering and missing fields. GCC enforces
these strictly: designators must appear in struct declaration order, and
skipped fields trigger -Wmissing-field-initializers (fatal under -Werror).
Fix Tables.cc by reordering all designators to match MapParams field order,
Error.cc by adding the skipped .code field, and suppress the missing-field
warning for the hrw4u target. Also add fPIC and ANTLR4 shared library
fallback for systems without the static ANTLR4 library.

The hrw_confcmp tool had three link failures on GNU ld: hrw_confcmp_lib
was missing its dependency on hrw_confcmp_parser (which provides Parser
and HRWSimpleTokenizer symbols), the CertBase::SAN::SANBase::Join stub
returned std::string instead of cripts::string (different mangled name),
and GNU ld's single-pass archive processing missed intra-library symbol
references that macOS ld resolves automatically.

(cherry picked from commit df0ed61c784f193a945a35a68e4f98a7ce59b503)
bneradt
bneradt previously approved these changes Feb 19, 2026
@zwoop
Copy link
Contributor Author

zwoop commented Feb 20, 2026

[approve ci]

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a native C++ ANTLR4-based parser for the header_rewrite plugin, enabling direct parsing of .hrw4u configuration files without requiring external Python tooling. The implementation adds a new src/hrw4u library with visitor-based parsing, integrates it into the header_rewrite plugin, and includes a comparison tool (hrw_confcmp) to validate equivalence between legacy and hrw4u configurations.

Changes:

  • Adds native C++ hrw4u parser library with ANTLR4 visitor implementation, type system, symbol resolution, and error handling
  • Integrates native parser into header_rewrite plugin with factory bridge, object type system, and comparison interfaces
  • Adds hrw_confcmp tool with test runner for validating configuration equivalence
  • Updates Python tools for new operators (set-cc-alg, set-effective-address), improved reverse compilation, and TXN_CLOSE hook support
  • Adds comprehensive .hrw4u test files and documentation updates

Reviewed changes

Copilot reviewed 85 out of 87 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/hrw4u/* Native C++ parser implementation with visitor, types, tables, and error handling
include/hrw4u/* Public API headers for parser library
plugins/header_rewrite/hrw4u.{cc,h} Integration layer between parser and plugin
plugins/header_rewrite/objtypes.{cc,h}, types.h Object type system for native parsing
plugins/header_rewrite/*.{cc,h} Added type_name(), equals(), initialize() methods for comparison
tools/hrw_confcmp/* Configuration comparison tool with test runner
tools/hrw4u/src/* Python tool updates for new features and fixes
tests/gold_tests/* New .hrw4u test files for existing tests
doc/admin-guide/* Updated documentation for native parsing
CMakeLists.txt Build system integration for new components

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

header_rewrite header_rewrite plugin hrw4u

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants