Skip to content

Support for custom struct tags and runtime context#942

Open
samlown wants to merge 4 commits intoexpr-lang:masterfrom
invopop:with-tag
Open

Support for custom struct tags and runtime context#942
samlown wants to merge 4 commits intoexpr-lang:masterfrom
invopop:with-tag

Conversation

@samlown
Copy link

@samlown samlown commented Mar 5, 2026

This PR tries to solve the issue of defining a custom struct tag to use when determining the name of fields. This is especially useful when dealing with a large and complex set of Go structs that already have a json tag set.

program, err := expr.Compile(`test`, expr.WithTag("json"))

Part of the implementation required the presence of a context to store the Tag being used, and ensure the struct caches are correctly associated with the context, given that different tags may be used with different compiler blocks. This adds a lot more complication. The runtime package also required a clearer context for some of the functions, it might be interesting to move all them to the runtime.Context for consistency, although not required.

The builtin.Function struct now has a FuncWithContext field, that always expects a function with a context argument and is required for some of the default methods that depend on the runtime context to be able to correctly determine some of the field details. I've used a generic context here and added support for a RunWithContext method to the vm that may be useful in some additional circumstances.

Fix: #234
Replaces: #829

Disclaimer: Claude Code was used to do much of the heavy lifting.

@antonmedv
Copy link
Member

Unfortunately, this approach will not work.

Run benchstat /tmp/old.txt /tmp/new.txt
goos: linux
goarch: amd64
pkg: github.com/expr-lang/expr
cpu: AMD EPYC 7763 64-Core Processor                
                           │ /tmp/old.txt │             /tmp/new.txt             │
                           │    sec/op    │    sec/op     vs base                │
_expr-4                       129.1n ± 1%    176.2n ± 2%  +36.52% (p=0.000 n=10)
_expr_eval-4                  8.402µ ± 0%    8.538µ ± 1%   +1.62% (p=0.000 n=10)
_expr_reuseVm-4               73.68n ± 0%   126.55n ± 1%  +71.76% (p=0.000 n=10)
_len-4                        78.79n ± 0%   123.25n ± 0%  +56.42% (p=0.000 n=10)
_filter-4                     61.90µ ± 1%    62.25µ ± 1%   +0.57% (p=0.009 n=10)
_filterLen-4                  56.02µ ± 0%    56.07µ ± 0%        ~ (p=0.353 n=10)
_filterFirst-4                590.4n ± 1%    649.0n ± 1%   +9.92% (p=0.000 n=10)
_filterLast-4                 1.324µ ± 0%    1.355µ ± 1%   +2.34% (p=0.000 n=10)
_filterMap-4                  5.865µ ± 0%    5.954µ ± 0%   +1.52% (p=0.000 n=10)
_arrayIndex-4                 123.9n ± 0%    178.5n ± 0%  +44.01% (p=0.000 n=10)
_envStruct-4                  87.83n ± 0%   145.60n ± 1%  +65.78% (p=0.000 n=10)
_envStruct_noEnv-4            287.6n ± 0%    331.1n ± 0%  +15.13% (p=0.000 n=10)
_envMap-4                     100.6n ± 0%    165.2n ± 1%  +64.21% (p=0.000 n=10)
_callFunc-4                   649.5n ± 0%    702.9n ± 0%   +8.21% (p=0.000 n=10)
_callMethod-4                 656.7n ± 0%    748.5n ± 1%  +13.99% (p=0.000 n=10)
_callField-4                  118.4n ± 0%    171.5n ± 1%  +44.85% (p=0.000 n=10)
_callFast-4                   116.0n ± 0%    167.2n ± 2%  +44.08% (p=0.000 n=10)
_callConstExpr-4              95.89n ± 1%   139.75n ± 1%  +45.73% (p=0.000 n=10)
_largeStructAccess-4          229.2n ± 1%    279.1n ± 1%  +21.75% (p=0.000 n=10)
_largeNestedStructAccess-4    234.4n ± 0%    295.2n ± 0%  +25.97% (p=0.000 n=10)
_largeNestedArrayAccess-4     1.315m ± 2%    1.399m ± 3%   +6.34% (p=0.000 n=10)
_sort-4                       6.633µ ± 0%    6.699µ ± 0%   +1.00% (p=0.003 n=10)
_sortBy-4                     19.24µ ± 1%    17.10µ ± 1%  -11.13% (p=0.000 n=10)
_groupBy-4                    12.19µ ± 1%    12.49µ ± 0%   +2.49% (p=0.001 n=10)
_reduce-4                     56.38n ± 1%    94.96n ± 0%  +68.41% (p=0.000 n=10)
_min-4                        1.002µ ± 0%    1.074µ ± 0%   +7.14% (p=0.000 n=10)
_max-4                        1.008µ ± 0%    1.086µ ± 0%   +7.69% (p=0.000 n=10)
_mean-4                       418.2n ± 0%    456.4n ± 0%   +9.13% (p=0.000 n=10)
_median-4                     3.715µ ± 0%    4.383µ ± 1%  +18.01% (p=0.000 n=10)
geomean                       941.1n         1.142µ       +21.33%

                           │  /tmp/old.txt  │             /tmp/new.txt              │
                           │      B/op      │     B/op      vs base                 │
_expr-4                        32.00 ± 0%       80.00 ± 0%  +150.00% (p=0.000 n=10)
_expr_eval-4                 4.867Ki ± 0%     5.008Ki ± 0%    +2.89% (p=0.000 n=10)
_expr_reuseVm-4                 0.00 ± 0%       48.00 ± 0%         ? (p=0.000 n=10)
_len-4                         32.00 ± 0%       80.00 ± 0%  +150.00% (p=0.000 n=10)
_filter-4                    17.08Ki ± 0%     17.12Ki ± 0%    +0.27% (p=0.000 n=10)
_filterLen-4                 6.039Ki ± 0%     6.086Ki ± 0%    +0.78% (p=0.000 n=10)
_filterFirst-4                 224.0 ± 0%       272.0 ± 0%   +21.43% (p=0.000 n=10)
_filterLast-4                  808.0 ± 0%       856.0 ± 0%    +5.94% (p=0.000 n=10)
_filterMap-4                   920.0 ± 0%       968.0 ± 0%    +5.22% (p=0.000 n=10)
_arrayIndex-4                  40.00 ± 0%       88.00 ± 0%  +120.00% (p=0.000 n=10)
_envStruct-4                   32.00 ± 0%       80.00 ± 0%  +150.00% (p=0.000 n=10)
_envStruct_noEnv-4             32.00 ± 0%       80.00 ± 0%  +150.00% (p=0.000 n=10)
_envMap-4                      32.00 ± 0%       80.00 ± 0%  +150.00% (p=0.000 n=10)
_callFunc-4                    192.0 ± 0%       240.0 ± 0%   +25.00% (p=0.000 n=10)
_callMethod-4                  192.0 ± 0%       240.0 ± 0%   +25.00% (p=0.000 n=10)
_callField-4                   96.00 ± 0%      144.00 ± 0%   +50.00% (p=0.000 n=10)
_callFast-4                    96.00 ± 0%      144.00 ± 0%   +50.00% (p=0.000 n=10)
_callConstExpr-4               96.00 ± 0%      144.00 ± 0%   +50.00% (p=0.000 n=10)
_largeStructAccess-4           56.00 ± 0%      104.00 ± 0%   +85.71% (p=0.000 n=10)
_largeNestedStructAccess-4     56.00 ± 0%      104.00 ± 0%   +85.71% (p=0.000 n=10)
_largeNestedArrayAccess-4    10.00Mi ± 0%     10.00Mi ± 0%    +0.00% (p=0.000 n=10)
_sort-4                      1.883Ki ± 0%     1.930Ki ± 0%    +2.49% (p=0.000 n=10)
_sortBy-4                    5.344Ki ± 0%     5.391Ki ± 0%    +0.88% (p=0.000 n=10)
_groupBy-4                   4.914Ki ± 0%     4.961Ki ± 0%    +0.95% (p=0.000 n=10)
_reduce-4                      32.00 ± 0%       80.00 ± 0%  +150.00% (p=0.000 n=10)
_min-4                         48.00 ± 0%       96.00 ± 0%  +100.00% (p=0.000 n=10)
_max-4                         48.00 ± 0%       96.00 ± 0%  +100.00% (p=0.000 n=10)
_mean-4                        56.00 ± 0%      104.00 ± 0%   +85.71% (p=0.000 n=10)
_median-4                    2.070Ki ± 0%     2.117Ki ± 0%    +2.26% (p=0.000 n=10)
geomean                                   ¹     470.2       ?
¹ summaries must be >0 to compute geomean

                           │ /tmp/old.txt │             /tmp/new.txt             │
                           │  allocs/op   │  allocs/op   vs base                 │
_expr-4                      1.000 ± 0%      2.000 ± 0%  +100.00% (p=0.000 n=10)
_expr_eval-4                 87.00 ± 0%      89.00 ± 0%    +2.30% (p=0.000 n=10)
_expr_reuseVm-4              0.000 ± 0%      1.000 ± 0%         ? (p=0.000 n=10)
_len-4                       1.000 ± 0%      2.000 ± 0%  +100.00% (p=0.000 n=10)
_filter-4                    864.0 ± 0%      865.0 ± 0%    +0.12% (p=0.000 n=10)
_filterLen-4                 749.0 ± 0%      750.0 ± 0%    +0.13% (p=0.000 n=10)
_filterFirst-4               4.000 ± 0%      5.000 ± 0%   +25.00% (p=0.000 n=10)
_filterLast-4                24.00 ± 0%      25.00 ± 0%    +4.17% (p=0.000 n=10)
_filterMap-4                 9.000 ± 0%     10.000 ± 0%   +11.11% (p=0.000 n=10)
_arrayIndex-4                2.000 ± 0%      3.000 ± 0%   +50.00% (p=0.000 n=10)
_envStruct-4                 1.000 ± 0%      2.000 ± 0%  +100.00% (p=0.000 n=10)
_envStruct_noEnv-4           1.000 ± 0%      2.000 ± 0%  +100.00% (p=0.000 n=10)
_envMap-4                    1.000 ± 0%      2.000 ± 0%  +100.00% (p=0.000 n=10)
_callFunc-4                  6.000 ± 0%      7.000 ± 0%   +16.67% (p=0.000 n=10)
_callMethod-4                6.000 ± 0%      7.000 ± 0%   +16.67% (p=0.000 n=10)
_callField-4                 2.000 ± 0%      3.000 ± 0%   +50.00% (p=0.000 n=10)
_callFast-4                  2.000 ± 0%      3.000 ± 0%   +50.00% (p=0.000 n=10)
_callConstExpr-4             2.000 ± 0%      3.000 ± 0%   +50.00% (p=0.000 n=10)
_largeStructAccess-4         4.000 ± 0%      5.000 ± 0%   +25.00% (p=0.000 n=10)
_largeNestedStructAccess-4   4.000 ± 0%      5.000 ± 0%   +25.00% (p=0.000 n=10)
_largeNestedArrayAccess-4    2.000 ± 0%      3.000 ± 0%   +50.00% (p=0.000 n=10)
_sort-4                      5.000 ± 0%      6.000 ± 0%   +20.00% (p=0.000 n=10)
_sortBy-4                    207.0 ± 0%      208.0 ± 0%    +0.48% (p=0.000 n=10)
_groupBy-4                   44.00 ± 0%      45.00 ± 0%    +2.27% (p=0.000 n=10)
_reduce-4                    1.000 ± 0%      2.000 ± 0%  +100.00% (p=0.000 n=10)
_min-4                       2.000 ± 0%      3.000 ± 0%   +50.00% (p=0.000 n=10)
_max-4                       2.000 ± 0%      3.000 ± 0%   +50.00% (p=0.000 n=10)
_mean-4                      3.000 ± 0%      4.000 ± 0%   +33.33% (p=0.000 n=10)
_median-4                    12.00 ± 0%      13.00 ± 0%    +8.33% (p=0.000 n=10)
geomean                                 ¹    7.609       ?
¹ summaries must be >0 to compute geomean

@samlown
Copy link
Author

samlown commented Mar 6, 2026

@antonmedv sorry, I'd noticed that after submitting. Small fix for an inefficient context assignment seems to have corrected the stats for most calls.

@samlown
Copy link
Author

samlown commented Mar 6, 2026

While testing locally, I discovered an issue with tag handling and extras after the commas. Latest commit fixes those.

@antonmedv
Copy link
Member

What if instead of context, we just gonna use some global variable?

@samlown
Copy link
Author

samlown commented Mar 6, 2026

What if instead of context, we just gonna use some global variable?

That's certainly the easiest option. If however multiple packages need to make use of expr independently, they could have different expectations of the tag to use, very hard to debug and fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for custom struct tags like custom:"name"

2 participants