Skip to content

Commit e931b7e

Browse files
mjbommarclaude
mjbommar
and
claude
committed
marshal: add final validation section + raw perf data folder
Appends to Misc/marshal-perf-diary.md the results of the full test suite rerun (48,932 tests pass, including the new RecursiveGraphTest combinatoric cases) and a `pyperformance` comparison against main on the same 10-benchmark marshal-adjacent slice the design doc used. Significant results on the pyperformance slice: python_startup 1.18x faster (t=59.80) python_startup_no_site 1.03x faster (t=12.90) All other slice benchmarks within noise; no regressions. Adds Misc/marshal-perf-data/ with the raw JSON backing every table in the diary: all per-experiment microbench runs (exp0..exp9, expC, final) plus the two pyperf-slice JSONs and a README describing the layout and reproduction commands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 4aaf064 commit e931b7e

40 files changed

Lines changed: 3877 additions & 0 deletions

‎Misc/marshal-perf-data/README.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Marshal Perf Raw Data
2+
3+
Raw JSON artifacts backing `Misc/marshal-perf-diary.md`. Everything here
4+
is regeneratable — kept checked in so reviewers can inspect numbers
5+
without rerunning the full methodology.
6+
7+
## Marshal microbench (`marshal_bench_cpu_stable.py`)
8+
9+
Three pinned runs per commit, 11 repeats per operation, 200k `loads` ×
10+
100k `dumps`. Pinning: `taskset -c 0`.
11+
12+
| File | Commit / idea |
13+
| --- | --- |
14+
| `exp0-run{1..3}.json` | `perf-baseline` self-check at `eb1c4b7` |
15+
| `exp1-run{1..3}.json` | Experiment 1 (raw `PyObject **` refs array) |
16+
| `exp2-run{1..3}.json` | Experiment 2 (tagged pointer state, stacked on 1) |
17+
| `exp3-run{1..3}.json` | Experiment 3 (collapse reserve+publish) |
18+
| `exp4-run{1..3}.json` | Experiment 4 (`has_incomplete` counter) |
19+
| `exp5-run{1..3}.json` | Experiment 5 (zero-init + drop mark_ready) |
20+
| `exp6-run{1..3}.json` | Experiment 6 (drop `allow_incomplete_hashable` param) |
21+
| `exp7-run{1..3}.json` | Experiment 7 (force-inline helpers) |
22+
| `exp8-run{1..3}.json` | Experiment 8 (`frozendict`-then-wrap when `flag==0`) |
23+
| `exp9-run{1..3}.json` | Experiment 9 (preallocate refs capacity) |
24+
| `expC-run{1..3}.json` | Combined (Exp 1+2+6+7) on `exp/combined-winners` |
25+
| `final-head-run{1..3}.json` | Post-squash HEAD `4aaf064344d` (same as Combined) |
26+
27+
Schema per bench (small_tuple / nested_dict / code_obj):
28+
29+
loads_number, dumps_number # inner loop counts
30+
loads_runs: [11 times] # raw timings
31+
dumps_runs: [11 times]
32+
loads_min, loads_median # statistics
33+
dumps_min, dumps_median
34+
35+
## pyperformance slice
36+
37+
Ten marshal-adjacent benchmarks (pickle × 4, unpickle × 3,
38+
python_startup × 2, unpack_sequence). Run with `uvx pyperformance run
39+
-b <list> --affinity 0` via `taskset -c 0`.
40+
41+
| File | Python |
42+
| --- | --- |
43+
| `pyperf-slice-baseline.json` | `main` tip `2faceeec` built at `/tmp/cpython-baseline/python` |
44+
| `pyperf-slice-current.json` | HEAD `4aaf064344d` built in-tree |
45+
46+
These are standard `pyperf` format; compare with:
47+
48+
uvx pyperformance compare pyperf-slice-baseline.json pyperf-slice-current.json
49+
50+
## Regeneration
51+
52+
See `Misc/marshal-perf-diary.md` → "Ground rules" for the exact harness
53+
(`/tmp/marshal_bench_cpu_stable.py` inline script) and methodology.

‎Misc/marshal-perf-data/exp0-run1.json‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
{
2+
"code_obj": {
3+
"dumps_median": 0.040405122010270134,
4+
"dumps_min": 0.039836373005528,
5+
"dumps_number": 100000,
6+
"dumps_runs": [
7+
0.040096846991218626,
8+
0.04028077999828383,
9+
0.04065272197476588,
10+
0.04065836101653986,
11+
0.04028131702216342,
12+
0.040413388021988794,
13+
0.039836373005528,
14+
0.04000361601356417,
15+
0.04100019999896176,
16+
0.040806725999573246,
17+
0.040405122010270134
18+
],
19+
"loads_median": 0.09791472702636383,
20+
"loads_min": 0.09674789398559369,
21+
"loads_number": 200000,
22+
"loads_runs": [
23+
0.09674789398559369,
24+
0.09708369200234301,
25+
0.09783264898578636,
26+
0.09744057801435702,
27+
0.09815629699733108,
28+
0.09906345797935501,
29+
0.09835395799018443,
30+
0.09974412800511345,
31+
0.09990381199168041,
32+
0.09737725998274982,
33+
0.09791472702636383
34+
]
35+
},
36+
"nested_dict": {
37+
"dumps_median": 0.07455040101194754,
38+
"dumps_min": 0.07420829197508283,
39+
"dumps_number": 100000,
40+
"dumps_runs": [
41+
0.07494553699507378,
42+
0.07451574600418098,
43+
0.07446633197832853,
44+
0.07566936200601049,
45+
0.07430389401270077,
46+
0.07455040101194754,
47+
0.07424808200448751,
48+
0.07420829197508283,
49+
0.07491164302336983,
50+
0.07538196400855668,
51+
0.0746764489740599
52+
],
53+
"loads_median": 0.08682646200759336,
54+
"loads_min": 0.08603669900912791,
55+
"loads_number": 200000,
56+
"loads_runs": [
57+
0.08728363699628972,
58+
0.08605418298975565,
59+
0.08732464100467041,
60+
0.08639801701065153,
61+
0.08744612199370749,
62+
0.08878127401112579,
63+
0.08682646200759336,
64+
0.08644566498696804,
65+
0.08648836001520976,
66+
0.08734603400807828,
67+
0.08603669900912791
68+
]
69+
},
70+
"small_tuple": {
71+
"dumps_median": 0.0156986290239729,
72+
"dumps_min": 0.01551589198061265,
73+
"dumps_number": 100000,
74+
"dumps_runs": [
75+
0.015826012007892132,
76+
0.015661484998418018,
77+
0.01551589198061265,
78+
0.01563638899824582,
79+
0.0156986290239729,
80+
0.015819393011042848,
81+
0.01584347800235264,
82+
0.01551837200531736,
83+
0.015644578001229092,
84+
0.01609343197196722,
85+
0.01589953200891614
86+
],
87+
"loads_median": 0.032818893989315256,
88+
"loads_min": 0.032478501991136,
89+
"loads_number": 200000,
90+
"loads_runs": [
91+
0.039915669010952115,
92+
0.034840578999137506,
93+
0.032818893989315256,
94+
0.032755594002082944,
95+
0.032478501991136,
96+
0.03285682899877429,
97+
0.032941445999313146,
98+
0.03272099400055595,
99+
0.032805391994770616,
100+
0.0325985630042851,
101+
0.032910926995100453
102+
]
103+
}
104+
}

‎Misc/marshal-perf-data/exp0-run2.json‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
{
2+
"code_obj": {
3+
"dumps_median": 0.039624935016036034,
4+
"dumps_min": 0.03912865900201723,
5+
"dumps_number": 100000,
6+
"dumps_runs": [
7+
0.04005826299544424,
8+
0.039624935016036034,
9+
0.03912865900201723,
10+
0.03937379698618315,
11+
0.03985618698061444,
12+
0.0396080739737954,
13+
0.03980735401273705,
14+
0.039562535006552935,
15+
0.03952119100722484,
16+
0.04000154402456246,
17+
0.03970523300813511
18+
],
19+
"loads_median": 0.09828513601678424,
20+
"loads_min": 0.0965760959952604,
21+
"loads_number": 200000,
22+
"loads_runs": [
23+
0.09915574299520813,
24+
0.0985370319976937,
25+
0.0965760959952604,
26+
0.09828084200853482,
27+
0.097619662003126,
28+
0.10028967598918825,
29+
0.0983922420127783,
30+
0.0987230259925127,
31+
0.09828513601678424,
32+
0.09821752097923309,
33+
0.09726348100230098
34+
]
35+
},
36+
"nested_dict": {
37+
"dumps_median": 0.07583842999883927,
38+
"dumps_min": 0.07371149398386478,
39+
"dumps_number": 100000,
40+
"dumps_runs": [
41+
0.07573145598871633,
42+
0.07466877298429608,
43+
0.07562672600033693,
44+
0.07371149398386478,
45+
0.07529966699075885,
46+
0.07686721297795884,
47+
0.07583842999883927,
48+
0.0770502710074652,
49+
0.076849859993672,
50+
0.07778827799484134,
51+
0.07668794898199849
52+
],
53+
"loads_median": 0.08660990899079479,
54+
"loads_min": 0.0859223100123927,
55+
"loads_number": 200000,
56+
"loads_runs": [
57+
0.08658557198941708,
58+
0.08649306499864906,
59+
0.08618080499581993,
60+
0.0874386359937489,
61+
0.08762964798370376,
62+
0.0859223100123927,
63+
0.0869109459745232,
64+
0.0878543850267306,
65+
0.08672871999442577,
66+
0.08612436000839807,
67+
0.08660990899079479
68+
]
69+
},
70+
"small_tuple": {
71+
"dumps_median": 0.015691205015173182,
72+
"dumps_min": 0.015531735989497975,
73+
"dumps_number": 100000,
74+
"dumps_runs": [
75+
0.015568955015623942,
76+
0.015662263002013788,
77+
0.01571622901246883,
78+
0.015571760974125937,
79+
0.015531735989497975,
80+
0.01582504398538731,
81+
0.01576065199333243,
82+
0.015691205015173182,
83+
0.015571972995530814,
84+
0.015708502003690228,
85+
0.015745237993542105
86+
],
87+
"loads_median": 0.03315894500701688,
88+
"loads_min": 0.03273815900320187,
89+
"loads_number": 200000,
90+
"loads_runs": [
91+
0.036090038978727534,
92+
0.03278564300853759,
93+
0.03315894500701688,
94+
0.03321557000163011,
95+
0.032992702996125445,
96+
0.0333719459886197,
97+
0.03368866798700765,
98+
0.03273815900320187,
99+
0.03284121898468584,
100+
0.03342736198101193,
101+
0.032767266005976126
102+
]
103+
}
104+
}

‎Misc/marshal-perf-data/exp0-run3.json‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
{
2+
"code_obj": {
3+
"dumps_median": 0.04041685801348649,
4+
"dumps_min": 0.03987288600183092,
5+
"dumps_number": 100000,
6+
"dumps_runs": [
7+
0.040255115018226206,
8+
0.040099431993439794,
9+
0.04095420497469604,
10+
0.03987288600183092,
11+
0.04041685801348649,
12+
0.040494924993254244,
13+
0.04049262698390521,
14+
0.04079927600105293,
15+
0.04118377499980852,
16+
0.04038904799381271,
17+
0.04038159898482263
18+
],
19+
"loads_median": 0.09712665400002152,
20+
"loads_min": 0.09544071301934309,
21+
"loads_number": 200000,
22+
"loads_runs": [
23+
0.09710476599866524,
24+
0.09544071301934309,
25+
0.09796852199360728,
26+
0.09641282400116324,
27+
0.09771751399966888,
28+
0.09875676699448377,
29+
0.09825973899569362,
30+
0.09712665400002152,
31+
0.09814226999878883,
32+
0.09603261601296254,
33+
0.0969423119968269
34+
]
35+
},
36+
"nested_dict": {
37+
"dumps_median": 0.07670464701368473,
38+
"dumps_min": 0.07443057300406508,
39+
"dumps_number": 100000,
40+
"dumps_runs": [
41+
0.07670464701368473,
42+
0.0755682380113285,
43+
0.07518417699611746,
44+
0.07505606301128864,
45+
0.07443057300406508,
46+
0.07728818198665977,
47+
0.07829541899263859,
48+
0.07810401200549677,
49+
0.07622043599258177,
50+
0.07752709698979743,
51+
0.07774014701135457
52+
],
53+
"loads_median": 0.08734592801192775,
54+
"loads_min": 0.08589171001221985,
55+
"loads_number": 200000,
56+
"loads_runs": [
57+
0.08709321398055181,
58+
0.08734592801192775,
59+
0.08714917601901107,
60+
0.08695745901786722,
61+
0.08589171001221985,
62+
0.08879310198244639,
63+
0.08659902500221506,
64+
0.08748295900295489,
65+
0.08759746799478307,
66+
0.08768528798827901,
67+
0.08783682601642795
68+
]
69+
},
70+
"small_tuple": {
71+
"dumps_median": 0.01571595700806938,
72+
"dumps_min": 0.015293899981770664,
73+
"dumps_number": 100000,
74+
"dumps_runs": [
75+
0.01571595700806938,
76+
0.015348431013990194,
77+
0.015293899981770664,
78+
0.015364045015303418,
79+
0.01530077401548624,
80+
0.015608929010340944,
81+
0.015807940013473853,
82+
0.01611778000369668,
83+
0.01599793799687177,
84+
0.01602914900286123,
85+
0.016072057012934238
86+
],
87+
"loads_median": 0.03303753398358822,
88+
"loads_min": 0.032468440011143684,
89+
"loads_number": 200000,
90+
"loads_runs": [
91+
0.036123027006397024,
92+
0.03290336497593671,
93+
0.032468440011143684,
94+
0.032748425990575925,
95+
0.03303753398358822,
96+
0.033069997996790335,
97+
0.03337217401713133,
98+
0.03351389098679647,
99+
0.033144343993626535,
100+
0.03277306197560392,
101+
0.03264542398392223
102+
]
103+
}
104+
}

0 commit comments

Comments
 (0)