dedup high-cost localization setup

[oech3]

mimalloc complained about localization.

[codspeed-hq]

Merging this PR will improve performance by ×6.5

⚡ 91 improved benchmarks
✅ 201 untouched benchmarks
⏩ 44 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	wc_lines_large_line_count[500000]	2.9 ms	2.8 ms	+4.02%
⚡	Simulation	wc_lines_variable_length[(50, 500)]	3.5 ms	3.3 ms	+3.24%
⚡	Simulation	wc_bytes_synthetic[500]	194.8 µs	87.5 µs	×2.2
⚡	Simulation	wc_chars_large_line_count[100000]	1,022.9 µs	912.9 µs	+12.05%
⚡	Simulation	b64_decode_synthetic	174.6 µs	63.3 µs	×2.8
⚡	Simulation	b64_decode_ignore_garbage_synthetic	169.9 µs	63.1 µs	×2.7
⚡	Simulation	b64_encode_synthetic	169.3 µs	58.5 µs	×2.9
⚡	Memory	factor_multiple_u64s[2]	93.4 KB	52.5 KB	+78.03%
⚡	Memory	seq_formatted	46.7 KB	10.1 KB	×4.6
⚡	Memory	seq_integers	46.6 KB	9.6 KB	×4.8
⚡	Memory	seq_with_step	46.6 KB	9.7 KB	×4.8
⚡	Simulation	mv_force_overwrite	141.9 ms	62.6 ms	×2.3
⚡	Simulation	mv_single_file	138.7 ms	62.6 ms	×2.2
⚡	Memory	df_with_path	46.6 KB	33.1 KB	+40.89%
⚡	Simulation	df_with_path	406.6 µs	306.4 µs	+32.71%
⚡	Simulation	rm_force_files	2.2 ms	2.1 ms	+4.73%
⚡	Simulation	rm_multiple_files	2.3 ms	2.2 ms	+4.48%
⚡	Simulation	rm_single_file	123.9 ms	46.7 ms	×2.7
⚡	Memory	split_lines	46.7 KB	32.1 KB	+45.5%
⚡	Memory	split_bytes	46.7 KB	32.1 KB	+45.5%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

---

_{Comparing oech3:reuse-localiz (ba9dc5b) with main (db77abe)}

Footnotes

1. 44 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[github-actions]

GNU testsuite comparison:

Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/inotify-dir-recreate (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/dd/no-allocate is now being skipped but was previously passing.
Note: The gnu test tests/rm/many-dir-entries-vs-OOM is now being skipped but was previously passing.
Congrats! The gnu test tests/tail/tail-n0f is now passing!

[oech3]

cc @ChrisDryden @lordeji @sylvestre

[github-actions]

GNU testsuite comparison:

Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/inotify-dir-recreate (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/tail/tail-n0f is now passing!

[lordeji]

@oech3 I think I'm missing something...

Of what I see, the goal seems to be to not do a duplicate setup, but I don't understand why you set LOCALIZER_REUSE_HINT.

You only need LOCALIZER_REUSE (imo, should be renamed something like LOCALIZER_IS_SET for clarity, because the name made me think that it would setup if it was true) and then :

pub fn setup_localization(p: &str) -> Result<(), LocalizationError> {
    if LOCALIZER_REUSE.with(Cell::get) {
        return Ok(());
    }
    [...]
    LOCALIZER_REUSE.with(|f| f.set(true));
    Ok(())
}

Do you want LOCALIZER_REUSE_HINT to be set in the command line to remove localization ? If so, it should fallback to embedded english

[oech3]

I defined it outside of thread_local! to avoid high cost check. expand regressed performance without it.

[oech3]

@lordeji was right. Maybe, I was if LOCALIZER.with(|lock| lock.get().is_some()) {return Ok(());} when expand regressed performance. I applied your suggestions. Thankyou.

[github-actions]

GNU testsuite comparison:

GNU test failed: tests/rm/isatty. tests/rm/isatty is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/inotify-dir-recreate (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/tail/tail-n0f is now passing!

[lordeji]

@oech3 If you confirm that the new variable has no goal to be modified externally, then I agree with you.
Just looking if LOCALIZER_IS_SET is the safest and most minimal solution for now.

I was waiting for PR #10911 but it seems to be inactive, if in march it did not changed, I will try to do the PR myself if I'm allowed.

[oech3]

I was waiting for PR #10911 but it seems to be inactive

I agree with lazy loading is preffered, but requires many diff and would take many times to review...

[github-actions]

GNU testsuite comparison:

GNU test failed: tests/rm/isatty. tests/rm/isatty is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/tail/symlink (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/inotify-dir-recreate (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/tail/tail-n0f is now passing!

[oech3]

If you confirm that the new variable has no goal to be modified externally, then I agree with you.

How to block touching LOCALIZER_IS_SET oute side of fn setup_localization?

[lordeji]

@oech3 Define it directly into the function

pub fn setup_localization(p: &str) -> Result<(), LocalizationError> {
    thread_local! {
        static LOCALIZER_IS_SET: Cell<bool> = const { Cell::new(false) };
    }
    [...]
}

I did not test it but static lifetime and local scope is not incompatible and the thread_local! macro doc doesn't specify anything about scope, worth a try.

[github-actions]

GNU testsuite comparison:

Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/inotify-dir-recreate (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/seq/seq-epipe is now being skipped but was previously passing.
Congrats! The gnu test tests/tail/tail-n0f is now passing!

[oech3]

@lordeji Test passed with the form. Thankyou.

@ChrisDryden Is this workaround OK for you until lazy loading was added?

[github-actions]

GNU testsuite comparison:

GNU test failed: tests/tail/retry. tests/tail/retry is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/follow-name (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/printf/printf-surprise is now passing!
Congrats! The gnu test tests/tail/tail-n0f is now passing!

[oech3]

I dropped broken bench to measure valid score.

[github-actions]

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/date/resolution (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/rm/many-dir-entries-vs-OOM is now passing!
Congrats! The gnu test tests/tail/tail-n0f is now passing!