Skip to content

Benchmark error fix and 0.42.0 additions #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
soypat wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Benchmark error fix and 0.42.0 additions #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
979f8c6
fix 'results failed open arguments file args.txt' and add 0.42.0 benc…
soypat May 10, 2026
db72371
fix microbenchmarking and having more than one long benchmark per case
soypat May 10, 2026
c73c35c
fix benchmark margins
soypat May 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions bench_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,10 +110,10 @@ func BenchmarkAll(b *testing.B) {
for _, testname := range benchnames {
argdata, err := os.ReadFile(testname + "/args.txt")
casesJoined := strings.TrimSpace(string(argdata))
if len(argdata) == 0 {
b.Fatalf("%s has empty 'args.txt' file", testname)
} else if err != nil {
b.Fatalf("%s failed open arguments file 'args.txt': %s", testname, err)
if err != nil {
b.Fatalf("benchmark %q: failed open arguments file 'args.txt': %s", testname, err)
} else if len(argdata) == 0 {
b.Fatalf("benchmark %q: has empty 'args.txt' file", testname)
}

cases := strings.Split(casesJoined, "\n")
Expand Down
5 changes: 2 additions & 3 deletions fannkuch-redux/args.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
6
7
9
9
10
4 changes: 2 additions & 2 deletions fasta/args.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
12500000
25000000
1000000
8000000
5 changes: 2 additions & 3 deletions n-body-nosqrt/args.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
50000
100000
200000
200000
300000
5 changes: 2 additions & 3 deletions n-body/args.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
50000
100000
200000
200000
300000
3 changes: 2 additions & 1 deletion plot_/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ func drawBenchmark(langs []langBench, savefile, baseCompiler string) error {
var (
maxBenchs = nBenchs * len(langs)
plotHeight = 10 * vg.Inch
plotWidth = plotHeight * vg.Length(maxBenchs) / 20
plotWidth = plotHeight*vg.Length(maxBenchs)/20 + 1*vg.Centimeter
benchWidth = plotWidth / vg.Length(nBenchs)

fontsize = plotHeight / 25
Expand Down Expand Up @@ -117,6 +117,7 @@ func drawBenchmark(langs []langBench, savefile, baseCompiler string) error {
}
p_time.Legend.Top = true
p_time.NominalX(nominals...)
p_time.X.Max *= 1.1 // Give bars margin so they don't overlap with legend.
p_time.X.Tick.Label.Font.Size = benchWidth / 12 // Nominal size.

p_time.Title.TextStyle.Font.Size = fontsize
Expand Down
0 results/tinygo-0.39.0.png → results_/tinygo-0.39.0.png
View file
Open in desktop
File renamed without changes
0 results/tinygo-0.40.0.png → results_/tinygo-0.40.0.png
View file
Open in desktop
File renamed without changes
Binary file added results_/tinygo-0.42.0.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
176 changes: 176 additions & 0 deletions results_/tinygo-0.42.0.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
goos: linux
goarch: amd64
pkg: tinybench
cpu: 12th Gen Intel(R) Core(TM) i5-12400F
BenchmarkAll
bench_test.go:107: found compiler zig 0.15.2
bench_test.go:107: found compiler rustc 1.95.0
bench_test.go:107: found compiler go 1.25.7
bench_test.go:107: found compiler tinygo 0.42.0
bench_test.go:107: found compiler gcc 15.2.1
bench_test.go:107: found compiler clang 20.1.1
bench_test.go:109: looking for benchmarks in [fannkuch-redux fasta n-body n-body-nosqrt spectral-norm]
BenchmarkAll/fannkuch-redux:args=9/zig/zig
bench_test.go:145: name="fannkuch-redux" compiler="zig" binarysize=2346112 version=0.15.2
BenchmarkAll/fannkuch-redux:args=9/zig/zig-12 84 13338193 ns/op
BenchmarkAll/fannkuch-redux:args=10/zig/zig
BenchmarkAll/fannkuch-redux:args=10/zig/zig-12 8 138295479 ns/op
BenchmarkAll/fannkuch-redux:args=9/rust/rustc
bench_test.go:145: name="fannkuch-redux" compiler="rustc" binarysize=4347344 version=1.95.0
BenchmarkAll/fannkuch-redux:args=9/rust/rustc-12 76 15641016 ns/op
BenchmarkAll/fannkuch-redux:args=10/rust/rustc
BenchmarkAll/fannkuch-redux:args=10/rust/rustc-12 7 165121882 ns/op
BenchmarkAll/fannkuch-redux:args=9/go/go
bench_test.go:145: name="fannkuch-redux" compiler="go" binarysize=2344532 version=1.25.7
BenchmarkAll/fannkuch-redux:args=9/go/go-12 69 16479844 ns/op
BenchmarkAll/fannkuch-redux:args=10/go/go
BenchmarkAll/fannkuch-redux:args=10/go/go-12 7 162629276 ns/op
BenchmarkAll/fannkuch-redux:args=9/go/tinygo
bench_test.go:145: name="fannkuch-redux" compiler="tinygo" binarysize=1541496 version=0.42.0
BenchmarkAll/fannkuch-redux:args=9/go/tinygo-12 86 13419369 ns/op
BenchmarkAll/fannkuch-redux:args=10/go/tinygo
BenchmarkAll/fannkuch-redux:args=10/go/tinygo-12 8 141694903 ns/op
BenchmarkAll/fannkuch-redux:args=9/c/gcc
bench_test.go:145: name="fannkuch-redux" compiler="gcc" binarysize=16328 version=15.2.1
BenchmarkAll/fannkuch-redux:args=9/c/gcc-12 70 15476821 ns/op
BenchmarkAll/fannkuch-redux:args=10/c/gcc
BenchmarkAll/fannkuch-redux:args=10/c/gcc-12 7 160211098 ns/op
BenchmarkAll/fannkuch-redux:args=9/c/clang
bench_test.go:145: name="fannkuch-redux" compiler="clang" binarysize=16280 version=20.1.1
BenchmarkAll/fannkuch-redux:args=9/c/clang-12 69 15687532 ns/op
BenchmarkAll/fannkuch-redux:args=10/c/clang
BenchmarkAll/fannkuch-redux:args=10/c/clang-12 7 160899619 ns/op
BenchmarkAll/fasta:args=1000000/zig/zig
bench_test.go:145: name="fasta" compiler="zig" binarysize=2348224 version=0.15.2
BenchmarkAll/fasta:args=1000000/zig/zig-12 10 115100689 ns/op
BenchmarkAll/fasta:args=8000000/zig/zig
BenchmarkAll/fasta:args=8000000/zig/zig-12 2 949552186 ns/op
BenchmarkAll/fasta:args=1000000/rust/rustc
bench_test.go:145: name="fasta" compiler="rustc" binarysize=4348144 version=1.95.0
BenchmarkAll/fasta:args=1000000/rust/rustc-12 10 110101344 ns/op
BenchmarkAll/fasta:args=8000000/rust/rustc
BenchmarkAll/fasta:args=8000000/rust/rustc-12 2 816579526 ns/op
BenchmarkAll/fasta:args=1000000/go/go
bench_test.go:145: name="fasta" compiler="go" binarysize=2457030 version=1.25.7
BenchmarkAll/fasta:args=1000000/go/go-12 10 107901207 ns/op
BenchmarkAll/fasta:args=8000000/go/go
BenchmarkAll/fasta:args=8000000/go/go-12 2 842708221 ns/op
BenchmarkAll/fasta:args=1000000/go/tinygo
bench_test.go:145: name="fasta" compiler="tinygo" binarysize=1679592 version=0.42.0
BenchmarkAll/fasta:args=1000000/go/tinygo-12 10 100115916 ns/op
BenchmarkAll/fasta:args=8000000/go/tinygo
BenchmarkAll/fasta:args=8000000/go/tinygo-12 2 780127678 ns/op
BenchmarkAll/fasta:args=1000000/c/gcc
bench_test.go:145: name="fasta" compiler="gcc" binarysize=16280 version=15.2.1
BenchmarkAll/fasta:args=1000000/c/gcc-12 12 96286704 ns/op
BenchmarkAll/fasta:args=8000000/c/gcc
BenchmarkAll/fasta:args=8000000/c/gcc-12 2 804399738 ns/op
BenchmarkAll/fasta:args=1000000/c/clang
bench_test.go:145: name="fasta" compiler="clang" binarysize=16200 version=20.1.1
BenchmarkAll/fasta:args=1000000/c/clang-12 12 99917398 ns/op
BenchmarkAll/fasta:args=8000000/c/clang
BenchmarkAll/fasta:args=8000000/c/clang-12 2 728786948 ns/op
BenchmarkAll/n-body:args=200000/zig/zig
bench_test.go:145: name="n-body" compiler="zig" binarysize=2389576 version=0.15.2
BenchmarkAll/n-body:args=200000/zig/zig-12 172 7789949 ns/op
BenchmarkAll/n-body:args=300000/zig/zig
BenchmarkAll/n-body:args=300000/zig/zig-12 120 10300733 ns/op
BenchmarkAll/n-body:args=200000/rust/rustc
bench_test.go:145: name="n-body" compiler="rustc" binarysize=4373424 version=1.95.0
BenchmarkAll/n-body:args=200000/rust/rustc-12 120 9743532 ns/op
BenchmarkAll/n-body:args=300000/rust/rustc
BenchmarkAll/n-body:args=300000/rust/rustc-12 86 13575028 ns/op
BenchmarkAll/n-body:args=200000/go/go
bench_test.go:145: name="n-body" compiler="go" binarysize=2344462 version=1.25.7
BenchmarkAll/n-body:args=200000/go/go-12 81 14521758 ns/op
BenchmarkAll/n-body:args=300000/go/go
BenchmarkAll/n-body:args=300000/go/go-12 57 20612938 ns/op
BenchmarkAll/n-body:args=200000/go/tinygo
bench_test.go:145: name="n-body" compiler="tinygo" binarysize=1554032 version=0.42.0
BenchmarkAll/n-body:args=200000/go/tinygo-12 120 10082239 ns/op
BenchmarkAll/n-body:args=300000/go/tinygo
BenchmarkAll/n-body:args=300000/go/tinygo-12 84 14288535 ns/op
BenchmarkAll/n-body:args=200000/c/gcc
bench_test.go:145: name="n-body" compiler="gcc" binarysize=16400 version=15.2.1
BenchmarkAll/n-body:args=200000/c/gcc-12 136 8271588 ns/op
BenchmarkAll/n-body:args=300000/c/gcc
BenchmarkAll/n-body:args=300000/c/gcc-12 98 11860719 ns/op
BenchmarkAll/n-body:args=200000/c/clang
bench_test.go:145: name="n-body" compiler="clang" binarysize=16384 version=20.1.1
BenchmarkAll/n-body:args=200000/c/clang-12 136 8943665 ns/op
BenchmarkAll/n-body:args=300000/c/clang
BenchmarkAll/n-body:args=300000/c/clang-12 87 12151455 ns/op
BenchmarkAll/n-body-nosqrt:args=200000/zig/zig
bench_test.go:145: name="n-body-nosqrt" compiler="zig" binarysize=2394784 version=0.15.2
BenchmarkAll/n-body-nosqrt:args=200000/zig/zig-12 25 45025962 ns/op
BenchmarkAll/n-body-nosqrt:args=300000/zig/zig
BenchmarkAll/n-body-nosqrt:args=300000/zig/zig-12 16 64477314 ns/op
BenchmarkAll/n-body-nosqrt:args=200000/rust/rustc
bench_test.go:145: name="n-body-nosqrt" compiler="rustc" binarysize=4373760 version=1.95.0
BenchmarkAll/n-body-nosqrt:args=200000/rust/rustc-12 24 49967861 ns/op
BenchmarkAll/n-body-nosqrt:args=300000/rust/rustc
BenchmarkAll/n-body-nosqrt:args=300000/rust/rustc-12 15 73972680 ns/op
BenchmarkAll/n-body-nosqrt:args=200000/go/go
bench_test.go:145: name="n-body-nosqrt" compiler="go" binarysize=2344394 version=1.25.7
BenchmarkAll/n-body-nosqrt:args=200000/go/go-12 19 56375283 ns/op
BenchmarkAll/n-body-nosqrt:args=300000/go/go
BenchmarkAll/n-body-nosqrt:args=300000/go/go-12 13 84115020 ns/op
BenchmarkAll/n-body-nosqrt:args=200000/go/tinygo
bench_test.go:145: name="n-body-nosqrt" compiler="tinygo" binarysize=1555056 version=0.42.0
BenchmarkAll/n-body-nosqrt:args=200000/go/tinygo-12 24 47262108 ns/op
BenchmarkAll/n-body-nosqrt:args=300000/go/tinygo
BenchmarkAll/n-body-nosqrt:args=300000/go/tinygo-12 15 69850927 ns/op
BenchmarkAll/n-body-nosqrt:args=200000/c/gcc
bench_test.go:145: name="n-body-nosqrt" compiler="gcc" binarysize=16480 version=15.2.1
BenchmarkAll/n-body-nosqrt:args=200000/c/gcc-12 21 48628458 ns/op
BenchmarkAll/n-body-nosqrt:args=300000/c/gcc
BenchmarkAll/n-body-nosqrt:args=300000/c/gcc-12 15 72387650 ns/op
BenchmarkAll/n-body-nosqrt:args=200000/c/clang
bench_test.go:145: name="n-body-nosqrt" compiler="clang" binarysize=16472 version=20.1.1
BenchmarkAll/n-body-nosqrt:args=200000/c/clang-12 24 45343357 ns/op
BenchmarkAll/n-body-nosqrt:args=300000/c/clang
BenchmarkAll/n-body-nosqrt:args=300000/c/clang-12 16 66182942 ns/op
BenchmarkAll/spectral-norm:args=1000/zig/zig
bench_test.go:145: name="spectral-norm" compiler="zig" binarysize=2395664 version=0.15.2
BenchmarkAll/spectral-norm:args=1000/zig/zig-12 26 45412660 ns/op
BenchmarkAll/spectral-norm:args=2500/zig/zig
BenchmarkAll/spectral-norm:args=2500/zig/zig-12 4 269827153 ns/op
BenchmarkAll/spectral-norm:args=5500/zig/zig
BenchmarkAll/spectral-norm:args=5500/zig/zig-12 1 1294434348 ns/op
BenchmarkAll/spectral-norm:args=1000/rust/rustc
bench_test.go:145: name="spectral-norm" compiler="rustc" binarysize=4369232 version=1.95.0
BenchmarkAll/spectral-norm:args=1000/rust/rustc-12 26 40112268 ns/op
BenchmarkAll/spectral-norm:args=2500/rust/rustc
BenchmarkAll/spectral-norm:args=2500/rust/rustc-12 5 237616028 ns/op
BenchmarkAll/spectral-norm:args=5500/rust/rustc
BenchmarkAll/spectral-norm:args=5500/rust/rustc-12 1 1146370868 ns/op
BenchmarkAll/spectral-norm:args=1000/go/go
bench_test.go:145: name="spectral-norm" compiler="go" binarysize=2445722 version=1.25.7
BenchmarkAll/spectral-norm:args=1000/go/go-12 28 40958338 ns/op
BenchmarkAll/spectral-norm:args=2500/go/go
BenchmarkAll/spectral-norm:args=2500/go/go-12 4 254545702 ns/op
BenchmarkAll/spectral-norm:args=5500/go/go
BenchmarkAll/spectral-norm:args=5500/go/go-12 1 1133576823 ns/op
BenchmarkAll/spectral-norm:args=1000/go/tinygo
bench_test.go:145: name="spectral-norm" compiler="tinygo" binarysize=1656672 version=0.42.0
BenchmarkAll/spectral-norm:args=1000/go/tinygo-12 28 39752987 ns/op
BenchmarkAll/spectral-norm:args=2500/go/tinygo
BenchmarkAll/spectral-norm:args=2500/go/tinygo-12 5 235496935 ns/op
BenchmarkAll/spectral-norm:args=5500/go/tinygo
BenchmarkAll/spectral-norm:args=5500/go/tinygo-12 1 1128457952 ns/op
BenchmarkAll/spectral-norm:args=1000/c/gcc
bench_test.go:145: name="spectral-norm" compiler="gcc" binarysize=16168 version=15.2.1
BenchmarkAll/spectral-norm:args=1000/c/gcc-12 38 28843310 ns/op
BenchmarkAll/spectral-norm:args=2500/c/gcc
BenchmarkAll/spectral-norm:args=2500/c/gcc-12 6 170103765 ns/op
BenchmarkAll/spectral-norm:args=5500/c/gcc
BenchmarkAll/spectral-norm:args=5500/c/gcc-12 2 827869374 ns/op
BenchmarkAll/spectral-norm:args=1000/c/clang
bench_test.go:145: name="spectral-norm" compiler="clang" binarysize=16144 version=20.1.1
BenchmarkAll/spectral-norm:args=1000/c/clang-12 28 39267236 ns/op
BenchmarkAll/spectral-norm:args=2500/c/clang
BenchmarkAll/spectral-norm:args=2500/c/clang-12 5 255589071 ns/op
BenchmarkAll/spectral-norm:args=5500/c/clang
BenchmarkAll/spectral-norm:args=5500/c/clang-12 1 1226402364 ns/op
PASS
ok tinybench 146.996s