sd: sync to master-560-e8323ca

[wbruna]

45 files changed, 9892 insertions(+), 9131 deletions(-)

And the only new functionality is this:

diff --git a/src/z_image.hpp b/src/z_image.hpp
index 9f17890..53a7cf8 100644
--- a/src/z_image.hpp
+++ b/src/z_image.hpp
@@ -131,7 +131,7 @@ namespace ZImage {
 
             auto x1 = w1->forward(ctx, x);
             auto x3 = w3->forward(ctx, x);
-            x       = ggml_mul(ctx->ggml_ctx, ggml_silu(ctx->ggml_ctx, x1), x3);
+            x       = ggml_swiglu_split(ctx->ggml_ctx, x1, x3);
             x       = w2->forward(ctx, x);
 
             return x;

Seems appropriate for an April 1st 🙂

[wbruna]

Updated to master-551-99c1de3, which included the Euler Ancestral algorithm for flow models (#2058). Most other changes are refactors or sd-server and sd-cli features.

I've sidestepped a build issue by adding vendor/nlohmann as an include path when building sd.cpp sources. If there aren't strong reasons to not do it this way, I'll clean up the other changed includes afterwards.

[LostRuins]

oh my this is massive.

but i trust when you say it's mostly just linting right? i'll just do a few quick tests and then we can probably merge before the next release

[wbruna]

but i trust when you say it's mostly just linting right?

Not quite. Part of it is things like removing struct from types, but the really big change was reorganizing the code to use a new Tensor class (replacing loops with higher-level operations), That commit alone changed ~7k lines. And we get that VAE reorg with it too, because the related VAE-on-CPU fix is entangled with it.

The jump from master-540 to master-551 is less disruptive: mostly replacing the big case labels on the denoiser with a function dispatch (and I did that exact refactor a few days ago, so I understand what it is doing 🙂). But if you prefer, we can get just master-540 for now, since it had more time to be tested upstream.

Oh, and master-552 will replace avi_writer.h 🙁

[wbruna]

Tested so far: Z-Image Turbo, Flux.2 Klein 9b, Qwen Image Edit (all working with Euler Ancestral); SD1.5, SDXL, SDXL Turbo; preloaded and dynamic LoRAs; and TAESD.

[LostRuins]

Great. I'll look into this once I get the gemma4 release out and good since people are chomping at the bit

[wbruna]

Hold on, the image mask is being ignored.

Edit: working on master-529 (current), broken on master-540.

[wbruna]

I think I was able to fix it: leejet/stable-diffusion.cpp#1390 .

I've applied a similar local fix for now (f4a1997), in case you want to merge this before upstream.

Edit: another fix got merged upstream; I'll overwrite mine in the next sync.

[wbruna]

Updated to master-555-7397dda, getting the image mask fix. Since the main API hasn't changed, I've just kept the local avi_writer.h for now; we can refactor it after merging this PR.

[LostRuins]

now that gemma 4 is finally mostly stable ill try to get this merged in the next 1 or 2 days

[wbruna]

Rebased, and updated to master-558-8afbeb6, which only changed examples code.

[wbruna]

Updated to master-560-e8323ca, getting a scheduler fix and support for the Flux.2 small decoder. Both seem to be working fine.

[LostRuins]

Sorry for the delays.
Tested SD1.5, Qwen Edit, Qwen Image and Z-Image, all ok. merging.