feat(gallery): Speed up load times and clean gallery entries#9211
feat(gallery): Speed up load times and clean gallery entries#9211richiejp wants to merge 1 commit intomudler:masterfrom
Conversation
| ) | ||
|
|
||
| // Model-level estimate result cache — keyed by model ID, avoids re-running | ||
| // the full estimation pipeline (HTTP HEAD, GGUF reads, HF API) on every |
There was a problem hiding this comment.
mmm readers are already cached in this file, I'm missing what's the issue trying to solve here? the first loading of the page is slow because it runs estimations, but if you go back to page already loaded that's not loaded from scratch- unless I'm missing something?
There was a problem hiding this comment.
I found it to be very slow and this speeds it up mainly because it does the VRAM estimation asynchronously, but yeah I got into a mess here, there is a lot of duplication of VRAM estimation and we are doing quite a bit of unnecessary work. So I've reworked all of it into a single function and replaced the TTL cache with a generational cache and stuff like this.
richiejp
force-pushed
the
chore/gallery-speed-meta
branch
2 times, most recently
from
86fda78 to
e98d30f
Compare
richiejp
force-pushed
the
chore/gallery-speed-meta
branch
from
e98d30f to
a525398
Compare
|
any chance to revive this PR before cutting a new rel ? 🫶 |
probably not unfortunately, the code changes are pretty much done, but I have to fixup at least all of the popular models in the gallery which I have an agent to do that, but it still requires some manual review. Probably we also don't want to rush that into a release |
Signed-off-by: Richard Palethorpe <io@richiejp.com>
richiejp
force-pushed
the
chore/gallery-speed-meta
branch
from
a525398 to
15d45dc
Compare
2 participants
Description
Improve the gallery load times and clean up the gallery entry meta-data, in particular the known_usecases.
clip-04-06_11-49-23.mp4
Notes for Reviewers
Signed commits