New Features and Code Optimizations#32
Open
ttlequals0 wants to merge 24 commits into
Open
Conversation
- Batch processing (BATCH_SIZE, BATCH_DELAY, ITEM_DELAY) to prevent API flooding on large libraries - Keyword prefix (KEYWORD_PREFIX) to separate TMDb keywords from real genres in Plex dropdown - Plex webhook server (WEBHOOK_ENABLED, WEBHOOK_PORT, WEBHOOK_DEBOUNCE) for real-time processing on media add events - Remove all emoji from Go source, replace with bracketed tags - Rewrite README with table of contents and accurate feature docs - Clean up CHANGELOG to remove AI writing patterns - Refactor: Clients struct, keyword cache, batch helpers, forEachLibrary, graceful shutdown, concurrency safety
Keep feature branch versions for all three conflicted files (CHANGELOG.md, README.md, processor.go). The origin/main content was the old pre-cleanup version that our branch replaces.
- Remove remaining emoji and unicode arrows from processor.go and tmdb/client.go - Expand webhook payload struct to match Plex spec - Use LibrarySectionType for correct media type resolution - Note Plex Pass requirement for webhooks in README
- Cache GetAllMovies/GetAllSeries results per client instance to eliminate thousands of redundant API calls per processing cycle - Add bidirectional title matching (either string contains the other) - Add CleanTitle matching (strips all punctuation for fuzzy matching) Fixes movies like "(500) Days of Summer" where Plex strips parens - Clear Radarr/Sonarr caches alongside keyword cache each cycle - Apply same improvements to both Radarr and Sonarr clients
Replace globe and timer emojis in verbose Plex API logging with bracketed tags [API] and [TIMING].
- Replace [ERROR] with [SKIP] for expected lookup misses (not found in Radarr/Sonarr, no TMDb ID in path). Keep [ERROR] only for real API/fetch failures. - Collapse duplicate file path iteration: check both API path match and TMDb ID regex in a single pass instead of listing paths twice. - Cap file path logging at 3 paths with "and N more" summary. - Consolidate episode fetching in TV show extraction to a single call instead of fetching twice (once for Sonarr, again for regex). - Shorter, more scannable log format for lookup results.
…ions Add build/test/deploy commands (Docker + Portainer webhook), architecture overview, coding conventions, key patterns, and gotchas.
- Create internal/version/version.go as single source of truth - Log version at startup before config loading - Tag Docker images with version (1.2.0 + latest) - Update CHANGELOG with 1.2.0 release date
feat: batch processing, keyword prefix, Plex webhooks
- Add ProcessSingleItem method that tags only the newly added item - Webhook accumulates rating keys during debounce window instead of keeping only the last one (fixes dropped items on rapid events) - ProcessSingleItem acquires the per-library mutex to prevent races with timer-triggered ProcessAllItems - ProcessSingleItem includes export logic for consistency - Remove library.on.deck event handling (fires on playback, not new media -- was triggering unnecessary reprocessing) - Falls back to full library scan when no rating key is in the payload
…ebhook-only mode ProcessSingleItem previously returned nil when the per-library mutex was held by a full scan, logging "queuing for next cycle" without any queue. Items were silently dropped. Replace the check-and-bail with a poll loop (5s interval, 2h deadline). ProcessAllItems keeps its skip-on-busy behavior so timer-driven scans do not stack; the asymmetry is documented inline. Add MOVIE_LIBRARY_EXCLUDE and TV_LIBRARY_EXCLUDE (comma-separated IDs) to skip specific libraries under *_PROCESS_ALL=true. Filter runs once at startup so both timer and webhook routing see the same set. Add WEBHOOK_ONLY=true to skip the startup full scan and periodic timer, leaving the webhook server as the sole trigger. Validated in config (requires WEBHOOK_ENABLED=true). Bumps version to 1.2.1.
fix: webhook items wait for in-flight scan; add library exclude and webhook-only mode
Plex deliveries were being rejected with bare HTTP 400s and no log output, making it impossible to tell which failure path fired (multipart parse, missing payload field, or JSON unmarshal). Each 400 branch now logs the error plus diagnostics (Content-Type, form/file part keys, payload snippet) so the root cause of real Plex failures can be diagnosed from logs alone.
Plex sends Metadata.Guid as an array of objects for items with multiple provider IDs (imdb, tmdb, tvdb). PlexWebhookPayload declared it as a string, so json.Unmarshal failed on every such payload and the handler returned 400, making labelarr ignore all real webhooks. The field was never read anywhere in the codebase; removing it lets the JSON decoder skip it regardless of shape.
fix(webhook): log reason for 400 responses
Adds a manual scan trigger so operators can force a catchup cycle without toggling WEBHOOK_ONLY. POST /scan runs a full scan; POST /scan?library=<id|name> targets a single library. Returns 202 on accept, 409 on concurrent scan, 404 on unknown library. A Scanner interface is introduced and implemented by a scanRunner in main; the periodic timer now uses the same code path.
feat(webhook): add POST /scan endpoint for manual scan triggers
Contributor
|
@nullable-eth is this still maintained? |
…kflow permissions) - plex/client.go: gate InsecureSkipVerify on new PLEX_INSECURE_SKIP_VERIFY env var (default false). Startup logs a [WARN] when the opt-in is active. Resolves go/disabled-certificate-check. - export/export.go: add safeJoin containment check and harden sanitizeFilename to reject '', '.', '..'. All filepath.Join calls onto the export root now verify the result stays inside EXPORT_LOCATION. Resolves go/path-injection. - .github/workflows/release.yml: add least-privilege top-level 'permissions: contents: read' so the check-changes job inherits a read-only token. Existing per-job elevated permissions unchanged. Resolves actions/missing-workflow-permissions. Bump to v1.3.1.
Clears 1 CRITICAL + 8 HIGH CVEs surfaced by Trivy on the prior golang:1.23-alpine / alpine:3.21 base (stdlib CVEs across crypto/tls, crypto/x509, archive/tar, archive/zip, net/url). - Dockerfile: golang:1.23-alpine -> 1.26-alpine, alpine:3.21 -> 3.22 - go.mod: go 1.21 -> 1.26 - release.yml: actions/setup-go pinned to 1.26 - .dockerignore: add .env*, *.pem, *.key, .claude, .github, CLAUDE.md, PR_DESCRIPTION.md, tmp/ to prevent accidental inclusion Post-bump Trivy scan: 0 HIGH / 0 CRITICAL on ttlequals0/labelarr:1.3.1.
….3.2 Transport failures on the Plex HTTP client bubble up Go's *url.Error, which embeds the full request URL including the token. With TLS verify now defaulting on, a misconfigured cert produces repeating error lines containing the Plex token, leaking it to stdout/Loki. - Add urlSecretRedactor regexp and redactURLSecrets helper - Add (*Client).safeDo wrapper that scrubs Do() errors before returning - Route all 9 httpClient.Do(req) call sites through safeDo - Redact url.Parse errors in UpdateMediaField/RemoveMediaFieldKeywords
fix(security): CodeQL alerts + Go/Alpine bump + Plex token redaction - v1.3.2
|
would be nice if this could be finally merged. |
Author
|
@Hustleberry for what's it's worth you can deploy this from my fork today. https://github.com/ttlequals0/labelarr |
|
thanks for your answers, as your repo does not allow questions/issues, I will abuse this one here at least for this question Does your fork really support batch processing?: [labelarr] 2026-05-07T17:24:50.130315947Z 📋 Fetching all movies from library... Dont see anything here which indicates that. Greetz |
Author
|
@Hustleberry Did you set the environment variables in your docker-compose? |
|
Ah yes, my bad was that I copied the compose in your fork, which still links to this repo. Just for you to know :) |
Fix image reference
Author
Thanks for catching that, it's been updated |
Owner
|
Sorry guys, been really busy and haven't checked back up on this. Will review and merge this tonight. |
| services: | ||
| labelarr: | ||
| image: ghcr.io/nullable-eth/labelarr:latest | ||
| image: ttlequals0/labelarr:latest |
Owner
There was a problem hiding this comment.
this should be changed back for merge
| RemoveMode: os.Getenv("REMOVE"), | ||
| TMDbReadAccessToken: os.Getenv("TMDB_READ_ACCESS_TOKEN"), | ||
| ProcessTimer: getProcessTimerFromEnv(), | ||
| PlexInsecureSkipVerify: getBoolEnvWithDefault("PLEX_INSECURE_SKIP_VERIFY", false), |
Owner
There was a problem hiding this comment.
this should be noted as a breaking change in the readme.
| }) | ||
| } | ||
|
|
||
| func appendUnique(slice []string, val string) []string { |
Owner
There was a problem hiding this comment.
if you switched this slice to a map you could eliminate this function/existence check entirely and the map is O(1) lookup instead of this O(n) lookup function.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[1.3.0] - 2026-04-12
Added
POST /scanendpoint on the webhook server for manual scan triggers.POST /scanruns a full scan across all non-excluded libraries.POST /scan?library=<id|name>scans a single library;libraryaccepts the numeric Plex section ID or a case-insensitive library
title.
202 Acceptedimmediately; the scan runs in thebackground. Returns
409 Conflictif a scan is already inprogress,
404 Not Foundif the library param does not match, and405 Method Not Allowedfor non-POST requests.WEBHOOK_ONLY=truemode — enables ad-hoc catchup scanswithout toggling environment variables.
MOVIE_LIBRARY_EXCLUDE/TV_LIBRARY_EXCLUDE: comma-separated libraryIDs to skip when
*_PROCESS_ALL=true. Excluded libraries are filteredfrom both timer-driven processing and webhook routing.
WEBHOOK_ONLY=true: skips the startup full scan and the periodic timerentirely, leaving the webhook server as the only trigger. Requires
WEBHOOK_ENABLED=true.Plex Webhook Support #22
Keyword Prefix #24
Batch Processing #26
Version Tracking
Changed