Skip to content

docs: align botasaurus + auto fallback + web release model#1136

Merged
gildesmarais merged 6 commits intomainfrom
feat/botasaurus
May 1, 2026
Merged

docs: align botasaurus + auto fallback + web release model#1136
gildesmarais merged 6 commits intomainfrom
feat/botasaurus

Conversation

@gildesmarais
Copy link
Copy Markdown
Member

@gildesmarais gildesmarais commented Apr 19, 2026

Summary

  • update Ruby gem docs to include botasaurus strategy coverage
  • simplify high-level wording to JavaScript/browser-based extraction while keeping concrete strategy details in reference pages
  • document CLI default fallback order: faraday -> botasaurus -> browserless
  • add BOTASAURUS_SCRAPER_URL guidance in strategy/CLI/troubleshooting docs

Why

This docs update tracks the Botasaurus strategy introduction in html2rss core:

and new default auto strategy:

Validation

  • npm run -s build

@gildesmarais gildesmarais self-assigned this Apr 19, 2026
@gildesmarais gildesmarais changed the title docs: align strategy docs with botasaurus-first auto fallback docs: align botasaurus + auto fallback + web release model May 1, 2026
@gildesmarais gildesmarais requested a review from Copilot May 1, 2026 09:11
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the documentation site to reflect new html2rss strategy behavior (Botasaurus + auto fallback) and the html2rss-web Docker release/tagging model, while simplifying some high-level onboarding copy.

Changes:

  • Document default auto strategy fallback order (faraday -> botasaurus -> browserless) and add BOTASAURUS_SCRAPER_URL guidance across strategy/CLI/troubleshooting docs.
  • Switch recommended html2rss-web Docker image usage from :latest to the major-version tag :1, and document the tagging scheme.
  • Simplify web-app onboarding and deployment wording; update Watchtower guidance to emphasize “updates follow the tag”.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/data/docker.ts Updates the canonical webImage tag used by generated Compose snippets to html2rss/web:1.
src/content/docs/web-application/reference/versioning-and-releases.mdx Replaces rolling-release wording with a versioned Docker tag model and recommended tag usage.
src/content/docs/web-application/reference/env-variables.mdx Clarifies BUILD_TAG/GIT_SHA as release metadata set by published images.
src/content/docs/web-application/index.mdx Simplifies overview copy and streamlines the recommended flow steps.
src/content/docs/web-application/how-to/use-automatic-feed-generation.mdx Adjusts UI flow text and reframes strategy behavior as backend-default-driven.
src/content/docs/web-application/how-to/deployment.mdx Updates deployment guidance to use html2rss/web:1 and refreshes reverse proxy / ops notes.
src/content/docs/web-application/how-to/automatic-updates.mdx Emphasizes Watchtower updates follow the selected Docker tag (major-version tag guidance).
src/content/docs/web-application/getting-started.mdx Simplifies local verification steps and reduces repeated onboarding prose.
src/content/docs/troubleshooting/troubleshooting.mdx Updates troubleshooting guidance for browser-based strategies and adds Botasaurus-related errors.
src/content/docs/ruby-gem/reference/strategy.mdx Documents auto default fallback chain and adds a Botasaurus strategy reference section.
src/content/docs/ruby-gem/reference/cli-reference.mdx Adds default auto strategy behavior, Botasaurus CLI examples, and BOTASAURUS env requirements.
src/content/docs/ruby-gem/how-to/handling-dynamic-content.mdx Generalizes “dynamic content” guidance to browser-based strategies (Browserless/Botasaurus).
src/content/docs/ruby-gem/how-to/custom-http-requests.mdx Notes request.botasaurus.* as strategy-specific request controls alongside Browserless.
src/content/docs/ruby-gem/how-to/advanced-features.mdx Updates strategy advice to “static vs browser-based extraction” framing.
src/content/docs/index.mdx Simplifies landing-page onboarding phrasing and removes some feed-directory emphasis.
src/content/docs/getting-started.mdx Updates getting-started description and adds a pointer to the Strategy reference.
src/content/docs/creating-custom-feeds.mdx Tightens copy and reframes escalation guidance to “browser-based strategy when needed”.
src/content/docs/common-use-cases.mdx Simplifies use-case narrative and replaces removed sections with a shorter “Next Steps”.
src/components/docs/DockerComposeSnippet.astro Removes explicit BUILD_TAG/GIT_SHA entries and inherits the new webImage tag in snippets.
examples/deployment/docker-compose.yml Updates the example deployment image tag from :latest to :1.

Comment thread src/content/docs/web-application/reference/versioning-and-releases.mdx Outdated
Comment on lines +51 to +56
<Code
code={`
CADDY_HOST=yourdomain.com
HTML2RSS_SECRET_KEY=<openssl rand -hex 32>
HEALTH_CHECK_TOKEN=<strong bearer token>
BROWSERLESS_IO_API_TOKEN=<browserless token>
gildesmarais and others added 2 commits May 1, 2026 11:21
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@gildesmarais gildesmarais merged commit 5fc67c3 into main May 1, 2026
5 checks passed
@gildesmarais gildesmarais deleted the feat/botasaurus branch May 1, 2026 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants