feat: Add shelf exclusion filter to BookStack connector#894
Open
vorsengineer wants to merge 1 commit intoMODSetter:mainfrom
Open
feat: Add shelf exclusion filter to BookStack connector#894vorsengineer wants to merge 1 commit intoMODSetter:mainfrom
vorsengineer wants to merge 1 commit intoMODSetter:mainfrom
Conversation
Add the ability to exclude specific BookStack shelves from indexing, allowing users to control which content gets synced into a search space. Backend: - Add get_all_shelves() and build_book_to_shelf_map() to BookStackConnector - Add excluded_shelf_ids parameter to get_all_pages() and get_pages_by_date_range() - Read BOOKSTACK_EXCLUDED_SHELF_IDS from connector config in the indexer - Add POST /api/v1/bookstack/shelves endpoint for frontend shelf picker Frontend: - Add shelf picker to bookstack-connect-form.tsx (initial setup) - Add shelf picker to bookstack-config.tsx (edit view) - Add listBookStackShelves() to connectors-api.service.ts - Shelves loaded on-demand via "Load Shelves" button after credentials entered - Checkbox UI: checked = included, unchecked = excluded - Excluded shelf IDs stored as BOOKSTACK_EXCLUDED_SHELF_IDS in connector config Fully backward compatible - existing connectors without excluded shelf IDs continue to index all pages. No database migration needed. Ref MODSetter#892 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
@vorsengineer is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel. A member of the Team first needs to authorize it. |
![recurseml[bot]](https://avatars.githubusercontent.com/in/1054195?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review by RecurseML
🔍 Review performed on 54e56e1..b8ea6e6
| Severity | Location | Issue | Delete |
|---|---|---|---|
 |
surfsense_backend/app/connectors/bookstack_connector.py:210 | KeyError in shelf map building |  |
✅ Files analyzed, no issues (4)
• surfsense_backend/app/tasks/connector_indexers/bookstack_indexer.py
• surfsense_web/components/assistant-ui/connector-popup/connect-forms/components/bookstack-connect-form.tsx
• surfsense_web/components/assistant-ui/connector-popup/connector-configs/components/bookstack-config.tsx
• surfsense_web/lib/apis/connectors-api.service.ts
Owner
|
@vorsengineer Thanks for your contributions. Can you please raise both PRs on the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
POST /api/v1/bookstack/shelvesendpoint to fetch shelves for the picker UIMotivation
BookStack organizes content in a Shelves > Books > Chapters > Pages hierarchy. Teams often have shelves that shouldn't be indexed into certain search spaces (e.g., internal/private shelves, archives). This feature gives users control over what gets synced.
FIX #892
Screenshots
The shelf picker appears in the connector setup after clicking "Load Shelves":
Changes
Backend (3 files)
bookstack_connector.py: Addedget_all_shelves(),build_book_to_shelf_map(), andexcluded_shelf_idsparameter to page fetching methodsbookstack_indexer.py: ReadsBOOKSTACK_EXCLUDED_SHELF_IDSfrom connector configsearch_source_connectors_routes.py: NewPOST /bookstack/shelvesendpointFrontend (3 files)
bookstack-connect-form.tsx: Shelf picker with "Load Shelves" buttonbookstack-config.tsx: Same shelf picker on the edit viewconnectors-api.service.ts:listBookStackShelves()methodAPI Changes
POST /api/v1/bookstack/shelves(accepts credentials, returns shelves with books)Type of Change
Testing
Quality Checklist
🤖 Generated with Claude Code
High-level PR Summary
This PR adds shelf exclusion filtering to the BookStack connector, allowing users to selectively choose which shelves to include or exclude during indexing. The implementation includes a new backend endpoint (
POST /api/v1/bookstack/shelves) to fetch available shelves, enhanced filtering logic in the connector and indexer to respect excluded shelf IDs stored in the connector configuration, and a shelf picker UI in both the connector setup and edit views. The feature is fully backward compatible with no database migrations required, as the exclusion list is stored in the existing connector config JSON field.⏱️ Estimated Review Time: 15-30 minutes
💡 Review Order Suggestion
surfsense_backend/app/routes/search_source_connectors_routes.pysurfsense_web/lib/apis/connectors-api.service.tssurfsense_backend/app/connectors/bookstack_connector.pysurfsense_backend/app/tasks/connector_indexers/bookstack_indexer.pysurfsense_web/components/assistant-ui/connector-popup/connect-forms/components/bookstack-connect-form.tsxsurfsense_web/components/assistant-ui/connector-popup/connector-configs/components/bookstack-config.tsx