feat: adding project-discovery-worker (CM-949)#3832
Conversation
ulemons
changed the title
There was a problem hiding this comment.
Pull request overview
Introduces a new Temporal worker app (automatic_projects_discovery_worker) and adds initial database schema to support automatic project discovery/evaluation.
Changes:
- Added a new worker service with Temporal workflow/activity scaffolding plus a schedule that registers a cron-based Temporal Schedule.
- Added Docker and compose definitions to run the worker in the existing services stack.
- Added DB migrations for
projectCatalogandevaluatedProjectstables.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| services/apps/automatic_projects_discovery_worker/tsconfig.json | Adds TS config for the new worker package. |
| services/apps/automatic_projects_discovery_worker/src/workflows/discoverProjects.ts | Introduces a placeholder Temporal workflow that calls a logging activity. |
| services/apps/automatic_projects_discovery_worker/src/workflows.ts | Exports the worker’s workflows for registration. |
| services/apps/automatic_projects_discovery_worker/src/schedules/scheduleProjectsDiscovery.ts | Registers a Temporal Schedule to trigger the workflow on a cron. |
| services/apps/automatic_projects_discovery_worker/src/main.ts | Worker bootstrap: init, schedule registration, start. |
| services/apps/automatic_projects_discovery_worker/src/activities/activities.ts | Placeholder activity implementation (logs a run). |
| services/apps/automatic_projects_discovery_worker/src/activities.ts | Barrel export for activities registration. |
| services/apps/automatic_projects_discovery_worker/package.json | Adds the new worker package and scripts. |
| scripts/services/docker/Dockerfile.automatic_projects_discovery_worker | Adds container build/run definition for the worker. |
| scripts/services/automatic-projects-discovery-worker.yaml | Adds docker-compose definitions for running the worker (prod + dev). |
| scripts/builders/automatic-projects-discovery-worker.env | Adds builder configuration for publishing/building the worker image. |
| pnpm-lock.yaml | Adds lockfile entries for the new workspace package (but currently includes an extra incorrect importer). |
| backend/src/database/migrations/V1770653666__add-automatic_projects_discovery-tables.sql | Adds tables/indexes for project discovery catalog + evaluations. |
| backend/src/database/migrations/U1770653666__add-automatic_projects_discovery-tables.sql | Adds down migration to drop the tables/indexes. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| version: 3.3.3 | ||
|
|
||
| services/apps/automatic_project_discovery_worker: | ||
| dependencies: | ||
| '@crowd/archetype-standard': | ||
| specifier: workspace:* | ||
| version: link:../../archetypes/standard | ||
| '@crowd/archetype-worker': | ||
| specifier: workspace:* | ||
| version: link:../../archetypes/worker | ||
| '@crowd/common': | ||
| specifier: workspace:* | ||
| version: link:../../libs/common | ||
| '@crowd/common_services': | ||
| specifier: workspace:* | ||
| version: link:../../libs/common_services | ||
| '@crowd/data-access-layer': | ||
| specifier: workspace:* | ||
| version: link:../../libs/data-access-layer | ||
| '@crowd/logging': | ||
| specifier: workspace:* | ||
| version: link:../../libs/logging | ||
| '@crowd/redis': | ||
| specifier: workspace:* | ||
| version: link:../../libs/redis | ||
| '@crowd/temporal': | ||
| specifier: workspace:* | ||
| version: link:../../libs/temporal | ||
| '@crowd/types': | ||
| specifier: workspace:* | ||
| version: link:../../libs/types | ||
| '@temporalio/activity': | ||
| specifier: ~1.11.8 | ||
| version: 1.11.8 | ||
| '@temporalio/client': | ||
| specifier: ~1.11.8 | ||
| version: 1.11.8 | ||
| '@temporalio/workflow': | ||
| specifier: ~1.11.8 | ||
| version: 1.11.8 | ||
| tsx: | ||
| specifier: ^4.7.1 | ||
| version: 4.7.3 | ||
| typescript: | ||
| specifier: ^5.6.3 | ||
| version: 5.6.3 | ||
| devDependencies: | ||
| '@types/node': | ||
| specifier: ^20.8.2 | ||
| version: 20.12.7 | ||
| nodemon: | ||
| specifier: ^3.0.1 | ||
| version: 3.1.0 | ||
|
|
||
| services/apps/automatic_projects_discovery_worker: | ||
| dependencies: | ||
| '@crowd/archetype-standard': | ||
| specifier: workspace:* | ||
| version: link:../../archetypes/standard | ||
| '@crowd/archetype-worker': | ||
| specifier: workspace:* | ||
| version: link:../../archetypes/worker | ||
| '@crowd/common': | ||
| specifier: workspace:* | ||
| version: link:../../libs/common | ||
| '@crowd/common_services': |
There was a problem hiding this comment.
pnpm-lock.yaml contains two importers for this new app: services/apps/automatic_projects_discovery_worker (correct) and services/apps/automatic_project_discovery_worker (missing the s in projects). The latter path doesn’t exist in the repo and will leave the lockfile in an inconsistent state (pnpm will warn/error depending on settings). Please regenerate or edit the lockfile so only the correct importer remains.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| nodemon: | ||
| specifier: ^3.0.1 | ||
| version: 3.1.0 | ||
|
|
There was a problem hiding this comment.
Stale workspace importer in lockfile
Medium Severity
pnpm-lock.yaml adds an importer for services/apps/automatic_project_discovery_worker, but that workspace path is not present while the real app is services/apps/automatic_projects_discovery_worker. This leaves a stale duplicate importer in the lockfile, creating an inconsistent workspace definition that can destabilize pnpm install --frozen-lockfile behavior and dependency maintenance.
| svc.log.info('Schedule already registered in Temporal.') | ||
| svc.log.info('Configuration may have changed since. Please make sure they are in sync.') | ||
| } else { | ||
| throw new Error(err) |
There was a problem hiding this comment.
throw new Error(err) loses original error context
Low Severity
throw new Error(err) wraps the caught err (likely already an Error) inside a new Error, which discards the original stack trace, error type, and any custom properties. The resulting error message will be the stringified version of the original error (e.g., "Error: ...") rather than the original message. Rethrowing err directly would preserve all original context, making debugging much easier.
2 participants
Summary
This PR introduces the basic structure for the new worker.
The implementation includes the foundational setup required to run the worker, without full business logic.
Included
Note
Medium Risk
Adds new database tables/migrations and deployable worker infrastructure; while the worker logic is minimal, schema changes and scheduled execution can affect deployments and runtime behavior if misconfigured.
Overview
Introduces a new
automatic_projects_discovery_workerservice with Docker/compose wiring and a minimal Temporal schedule + workflow that runs on a configurable cron (CROWD_AUTOMATIC_PROJECTS_DISCOVERY_CRON) and currently only logs a successful run.Adds DB migrations creating
projectCatalogandevaluatedProjectstables (with supporting indexes and a cascading FK) to persist discovered project candidates and evaluation results, plus the corresponding down migration to drop them. Updatespnpm-lock.yamlto include the new workspace package/dependency graph.Written by Cursor Bugbot for commit e669595. This will update automatically on new commits. Configure here.