feat: connect orchestrator and frontend

[Scra3]

MAIN BRANCH TO INTRODUCE WORKFLOW EXECUTOR.

Note

Add @forestadmin/workflow-executor package connecting the orchestrator and frontend

• Introduces a new @forestadmin/workflow-executor package with a CLI (forest-workflow-executor), an HTTP server with JWT auth, and a polling runner that fetches and executes workflow steps from the Forest server.
• Implements step executors for all supported step types: condition, read-record, update-record, trigger-action, load-related-record, MCP, and guidance — each with idempotency, activity logging, and AI-driven field/record/action selection.
• Adds adapters for all ports: AgentClientAgentPort, ForestServerWorkflowPort, AiClientAdapter, ServerAiAdapter, database and in-memory RunStore, and ForestadminClientActivityLogPort with retry and drain semantics.
• Adds proxy routes to the agent (GET/_internal/workflow-executions/:runId and POST .../trigger) in workflow-executor-proxy.ts, gated on the new workflowExecutorUrl agent option.
• Extends @forestadmin/ai-proxy with AiClient, createBaseChatModel, getAiConfiguration, and mcpServerId propagation across MCP and Forest integration tool factories.
• Risk: The new package requires FOREST_ENV_SECRET, FOREST_AUTH_SECRET, AGENT_URL, and DATABASE_URL at startup; missing or invalid values throw a ConfigurationError and prevent the process from starting.

^{Macroscope summarized 82dd057.}

[macroscopeapp]

🟢 Low adapters/run-to-available-step-mapper.ts:64

The guidance branch converts 'awaiting-input' status to 'success' because it uses ctx.status === 'error' ? 'error' : 'success' instead of toRecordStatus. Since GuidanceStepOutcomeSchema explicitly allows 'awaiting-input' via RecordStepStatusSchema, this valid state is incorrectly overwritten. Consider using toRecordStatus(ctx.status) to preserve all valid status values.

Suggested change

	const status: GuidanceStepOutcome['status'] = ctx.status === 'error' ? 'error' : 'success';
	return { type: 'guidance', ...baseFromCtx, status };
	}
	if (outcomeType === 'guidance') {
	const status: GuidanceStepOutcome['status'] = toRecordStatus(ctx.status);
	return { type: 'guidance', ...baseFromCtx, status };
	}

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file packages/workflow-executor/src/adapters/run-to-available-step-mapper.ts around lines 64-67:

The `guidance` branch converts `'awaiting-input'` status to `'success'` because it uses `ctx.status === 'error' ? 'error' : 'success'` instead of `toRecordStatus`. Since `GuidanceStepOutcomeSchema` explicitly allows `'awaiting-input'` via `RecordStepStatusSchema`, this valid state is incorrectly overwritten. Consider using `toRecordStatus(ctx.status)` to preserve all valid status values.

Evidence trail:
packages/workflow-executor/src/adapters/run-to-available-step-mapper.ts lines 30-35 (toRecordStatus helper), line 64 (guidance branch using ternary instead of toRecordStatus); packages/workflow-executor/src/types/validated/step-outcome.ts lines 11 (RecordStepStatusSchema allows 'awaiting-input'), lines 57-63 (GuidanceStepOutcomeSchema uses RecordStepStatusSchema); packages/workflow-executor/src/executors/guidance-step-executor.ts line 14 (executor produces 'awaiting-input' for guidance steps)

[macroscopeapp]

🟡 Medium executors/trigger-record-action-step-executor.ts:185

saveFrontendResult spreads existingExecution without setting idempotencyPhase: 'done', so after Branch C completes the step data lacks that field. If the executor restarts, checkIdempotency() (line 53) fails to recognize completion because idempotencyPhase is undefined, causing the step to incorrectly re-enter the confirmation flow.

    await this.context.runStore.saveStepExecution(this.context.runId, {
      ...existingExecution,
      type: 'trigger-action',
      stepIndex: this.context.stepIndex,
      executionParams: { displayName, name },
      executionResult: { success: true, actionResult },
      selectedRecordRef,
+      idempotencyPhase: 'done',
    });

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file packages/workflow-executor/src/executors/trigger-record-action-step-executor.ts around lines 185-192:

`saveFrontendResult` spreads `existingExecution` without setting `idempotencyPhase: 'done'`, so after Branch C completes the step data lacks that field. If the executor restarts, `checkIdempotency()` (line 53) fails to recognize completion because `idempotencyPhase` is undefined, causing the step to incorrectly re-enter the confirmation flow.

Evidence trail:
packages/workflow-executor/src/executors/trigger-record-action-step-executor.ts lines 48-62 (checkIdempotency), 134-142 (Branch C save without idempotencyPhase), 178-195 (saveFrontendResult missing idempotencyPhase:'done'), 146-174 (executeOnExecutor correctly sets idempotencyPhase). packages/workflow-executor/src/executors/update-record-step-executor.ts lines 205-239 (resolveAndUpdate sets idempotencyPhase:'done' even in confirmation flow). packages/workflow-executor/src/types/step-execution-data.ts lines 13-14 (MutatingStepExecutionData.idempotencyPhase definition), lines 76-83 (TriggerRecordActionStepExecutionData extends MutatingStepExecutionData). packages/workflow-executor/src/executors/base-step-executor.ts lines 194-234 (findPendingExecution, patchAndReloadPendingData), lines 238-262 (handleConfirmationFlow).

[macroscopeapp]

🟢 Low adapters/console-logger.ts:5

In error, warn, and info, the spread order { message, timestamp, ...context } allows context to overwrite message and timestamp when the caller passes those keys. This corrupts the log output by replacing the actual message or timestamp with values from context. Reorder the spread to { ...context, message, timestamp } to protect the primary log fields.

-      console.error(JSON.stringify({ message, timestamp: new Date().toISOString(), ...context }));
+      console.error(JSON.stringify({ ...context, message, timestamp: new Date().toISOString() }));

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file packages/workflow-executor/src/adapters/console-logger.ts around line 5:

In `error`, `warn`, and `info`, the spread order `{ message, timestamp, ...context }` allows `context` to overwrite `message` and `timestamp` when the caller passes those keys. This corrupts the log output by replacing the actual message or timestamp with values from context. Reorder the spread to `{ ...context, message, timestamp }` to protect the primary log fields.

Evidence trail:
packages/workflow-executor/src/adapters/console-logger.ts lines 4-14 at REVIEWED_COMMIT — `...context` is spread last in the object literal on lines 5, 9, and 13, meaning any `message` or `timestamp` key in the `context: Record<string, unknown>` parameter overwrites the primary fields. JavaScript spec (ES2018+) defines that later spread properties overwrite earlier ones with the same key.

[macroscopeapp]

🟢 Low adapters/with-retry.ts:14

When fn throws null or undefined, the destructuring const { status } = err as { status?: number } throws a TypeError before line 17 can check the status. This crashes the retry mechanism instead of treating the error as non-retryable. Consider adding a null guard before destructuring, or use optional chaining to safely access status.

 function isRetryable(err: unknown): boolean {
-  const { status } = err as { status?: number };
+  if (err == null) return false;
+  const status = (err as { status?: number }).status;
 
   return typeof status === 'number' && RETRYABLE_STATUS.has(status);
 }

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file packages/workflow-executor/src/adapters/with-retry.ts around lines 14-18:

When `fn` throws `null` or `undefined`, the destructuring `const { status } = err as { status?: number }` throws a `TypeError` before line 17 can check the status. This crashes the retry mechanism instead of treating the error as non-retryable. Consider adding a null guard before destructuring, or use optional chaining to safely access `status`.

Evidence trail:
packages/workflow-executor/src/adapters/with-retry.ts lines 14-18 at REVIEWED_COMMIT: `isRetryable` destructures `err` without null guard. JavaScript behavior: `const { x } = null` throws TypeError. Callers at packages/workflow-executor/src/adapters/forest-server-workflow-port.ts:231, packages/workflow-executor/src/adapters/forestadmin-client-activity-log-port.ts:26,58,80.

[macroscopeapp]

🟡 Medium src/ai-client.ts:39

If newClient.loadTools() throws, the McpClient instance created on line 44 is never disposed. Since this.mcpClient is only assigned after loadTools() succeeds, a failed call leaves newClient dangling with open connections, causing a resource leak. Consider wrapping loadTools() in a try/finally to ensure cleanup on failure.

  async loadRemoteTools(
    mcpConfig: McpConfiguration,
  ): Promise<Awaited<ReturnType<McpClient['loadTools']>>> {
    await this.disposeMcpClient('Error closing previous MCP connection');

    const newClient = new McpClient(mcpConfig, this.logger);
+    try {
      const tools = await newClient.loadTools();
      this.mcpClient = newClient;

      return tools;
+    } catch (error) {
+      await newClient.dispose();
+      throw error;
+    }
  }

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file packages/ai-proxy/src/ai-client.ts around lines 39-49:

If `newClient.loadTools()` throws, the `McpClient` instance created on line 44 is never disposed. Since `this.mcpClient` is only assigned after `loadTools()` succeeds, a failed call leaves `newClient` dangling with open connections, causing a resource leak. Consider wrapping `loadTools()` in a try/finally to ensure cleanup on failure.

Evidence trail:
packages/ai-proxy/src/ai-client.ts lines 39-49 (REVIEWED_COMMIT): loadRemoteTools creates newClient on line 44, calls loadTools() on line 45, assigns to this.mcpClient on line 46. No try/finally wrapping.
packages/ai-proxy/src/mcp-client.ts lines 24-37 (REVIEWED_COMMIT): McpClient constructor creates MultiServerMCPClient instances stored in this.mcpClients.
packages/ai-proxy/src/mcp-client.ts lines 95-118 (REVIEWED_COMMIT): dispose() method calls client.close() on each MultiServerMCPClient - explicit cleanup needed.
packages/ai-proxy/src/ai-client.ts lines 55-66 (REVIEWED_COMMIT): disposeMcpClient only disposes this.mcpClient, not a local newClient variable.

[macroscopeapp]

🟢 Low adapters/forestadmin-client-activity-log-port.ts:77

In markFailed, the errorMessage parameter is accepted but never passed to updateActivityLogStatus. The status is set to 'failed' without persisting the error context to the audit log, so the failure reason is lost. The message is only used for local logging when the update itself fails.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file packages/workflow-executor/src/adapters/forestadmin-client-activity-log-port.ts around line 77:

In `markFailed`, the `errorMessage` parameter is accepted but never passed to `updateActivityLogStatus`. The status is set to `'failed'` without persisting the error context to the audit log, so the failure reason is lost. The message is only used for local logging when the update itself fails.

Evidence trail:
packages/workflow-executor/src/adapters/forestadmin-client-activity-log-port.ts lines 77-98 (markFailed method), packages/workflow-executor/src/ports/activity-log-port.ts line 20 (ActivityLogPort interface), packages/forestadmin-client/src/types.ts lines 254-258 (UpdateActivityLogStatusParams - no errorMessage field), packages/forestadmin-client/src/activity-logs/index.ts lines 97-107 (updateActivityLogStatus implementation), packages/forestadmin-client/CHANGELOG.md line 43 (intentional removal of errorMessage from updateActivityLogStatus)