Skip to content

feat: Package-level unified metadata publishing mechanism (publishPackage)#825

Merged
hotlong merged 7 commits intomainfrom
copilot/implement-package-metadata-publishing
Feb 27, 2026
Merged

feat: Package-level unified metadata publishing mechanism (publishPackage)#825
hotlong merged 7 commits intomainfrom
copilot/implement-package-metadata-publishing

Conversation

Copy link
Contributor

Copilot AI commented Feb 27, 2026

Metadata items have cross-dependencies (Interface → View → Object → Field → Flow). Publishing individually risks broken references for end users. This implements atomic package-level publishing aligned with Salesforce 2GP / Power Platform Solution patterns.

Protocol (packages/spec)

  • MetadataRecordSchema — 3 new optional fields: publishedDefinition, publishedAt, publishedBy
  • PackagePublishResultSchema — new Zod schema for publish results
  • IMetadataService — 3 new optional methods on the contract

Runtime (packages/metadata)

  • MetadataManager.publishPackage() — schema validation → dependency validation → snapshot → version++ → state=active
    • Dependency validation: verifies all referenced items (via dependency tracking) exist either in the same package or are already published
  • MetadataManager.revertPackage() — restore all items from published snapshots
  • MetadataManager.getPublished() — return published definition, fallback to current

API (packages/runtime)

  • POST /packages/:id/publish and POST /packages/:id/revert on handlePackages()
  • GET /metadata/:type/:name/published on handleMetadata()
  • Routes registered in dispatcher-plugin.ts

Usage

// Designer edits (draft, not visible to end users)
await metadata.register('object', 'opportunity', { ...def, packageId: 'crm', state: 'draft' });

// Atomic publish — all items snapshot + activate
const result = await metadata.publishPackage('crm', { publishedBy: 'admin' });
// { success: true, version: 2, itemsPublished: 5 }

// End users see published version
const published = await metadata.getPublished('object', 'opportunity');

// Discard unpublished edits
await metadata.revertPackage('crm');

Tests

27 new tests across spec (6), metadata (15), runtime (6). All existing tests pass.

  • publishPackage(): normal publish, version increment, empty package, schema validation failure, dependency not found, dependency in same package (pass), dependency already published (pass), skip validation
  • revertPackage(): normal revert, non-existent package, never-published error
  • getPublished(): published returns snapshot, unpublished returns definition, non-existent returns undefined
  • Integration: edit → publish → edit → revert cycle
  • HTTP endpoints: publish/revert/getPublished with service + broker fallback

Docs

Updated metadata-service.mdx, packages/metadata/README.md, ROADMAP.md.

Original prompt

This section details on the original issue you should resolve

<issue_title>[协议+Runtime] Package 级统一元数据发布机制 (publishPackage)</issue_title>
<issue_description>## 背景

基于对 Airtable Interface Designer、Salesforce 2GP Package、Power Platform Solution、Retool/Appsmith 等平台的分析(参见 #823),我们确定:

发布粒度应该是 Package(解决方案),而不是单个元数据项。

为什么不是按单个元数据发布?

元数据之间有依赖关系:Interface → View → Object → Field → Flow。如果只发布了 Interface 但没发布它引用的新 Object 字段,终端用户看到的界面会报错。

Salesforce 和 Power Platform 的做法是:整个 Package/Solution 一起发布,要么全部上线,要么全部不上。

行业对标

平台 发布粒度 机制
Salesforce 2GP Package(整包) sfdx force:package:version:create
Power Platform Solution(整包) Export Managed Solution → Import
Retool 单个 App Versioned release
Airtable 单个 Interface Publish button

ObjectStack 定位企业级 → 采用 Salesforce/Power Platform 的 Package 级发布


现有基础设施

已有的组件已经覆盖了 80%:

现有组件 作用 状态
MetadataRecordSchema (metadata-persistence.zod.ts) 存储所有元数据,含 state: draft/active/archived ✅ 已有
MetadataStateSchema 生命周期状态枚举 ✅ 已有
IMetadataService (metadata-service.ts) CRUD 合约 ✅ 已有
MetadataManager 实现类 ✅ 已有
DeployBundleSchema (deploy-bundle.zod.ts) 批量部署 ✅ 已有
InstalledPackageSchema (package-registry.zod.ts) 包安装状态 ✅ 已有
package_id 字段 on sys_metadata 元数据归属包 ✅ 已有

需要做的事情

A. 协议层 (packages/spec)

A1. 扩展 MetadataRecordSchema — 加 3 个字段

// packages/spec/src/system/metadata-persistence.zod.ts
// 在现有 MetadataRecordSchema 上补充:

version: z.number().int().min(0).default(0)
  .describe('Version counter, increments on each package publish'),

published_definition: z.unknown().optional()
  .describe('Snapshot of the last published definition'),

published_at: z.string().datetime().optional()
  .describe('When this metadata was last published'),

published_by: z.string().optional()
  .describe('Who published this version'),

A2. 扩展 IMetadataService 合约 — 加 3 个方法

// packages/spec/src/contracts/metadata-service.ts

/**
 * Publish an entire package:
 * 1. Validate all draft items (dependency check)
 * 2. Snapshot all items in the package
 * 3. Increment package version
 * 4. Set all items state → active
 */
publishPackage(packageId: string, options?: {
  changeNote?: string;
  publishedBy?: string;
  validate?: boolean;
}): Promise<PackagePublishResult>;

/**
 * Revert entire package to last published state
 */
revertPackage(packageId: string): Promise<void>;

/**
 * Get the published version of any metadata item (for runtime serving)
 * Returns published_definition if exists, else current definition
 */
getPublished(type: string, name: string): Promise<unknown | undefined>;

A3. 定义 PackagePublishResult 类型

// packages/spec/src/system/metadata-persistence.zod.ts 或新文件

export const PackagePublishResultSchema = z.object({
  success: z.boolean(),
  packageId: z.string(),
  version: z.number().int(),
  publishedAt: z.string().datetime(),
  itemsPublished: z.number().int(),
  validationErrors: z.array(z.object({
    type: z.string(),
    name: z.string(), 
    message: z.string(),
  })).optional(),
});

B. Runtime 层 (packages/metadata)

B1. MetadataManager 实现 publishPackage()

  • 查询 package_id === packageId 的所有元数据
  • 依赖校验:检查新引用的 View/Object 是否在同一个包或已发布的包中
  • Schema 校验:所有 draft 元数据格式合法
  • 快照:published_definition = clone(definition) for each item
  • 版本号递增
  • 状态:所有 draftactive

B2. MetadataManager 实现 revertPackage()

  • 查询包下所有元数据
  • definition = clone(published_definition) for each item
  • state → active(丢弃未发布修改)

B3. MetadataManager 实现 getPublished()

  • 返回 published_definition if exists
  • Falls back to definition if never published

C. API 层 (packages/runtime)

C1. HttpDispatcher 加 publish 端点

POST /api/v1/packages/:packageId/publish    → publishPackage()
POST /api/v1/packages/:packageId/revert     → revertPackage()
GET  /api/v1/metadata/:type/:name/published → getPublished()

D. 测试

  • publishPackage() 单元测试:正常发布、依赖校验失败、空包
  • revertPackage() 单元测试:正常回滚、从未发布过时报错
  • getPublished() 单元测试:已发布返回快照、未发布返回 definition
  • 集成测试:编辑 → 发布 → 编辑 → 回滚 → 终端用户看到的版本正确

E. 文档 & ROADMAP

  • 更新 packages/metadata/README.md 加 Package Publish 章节
  • 更新 ROADMAP.md
  • 更新 content/docs/guides/contracts/metadata-service.mdx 加 publishPackage 文档

数据流

设计者日常编辑(自动保存到 draft)
├── 改了 Object "opportunity" 加了个字段
├── 改了 View "opp_list" 显示新字段  
├── 改了 Interface "sales_dashboard" 布局
│
▼  所有改动都在 draft,终端用户看不到
│
▼  [点击 Publish Package "crm"]
│
├── 1. 依赖校验:新 Interface 引用的 View/Object 都在这个包里吗?✓
├── 2. Schema 校验:所有 draft 元数据格式合法吗?✓  
├── 3. 快照:crm 包下所有元数据 → published_definition = clone(definition)
├── 4. 版本号:crm package version++
├── 5. 状态:所有 draft → active
│
▼  终端用户立刻看到所有变更,且保证一致性

工作量估计

改动 工作量
`MetadataRecordS...

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

@vercel
Copy link

vercel bot commented Feb 27, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
objectstack-play Ready Ready Preview, Comment Feb 27, 2026 7:14am
spec Ready Ready Preview, Comment Feb 27, 2026 7:14am

Request Review

Copilot AI and others added 2 commits February 27, 2026 06:23
…s, PackagePublishResultSchema, IMetadataService methods

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
… in MetadataManager with tests

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Copilot AI and others added 3 commits February 27, 2026 06:27
…Dispatcher and dispatcher-plugin

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
…EADME.md, and ROADMAP.md

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
… for deep cloning

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Copilot AI changed the title [WIP] Add unified metadata publishing mechanism for packages feat: Package-level unified metadata publishing mechanism (publishPackage) Feb 27, 2026
@hotlong
Copy link
Contributor

hotlong commented Feb 27, 2026

@copilot 仔细评估是否要完成的任务中要求的所有开发

…eferenced items exist in package or are already published

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 27, 2026

@copilot 仔细评估是否要完成的任务中要求的所有开发

Completed a thorough audit against all issue requirements. Here's the status:

A. Protocol (packages/spec) — ✅ All done

  • A1. MetadataRecordSchema + 3 fields (publishedDefinition, publishedAt, publishedBy) with .describe() and .optional()
  • A2. IMetadataService + 3 methods (publishPackage, revertPackage, getPublished)
  • A3. PackagePublishResultSchema with all fields

B. Runtime (packages/metadata) — ✅ All done

  • B1. publishPackage(): schema validation + dependency validation (b7b8c3d adds cross-reference check — items must be in same package or already published) + snapshot + version++ + state→active
  • B2. revertPackage(): restore from published snapshots
  • B3. getPublished(): return snapshot or fallback to current definition

C. API (packages/runtime) — ✅ All done

  • POST /packages/:id/publish, POST /packages/:id/revert, GET /metadata/:type/:name/published
  • Registered in both HttpDispatcher and dispatcher-plugin.ts

D. Tests — ✅ All done (27 new tests)

  • publishPackage(): normal publish, version increment, empty package, schema validation failure, dependency not found, dependency in same package, dependency already published, skip validation
  • revertPackage(): normal revert, non-existent package, never-published error
  • getPublished(): published returns snapshot, unpublished returns definition, non-existent returns undefined
  • Integration: edit → publish → edit → revert cycle
  • HTTP endpoints: publish/revert/getPublished with service + broker fallback

E. Docs & ROADMAP — ✅ All done

  • metadata-service.mdx, packages/metadata/README.md, ROADMAP.md

The dependency validation was the main gap I found and fixed in b7b8c3d.

@hotlong hotlong marked this pull request as ready for review February 27, 2026 07:37
Copilot AI review requested due to automatic review settings February 27, 2026 07:37
@hotlong hotlong merged commit 33899db into main Feb 27, 2026
4 checks passed
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements a package-level metadata publishing workflow across the spec contract, metadata runtime manager, and runtime HTTP surface so dependent metadata can be published/reverted atomically and consumers can fetch the last published snapshot.

Changes:

  • Extend the spec metadata persistence model with publishedDefinition/publishedAt/publishedBy and add PackagePublishResult + new IMetadataService contract methods.
  • Add publishPackage(), revertPackage(), and getPublished() to MetadataManager with validation + snapshotting behavior, plus corresponding tests.
  • Expose publish/revert and getPublished behaviors through runtime HTTP dispatching and update docs/roadmap.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
packages/spec/src/system/metadata-persistence.zod.ts Adds published snapshot fields to MetadataRecordSchema and introduces PackagePublishResultSchema.
packages/spec/src/system/metadata-persistence.test.ts Adds tests for new publishing fields and publish result schema validation.
packages/spec/src/contracts/metadata-service.ts Extends IMetadataService with optional package publish/revert/getPublished methods.
packages/spec/src/contracts/metadata-service.test.ts Updates contract tests to include the new optional methods.
packages/metadata/src/metadata-manager.ts Implements publishPackage, revertPackage, and getPublished in the runtime metadata manager.
packages/metadata/src/metadata-service.test.ts Adds unit + integration-style tests for package publish/revert/getPublished behavior.
packages/runtime/src/http-dispatcher.ts Adds HTTP dispatch support for publish/revert and .../published retrieval.
packages/runtime/src/http-dispatcher.test.ts Adds tests for the new dispatcher endpoints and broker fallbacks.
packages/runtime/src/dispatcher-plugin.ts Registers new package publish/revert HTTP routes in the dispatcher plugin.
packages/metadata/README.md Documents the new package publishing APIs.
content/docs/guides/contracts/metadata-service.mdx Adds documentation for publish/revert/getPublished and REST endpoints.
ROADMAP.md Marks package-level metadata publishing as completed and references follow-up work.

Comment on lines +344 to +352
async publishPackage(packageId: string, options?: {
changeNote?: string;
publishedBy?: string;
validate?: boolean;
}): Promise<PackagePublishResult> {
const now = new Date().toISOString();
const shouldValidate = options?.validate !== false;
const publishedBy = options?.publishedBy;

Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

publishPackage() accepts changeNote in options but never uses or persists it. If it’s intentionally not supported yet, consider removing it from the options (or explicitly documenting it as not implemented); otherwise persist it as part of the publish audit trail so callers don’t assume it’s recorded.

Copilot uses AI. Check for mistakes.
|:---|:---|:---|
| `POST` | `/packages/:id/publish` | Publish a package |
| `POST` | `/packages/:id/revert` | Revert a package to last published state |
| `GET` | `/metadata/:type/:name/published` | Get published version of a metadata item |
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs list REST endpoints under /metadata/..., but the runtime/adapters in this repo expose metadata routes under /meta (e.g. /api/v1/meta/...). To avoid sending users to non-existent endpoints, update these paths to the actual /meta/:type/:name/published route (and keep it consistent with the rest of the docs).

Suggested change
| `GET` | `/metadata/:type/:name/published` | Get published version of a metadata item |
| `GET` | `/meta/:type/:name/published` | Get published version of a metadata item |

Copilot uses AI. Check for mistakes.
Comment on lines +548 to +562
// POST /packages/:id/publish → publish package metadata
if (parts.length === 2 && parts[1] === 'publish' && m === 'POST') {
const id = decodeURIComponent(parts[0]);
const metadataService = await this.getService(CoreServiceName.enum.metadata);
if (metadataService && typeof (metadataService as any).publishPackage === 'function') {
const result = await (metadataService as any).publishPackage(id, body || {});
return { handled: true, response: this.success(result) };
}
// Broker fallback
if (this.kernel.broker) {
const result = await this.kernel.broker.call('metadata.publishPackage', { packageId: id, ...body }, { request: context.request });
return { handled: true, response: this.success(result) };
}
return { handled: true, response: this.error('Metadata service not available', 503) };
}
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handlePackages() currently requires an ObjectQL registry to be present before it will route any requests. That means the new /packages/:id/publish and /packages/:id/revert endpoints won’t work when the ObjectQL service/registry is unavailable (even if the metadata service is available), because the handler returns early to handlePackagesViaBroker() which doesn’t implement publish/revert. Consider handling publish/revert before the registry availability check, or adding publish/revert support to the broker fallback path.

Copilot uses AI. Check for mistakes.
Comment on lines +229 to +235
// Broker fallback
try {
const data = await broker.call('metadata.getPublished', { type, name }, { request: context.request });
return { handled: true, response: this.success(data) };
} catch (e: any) {
return { handled: true, response: this.error(e.message, 404) };
}
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the broker fallback for GET /meta/:type/:name/published, the result is always returned as 200 even if the broker returns undefined. The metadata-service path treats undefined as not found (404), so the broker fallback should mirror that behavior (check for undefined and return 404) to keep the endpoint semantics consistent.

Copilot uses AI. Check for mistakes.
Comment on lines +354 to +360
const packageItems: Array<{ type: string; name: string; data: any }> = [];
for (const [type, typeStore] of this.registry) {
for (const [name, data] of typeStore) {
const meta = data as any;
if (meta?.packageId === packageId || meta?.package === packageId) {
packageItems.push({ type, name, data: meta });
}
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

publishPackage() only scans the in-memory registry to collect package items. Items that exist only in loaders (e.g. DatabaseLoader/FilesystemLoader) won’t be included, so publishing a persisted package can incorrectly return “No metadata items found” or publish an incomplete set. Consider collecting items via the loader-backed APIs (e.g. iterate registered types and call list()/loadMany() then filter by packageId), and ensure the publish operation covers the full persisted package state.

Suggested change
const packageItems: Array<{ type: string; name: string; data: any }> = [];
for (const [type, typeStore] of this.registry) {
for (const [name, data] of typeStore) {
const meta = data as any;
if (meta?.packageId === packageId || meta?.package === packageId) {
packageItems.push({ type, name, data: meta });
}
const packageItems: Array<{ type: string; name: string; data: any }> = [];
const seenKeys = new Set<string>();
// 1) In-memory registry scan (existing behavior)
for (const [type, typeStore] of this.registry) {
for (const [name, data] of typeStore) {
const meta = data as any;
if (meta?.packageId === packageId || meta?.package === packageId) {
const key = `${type}:${name}`;
if (!seenKeys.has(key)) {
seenKeys.add(key);
packageItems.push({ type, name, data: meta });
}
}
}
}
// 2) Loader-backed collection via query API (ensures persisted-only items are included)
// We access `query` through `any` to avoid changing the public interface here while still
// leveraging the loader-backed implementation that likely exists behind IMetadataService.
if (typeof (this as any).query === 'function') {
try {
const queryResult = await (this as any).query({ packageId } as any);
const rawItems: any[] =
(queryResult && (queryResult.items ?? queryResult.results ?? queryResult.data)) ?? [];
for (const raw of rawItems) {
const meta = raw as any;
const itemPackageId = meta?.packageId ?? meta?.package;
if (itemPackageId !== packageId) continue;
const type: string | undefined =
meta?.type ?? meta?.metadataType ?? meta?.kind;
const name: string | undefined =
meta?.name ?? meta?.id;
if (!type || !name) continue;
const key = `${type}:${name}`;
if (seenKeys.has(key)) continue;
seenKeys.add(key);
packageItems.push({ type, name, data: meta });
}
} catch (err) {
// Fallback silently to registry-only behavior if the query API fails;
// publishing should not crash due to an unsupported/failed query call.
if ((this as any).logger && typeof (this as any).logger.warn === 'function') {
(this as any).logger.warn(
`publishPackage: loader-backed query failed for package '${packageId}':`,
err,
);
}

Copilot uses AI. Check for mistakes.
Comment on lines +434 to +451
// Determine the next version by finding the max current version across items
let maxVersion = 0;
for (const item of packageItems) {
const v = typeof item.data.version === 'number' ? item.data.version : 0;
if (v > maxVersion) maxVersion = v;
}
const newVersion = maxVersion + 1;

// Snapshot and update all items
for (const item of packageItems) {
const updated = {
...item.data,
publishedDefinition: structuredClone(item.data.metadata ?? item.data),
publishedAt: now,
publishedBy: publishedBy ?? item.data.publishedBy,
version: newVersion,
state: 'active',
};
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code uses the existing version field to represent the package publish version (newVersion) and overwrites it for every item. In this codebase, MetadataRecord.version is already used for optimistic concurrency / update counting (e.g. DatabaseLoader increments it on each save). Mixing these semantics will make publish versions drift with edits and can break concurrency expectations. Consider introducing a dedicated publish/version field (e.g. publishedVersion/packageVersion) and keep version for concurrency, or compute publish version from a separate stored counter.

Copilot uses AI. Check for mistakes.
Comment on lines +444 to +451
const updated = {
...item.data,
publishedDefinition: structuredClone(item.data.metadata ?? item.data),
publishedAt: now,
publishedBy: publishedBy ?? item.data.publishedBy,
version: newVersion,
state: 'active',
};
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

publishedDefinition is set via structuredClone(item.data.metadata ?? item.data). If an item is missing the metadata payload envelope, this snapshots the entire record object (including audit fields / previously publishedDefinition), and revertPackage() later writes that back into metadata, corrupting the stored shape. It would be safer to snapshot only item.data.metadata (and treat missing metadata as a validation error) so publishedDefinition consistently represents the published definition payload.

Copilot uses AI. Check for mistakes.
metadata: structuredClone(item.data.publishedDefinition),
state: 'active',
};
await this.register(item.type, item.name, reverted);
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revertPackage() only updates items that already have publishedDefinition. New items created after the last publish (no publishedDefinition) are left untouched, so a “revert” can still leave unpublished additions in the package. Consider removing/archiving items without a published snapshot when reverting, so the package truly returns to the last published state.

Suggested change
await this.register(item.type, item.name, reverted);
await this.register(item.type, item.name, reverted);
} else {
// Item was created after the last publish and has no published snapshot.
// Remove it so the package truly matches the last published state.
const typeStore = this.registry.get(item.type);
typeStore?.delete(item.name);

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[协议+Runtime] Package 级统一元数据发布机制 (publishPackage)

3 participants