Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,7 @@ business/custom objects, aligning with industry best practices (e.g., ServiceNow
- [x] **In-Memory Driver** — Full CRUD, bulk ops, transactions, aggregation pipeline (Mingo), streaming
- [x] **In-Memory Driver Persistence** — File-system (Node.js) and localStorage (Browser) persistence adapters with auto-save, custom adapter support
- [x] **Metadata Service** — CRUD, query, bulk ops, overlay system, dependency tracking, import/export, file watching
- [x] **Metadata Package Publishing** — `publishPackage`, `revertPackage`, `getPublished` for atomic package-level metadata publishing with version snapshots
- [x] **Serializers** — JSON, YAML, TypeScript format support
- [x] **Loaders** — Memory, Filesystem, Remote (HTTP) loaders
- [x] **REST API** — Auto-generated CRUD/Metadata/Batch/Discovery endpoints
Expand Down Expand Up @@ -452,6 +453,7 @@ business/custom objects, aligning with industry best practices (e.g., ServiceNow
- User overlay persistence across sessions
- Multi-instance metadata synchronization
- Production-grade metadata storage
- Package-level metadata publishing (publishPackage / revertPackage / getPublished)

### Phase 4b: Infrastructure Service Upgrades (P1 — Weeks 3-4)

Expand Down
56 changes: 56 additions & 0 deletions content/docs/guides/contracts/metadata-service.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -352,3 +352,59 @@ const filteredViews = views.filter(view => {
return !v.requiredPermission || userPermissions.includes(v.requiredPermission);
});
```

---

## Package Publishing

ObjectStack uses **package-level publishing** to ensure metadata consistency. All metadata items within a package are published atomically — either everything goes live, or nothing does.

### publishPackage

Publishes all metadata items in a package:
1. Validates all items (optional)
2. Snapshots each item's definition into `publishedDefinition`
3. Increments the package version
4. Sets all items to `active` state

```typescript
const result = await metadataService.publishPackage('com.acme.crm', {
publishedBy: 'admin-user',
validate: true, // default: true
changeNote: 'Added opportunity fields',
});

console.log(result.success); // true
console.log(result.version); // 2
console.log(result.itemsPublished); // 5
console.log(result.publishedAt); // "2025-06-01T12:00:00Z"
```

### revertPackage

Reverts all metadata items in a package to their last published state. Discards any unpublished changes.

```typescript
await metadataService.revertPackage('com.acme.crm');
// All items restored to their publishedDefinition snapshots
```

### getPublished

Returns the published version of a metadata item (for runtime/end-user serving). Falls back to the current definition if the item has never been published.

```typescript
// End user sees the published version
const published = await metadataService.getPublished('object', 'opportunity');

// Designer sees the draft version (via regular get)
const draft = await metadataService.get('object', 'opportunity');
```

### REST Endpoints

| Method | Path | Description |
|:---|:---|:---|
| `POST` | `/packages/:id/publish` | Publish a package |
| `POST` | `/packages/:id/revert` | Revert a package to last published state |
| `GET` | `/metadata/:type/:name/published` | Get published version of a metadata item |
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs list REST endpoints under /metadata/..., but the runtime/adapters in this repo expose metadata routes under /meta (e.g. /api/v1/meta/...). To avoid sending users to non-existent endpoints, update these paths to the actual /meta/:type/:name/published route (and keep it consistent with the rest of the docs).

Suggested change
| `GET` | `/metadata/:type/:name/published` | Get published version of a metadata item |
| `GET` | `/meta/:type/:name/published` | Get published version of a metadata item |

Copilot uses AI. Check for mistakes.
29 changes: 29 additions & 0 deletions packages/metadata/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ The `MetadataManager` is the main orchestrator. It provides:
- **Core CRUD**: `register`, `get`, `list`, `unregister`, `exists`, `listNames`
- **Convenience**: `getObject`, `listObjects`
- **Package Management**: `unregisterPackage` — unload all metadata from a package
- **Package Publishing**: `publishPackage`, `revertPackage`, `getPublished` — atomic package-level metadata publishing
- **Query / Search**: `query` with filtering, pagination, sorting by type/scope/state/tags
- **Bulk Operations**: `bulkRegister`, `bulkUnregister` with error handling
- **Import / Export**: `exportMetadata`, `importMetadata` with conflict resolution (skip/overwrite/merge)
Expand Down Expand Up @@ -201,6 +202,34 @@ const plugin = MetadataPlugin({
kernel.use(plugin);
```

## Package Publishing

ObjectStack supports **package-level metadata publishing** — all metadata items within a package are published atomically.

### Publish a Package

```typescript
const result = await manager.publishPackage('com.acme.crm', {
publishedBy: 'admin',
validate: true,
});
// result: { success: true, version: 2, itemsPublished: 5, publishedAt: '...' }
```

### Revert to Last Published State

```typescript
await manager.revertPackage('com.acme.crm');
// All items restored to their publishedDefinition snapshots
```

### Get Published Version (Runtime Serving)

```typescript
const published = await manager.getPublished('object', 'opportunity');
// Returns publishedDefinition if exists, else current definition
```

## Package Structure

```
Expand Down
182 changes: 182 additions & 0 deletions packages/metadata/src/metadata-manager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import type {
MetadataSaveResult,
MetadataWatchEvent,
MetadataFormat,
PackagePublishResult,
} from '@objectstack/spec/system';
import type {
IMetadataService,
Expand Down Expand Up @@ -333,6 +334,187 @@ export class MetadataManager implements IMetadataService {
}
}

/**
* Publish an entire package:
* 1. Validate all draft items
* 2. Snapshot all items in the package (publishedDefinition = clone(metadata))
* 3. Increment version
* 4. Set all items state → active
*/
async publishPackage(packageId: string, options?: {
changeNote?: string;
publishedBy?: string;
validate?: boolean;
}): Promise<PackagePublishResult> {
const now = new Date().toISOString();
const shouldValidate = options?.validate !== false;
const publishedBy = options?.publishedBy;

Comment on lines +344 to +352
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

publishPackage() accepts changeNote in options but never uses or persists it. If it’s intentionally not supported yet, consider removing it from the options (or explicitly documenting it as not implemented); otherwise persist it as part of the publish audit trail so callers don’t assume it’s recorded.

Copilot uses AI. Check for mistakes.
// Collect all items belonging to this package
const packageItems: Array<{ type: string; name: string; data: any }> = [];
for (const [type, typeStore] of this.registry) {
for (const [name, data] of typeStore) {
const meta = data as any;
if (meta?.packageId === packageId || meta?.package === packageId) {
packageItems.push({ type, name, data: meta });
}
Comment on lines +354 to +360
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

publishPackage() only scans the in-memory registry to collect package items. Items that exist only in loaders (e.g. DatabaseLoader/FilesystemLoader) won’t be included, so publishing a persisted package can incorrectly return “No metadata items found” or publish an incomplete set. Consider collecting items via the loader-backed APIs (e.g. iterate registered types and call list()/loadMany() then filter by packageId), and ensure the publish operation covers the full persisted package state.

Suggested change
const packageItems: Array<{ type: string; name: string; data: any }> = [];
for (const [type, typeStore] of this.registry) {
for (const [name, data] of typeStore) {
const meta = data as any;
if (meta?.packageId === packageId || meta?.package === packageId) {
packageItems.push({ type, name, data: meta });
}
const packageItems: Array<{ type: string; name: string; data: any }> = [];
const seenKeys = new Set<string>();
// 1) In-memory registry scan (existing behavior)
for (const [type, typeStore] of this.registry) {
for (const [name, data] of typeStore) {
const meta = data as any;
if (meta?.packageId === packageId || meta?.package === packageId) {
const key = `${type}:${name}`;
if (!seenKeys.has(key)) {
seenKeys.add(key);
packageItems.push({ type, name, data: meta });
}
}
}
}
// 2) Loader-backed collection via query API (ensures persisted-only items are included)
// We access `query` through `any` to avoid changing the public interface here while still
// leveraging the loader-backed implementation that likely exists behind IMetadataService.
if (typeof (this as any).query === 'function') {
try {
const queryResult = await (this as any).query({ packageId } as any);
const rawItems: any[] =
(queryResult && (queryResult.items ?? queryResult.results ?? queryResult.data)) ?? [];
for (const raw of rawItems) {
const meta = raw as any;
const itemPackageId = meta?.packageId ?? meta?.package;
if (itemPackageId !== packageId) continue;
const type: string | undefined =
meta?.type ?? meta?.metadataType ?? meta?.kind;
const name: string | undefined =
meta?.name ?? meta?.id;
if (!type || !name) continue;
const key = `${type}:${name}`;
if (seenKeys.has(key)) continue;
seenKeys.add(key);
packageItems.push({ type, name, data: meta });
}
} catch (err) {
// Fallback silently to registry-only behavior if the query API fails;
// publishing should not crash due to an unsupported/failed query call.
if ((this as any).logger && typeof (this as any).logger.warn === 'function') {
(this as any).logger.warn(
`publishPackage: loader-backed query failed for package '${packageId}':`,
err,
);
}

Copilot uses AI. Check for mistakes.
}
}

if (packageItems.length === 0) {
return {
success: false,
packageId,
version: 0,
publishedAt: now,
itemsPublished: 0,
validationErrors: [{ type: '', name: '', message: `No metadata items found for package '${packageId}'` }],
};
}

// Validation pass
if (shouldValidate) {
const validationErrors: Array<{ type: string; name: string; message: string }> = [];

// Schema validation
for (const item of packageItems) {
const result = await this.validate(item.type, item.data);
if (!result.valid && result.errors) {
for (const err of result.errors) {
validationErrors.push({
type: item.type,
name: item.name,
message: err.message,
});
}
}
}

// Dependency validation: referenced items must be in the same package or already published
const packageItemKeys = new Set(packageItems.map(i => `${i.type}:${i.name}`));
for (const item of packageItems) {
const deps = await this.getDependencies(item.type, item.name);
for (const dep of deps) {
const depKey = `${dep.targetType}:${dep.targetName}`;
// Skip if the dependency is within this package
if (packageItemKeys.has(depKey)) continue;
// Check if the dependency exists and has been published
const depItem = await this.get(dep.targetType, dep.targetName);
if (!depItem) {
validationErrors.push({
type: item.type,
name: item.name,
message: `Dependency '${dep.targetType}:${dep.targetName}' not found`,
});
} else {
const depMeta = depItem as any;
if (depMeta.publishedDefinition === undefined && depMeta.state !== 'active') {
validationErrors.push({
type: item.type,
name: item.name,
message: `Dependency '${dep.targetType}:${dep.targetName}' is not published`,
});
}
}
}
}

if (validationErrors.length > 0) {
return {
success: false,
packageId,
version: 0,
publishedAt: now,
itemsPublished: 0,
validationErrors,
};
}
}

// Determine the next version by finding the max current version across items
let maxVersion = 0;
for (const item of packageItems) {
const v = typeof item.data.version === 'number' ? item.data.version : 0;
if (v > maxVersion) maxVersion = v;
}
const newVersion = maxVersion + 1;

// Snapshot and update all items
for (const item of packageItems) {
const updated = {
...item.data,
publishedDefinition: structuredClone(item.data.metadata ?? item.data),
publishedAt: now,
publishedBy: publishedBy ?? item.data.publishedBy,
version: newVersion,
state: 'active',
};
Comment on lines +434 to +451
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code uses the existing version field to represent the package publish version (newVersion) and overwrites it for every item. In this codebase, MetadataRecord.version is already used for optimistic concurrency / update counting (e.g. DatabaseLoader increments it on each save). Mixing these semantics will make publish versions drift with edits and can break concurrency expectations. Consider introducing a dedicated publish/version field (e.g. publishedVersion/packageVersion) and keep version for concurrency, or compute publish version from a separate stored counter.

Copilot uses AI. Check for mistakes.
Comment on lines +444 to +451
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

publishedDefinition is set via structuredClone(item.data.metadata ?? item.data). If an item is missing the metadata payload envelope, this snapshots the entire record object (including audit fields / previously publishedDefinition), and revertPackage() later writes that back into metadata, corrupting the stored shape. It would be safer to snapshot only item.data.metadata (and treat missing metadata as a validation error) so publishedDefinition consistently represents the published definition payload.

Copilot uses AI. Check for mistakes.
await this.register(item.type, item.name, updated);
}

return {
success: true,
packageId,
version: newVersion,
publishedAt: now,
itemsPublished: packageItems.length,
};
}

/**
* Revert entire package to last published state.
* Restores all metadata definitions from their published snapshots.
*/
async revertPackage(packageId: string): Promise<void> {
const packageItems: Array<{ type: string; name: string; data: any }> = [];
for (const [type, typeStore] of this.registry) {
for (const [name, data] of typeStore) {
const meta = data as any;
if (meta?.packageId === packageId || meta?.package === packageId) {
packageItems.push({ type, name, data: meta });
}
}
}

if (packageItems.length === 0) {
throw new Error(`No metadata items found for package '${packageId}'`);
}

// Check that at least one item has a published snapshot
const hasPublished = packageItems.some(item => item.data.publishedDefinition !== undefined);
if (!hasPublished) {
throw new Error(`Package '${packageId}' has never been published`);
}

for (const item of packageItems) {
if (item.data.publishedDefinition !== undefined) {
const reverted = {
...item.data,
metadata: structuredClone(item.data.publishedDefinition),
state: 'active',
};
await this.register(item.type, item.name, reverted);
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revertPackage() only updates items that already have publishedDefinition. New items created after the last publish (no publishedDefinition) are left untouched, so a “revert” can still leave unpublished additions in the package. Consider removing/archiving items without a published snapshot when reverting, so the package truly returns to the last published state.

Suggested change
await this.register(item.type, item.name, reverted);
await this.register(item.type, item.name, reverted);
} else {
// Item was created after the last publish and has no published snapshot.
// Remove it so the package truly matches the last published state.
const typeStore = this.registry.get(item.type);
typeStore?.delete(item.name);

Copilot uses AI. Check for mistakes.
}
}
}

/**
* Get the published version of any metadata item (for runtime serving).
* Returns publishedDefinition if exists, else current definition.
*/
async getPublished(type: string, name: string): Promise<unknown | undefined> {
const item = await this.get(type, name);
if (!item) return undefined;

const meta = item as any;
if (meta.publishedDefinition !== undefined) {
return meta.publishedDefinition;
}

// Fall back to current definition (metadata field or the item itself)
return meta.metadata ?? item;
}

// ==========================================
// Query / Search
// ==========================================
Expand Down
Loading