Skip to content

rush-resolver-cache-plugin: add pnpm 10 / lockfile v9 compatibility #5749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bmiddha wants to merge 1 commit into
base: main
Choose a base branch
from
Open

rush-resolver-cache-plugin: add pnpm 10 / lockfile v9 compatibility #5749

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions common/changes/@microsoft/rush/bmiddha-resolver-cache-pnpm-10_2026-04-07-06-16.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@microsoft/rush",
"comment": "rush-resolver-cache-plugin: add pnpm 10 / lockfile v9 compatibility",
"type": "none"
}
],
"packageName": "@microsoft/rush"
}
53 changes: 50 additions & 3 deletions rush-plugins/rush-resolver-cache-plugin/src/afterInstallAsync.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
// Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT license.
// See LICENSE in the project root for license information.

import { existsSync, readdirSync } from 'node:fs';

import type {
RushSession,
RushConfiguration,
Expand Down Expand Up @@ -79,7 +81,12 @@ export async function afterInstallAsync(

const lockFilePath: string = subspace.getCommittedShrinkwrapFilePath(variant);

const pnpmStoreDir: string = `${rushConfiguration.pnpmOptions.pnpmStorePath}/v3/files/`;
const pnpmStorePath: string = rushConfiguration.pnpmOptions.pnpmStorePath;
// pnpm 10 uses v10/index/ for index files; pnpm 8 uses v3/files/
const pnpmStoreV10IndexDir: string = `${pnpmStorePath}/v10/index/`;
const pnpmStoreV3FilesDir: string = `${pnpmStorePath}/v3/files/`;
const useV10Store: boolean = existsSync(pnpmStoreV10IndexDir);
const pnpmStoreDir: string = useV10Store ? pnpmStoreV10IndexDir : pnpmStoreV3FilesDir;

terminal.writeLine(`Using pnpm-lock from: ${lockFilePath}`);
terminal.writeLine(`Using pnpm store folder: ${pnpmStoreDir}`);
Expand Down Expand Up @@ -167,9 +174,49 @@ export async function afterInstallAsync(
const hash: string = Buffer.from(descriptionFileHash.slice(prefixIndex + 1), 'base64').toString('hex');

// The pnpm store directory has index files of package contents at paths:
// <store>/v3/files/<hash (0-2)>/<hash (2-)>-index.json
// pnpm 8: <store>/v3/files/<hash (0-2)>/<hash (2-)>-index.json
// pnpm 10: <store>/v10/index/<hash (0-2)>/<hash (2-64)>-<name>@<version>.json
// See https://github.com/pnpm/pnpm/blob/f394cfccda7bc519ceee8c33fc9b68a0f4235532/store/cafs/src/getFilePathInCafs.ts#L33
const indexPath: string = `${pnpmStoreDir}${hash.slice(0, 2)}/${hash.slice(2)}-index.json`;
let indexPath: string;
if (useV10Store) {
// pnpm 10 truncates integrity hashes to 32 bytes (64 hex chars) for index paths.
const truncHash: string = hash.length > 64 ? hash.slice(0, 64) : hash;
const hashDir: string = truncHash.slice(0, 2);
const hashRest: string = truncHash.slice(2);
// Build the bare name@version using context.name and version from the .pnpm folder path.
// The .pnpm folder name format is: <name+ver>_<peer1>_<peer2>/node_modules/<name>
const pkgName: string = (context.name || '').replace(/\//g, '+');
Copy link
Copy Markdown
Contributor

@dmichon-msft dmichon-msft Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just plumb the version that we extract from the key in the lockfile into the context object instead of recomputing it.

const pnpmSegment: string | undefined = context.descriptionFileRoot.split('/node_modules/.pnpm/')[1];
const folderName: string = pnpmSegment ? pnpmSegment.split('/node_modules/')[0] : '';
Comment on lines +189 to +190
Copy link
Copy Markdown
Contributor

@dmichon-msft dmichon-msft Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to avoid .split in this code; this is a hot path.

const namePrefix: string = `${pkgName}@`;
const nameStart: number = folderName.indexOf(namePrefix);
let nameVer: string = folderName;
if (nameStart !== -1) {
const afterName: string = folderName.slice(nameStart + namePrefix.length);
// Version ends at first _ followed by a letter/@ (peer dep separator)
const peerSep: number = afterName.search(/_[a-zA-Z@]/);
const version: string = peerSep !== -1 ? afterName.slice(0, peerSep) : afterName;
nameVer = `${pkgName}@${version}`;
}
indexPath = `${pnpmStoreDir}${hashDir}/${hashRest}-${nameVer}.json`;
// For truncated/hashed folder names, nameVer from the folder path may be wrong.
// Fallback: scan the directory for a file matching the hash prefix.
if (!existsSync(indexPath)) {
const dir: string = `${pnpmStoreDir}${hashDir}/`;
const filePrefix: string = `${hashRest}-`;
try {
const files: string[] = readdirSync(dir);
const match: string | undefined = files.find((f) => f.startsWith(filePrefix));
if (match) {
indexPath = dir + match;
}
} catch {
// ignore
}
}
Comment on lines +182 to +216
Copy link
Copy Markdown
Contributor

@dmichon-msft dmichon-msft Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put this whole thing in a helper function.

} else {
indexPath = `${pnpmStoreDir}${hash.slice(0, 2)}/${hash.slice(2)}-index.json`;
}

try {
const indexContent: string = await FileSystem.readFileAsync(indexPath);
Expand Down
28 changes: 23 additions & 5 deletions rush-plugins/rush-resolver-cache-plugin/src/computeResolverCacheFromLockfileAsync.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,15 @@ export async function computeResolverCacheFromLockfileAsync(
const contexts: Map<string, IResolverContext> = new Map();
const missingOptionalDependencies: Set<string> = new Set();

// Detect v9+ lockfile format by checking if the lockfile has shrinkwrapFileMajorVersion >= 9,
// or by checking if the first package key lacks a leading '/' (v6 keys always start with '/').
const isV9Lockfile: boolean =
(lockfile as { shrinkwrapFileMajorVersion?: number }).shrinkwrapFileMajorVersion !== undefined
? (lockfile as { shrinkwrapFileMajorVersion?: number }).shrinkwrapFileMajorVersion! >= 9
: !Array.from(lockfile.packages.keys())
.find((k) => !k.startsWith('file:'))
?.startsWith('/');
Comment on lines +177 to +179
Copy link
Copy Markdown
Contributor

@dmichon-msft dmichon-msft Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create a helper function that early returns instead of cloning the entire key space.


// Enumerate external dependencies first, to simplify looping over them for store data
for (const [key, pack] of lockfile.packages) {
let name: string | undefined = pack.name;
Expand All @@ -187,6 +196,15 @@ export async function computeResolverCacheFromLockfileAsync(
name = key.slice(1, versionIndex);
}

if (!name) {
// Handle v9 lockfile keys: @scope/name@version or name@version
const searchStart: number = key.startsWith('@') ? key.indexOf('/') + 1 : 0;
const versionIndex: number = key.indexOf('@', searchStart);
if (versionIndex !== -1) {
Copy link
Copy Markdown
Contributor

@dmichon-msft dmichon-msft Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's put the versions on context so we don't have to compute them later.

name = key.slice(0, versionIndex);
}
}

if (!name) {
throw new Error(`Missing name for ${key}`);
}
Expand All @@ -204,10 +222,10 @@ export async function computeResolverCacheFromLockfileAsync(
contexts.set(descriptionFileRoot, context);

if (pack.dependencies) {
resolveDependencies(workspaceRoot, pack.dependencies, context);
resolveDependencies(workspaceRoot, pack.dependencies, context, isV9Lockfile);
}
if (pack.optionalDependencies) {
resolveDependencies(workspaceRoot, pack.optionalDependencies, context);
resolveDependencies(workspaceRoot, pack.optionalDependencies, context, isV9Lockfile);
}
}

Expand Down Expand Up @@ -248,13 +266,13 @@ export async function computeResolverCacheFromLockfileAsync(
contexts.set(descriptionFileRoot, context);

if (importer.dependencies) {
resolveDependencies(workspaceRoot, importer.dependencies, context);
resolveDependencies(workspaceRoot, importer.dependencies, context, isV9Lockfile);
}
if (importer.devDependencies) {
resolveDependencies(workspaceRoot, importer.devDependencies, context);
resolveDependencies(workspaceRoot, importer.devDependencies, context, isV9Lockfile);
}
if (importer.optionalDependencies) {
resolveDependencies(workspaceRoot, importer.optionalDependencies, context);
resolveDependencies(workspaceRoot, importer.optionalDependencies, context, isV9Lockfile);
}
}

Expand Down
70 changes: 57 additions & 13 deletions rush-plugins/rush-resolver-cache-plugin/src/helpers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ import type { ISerializedResolveContext } from '@rushstack/webpack-workspace-res

import type { IDependencyEntry, IResolverContext } from './types';

const MAX_LENGTH_WITHOUT_HASH: number = 120 - 26 - 1;
const PNPM8_MAX_LENGTH_WITHOUT_HASH: number = 120 - 26 - 1;
// pnpm 10 uses SHA-256 hex (32 chars) + underscore separator
const PNPM10_MAX_LENGTH_WITHOUT_HASH: number = 120 - 32 - 1;
const BASE32: string[] = 'abcdefghijklmnopqrstuvwxyz234567'.split('');

// https://github.com/swansontec/rfc4648.js/blob/ead9c9b4b68e5d4a529f32925da02c02984e772c/src/codec.ts#L82-L118
Expand Down Expand Up @@ -42,14 +44,32 @@ export function createBase32Hash(input: string): string {
return out;
}

/**
* Creates a short SHA-256 hex hash, matching pnpm 10's createShortHash.
*/
export function createShortSha256Hash(input: string): string {
return createHash('sha256').update(input).digest('hex').substring(0, 32);
}

// https://github.com/pnpm/pnpm/blob/f394cfccda7bc519ceee8c33fc9b68a0f4235532/packages/dependency-path/src/index.ts#L167-L189
export function depPathToFilename(depPath: string): string {
export function depPathToFilename(depPath: string, usePnpm10Hashing?: boolean): string {
let filename: string = depPathToFilenameUnescaped(depPath).replace(/[\\/:*?"<>|]/g, '+');
if (filename.includes('(')) {
filename = filename.replace(/(\)\()|\(/g, '_').replace(/\)$/, '');
}
if (filename.length > 120 || (filename !== filename.toLowerCase() && !filename.startsWith('file+'))) {
return `${filename.substring(0, MAX_LENGTH_WITHOUT_HASH)}_${createBase32Hash(filename)}`;
if (usePnpm10Hashing) {
// pnpm 10 also replaces `#` and handles parentheses differently
filename = filename.replace(/#/g, '+');
if (filename.includes('(')) {
filename = filename.replace(/\)$/, '').replace(/\)\(|\(|\)/g, '_');
}
if (filename.length > 120 || (filename !== filename.toLowerCase() && !filename.startsWith('file+'))) {
return `${filename.substring(0, PNPM10_MAX_LENGTH_WITHOUT_HASH)}_${createShortSha256Hash(filename)}`;
}
} else {
if (filename.includes('(')) {
filename = filename.replace(/(\)\()|\(/g, '_').replace(/\)$/, '');
}
if (filename.length > 120 || (filename !== filename.toLowerCase() && !filename.startsWith('file+'))) {
return `${filename.substring(0, PNPM8_MAX_LENGTH_WITHOUT_HASH)}_${createBase32Hash(filename)}`;
}
}
Copy link
Copy Markdown
Contributor

@dmichon-msft dmichon-msft Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hoist regexes

return filename;
}
Expand All @@ -66,7 +86,8 @@ export function resolveDependencyKey(
lockfileFolder: string,
key: string,
specifier: string,
context: IResolverContext
context: IResolverContext,
isV9Lockfile?: boolean
): string {
if (specifier.startsWith('/')) {
return getDescriptionFileRootFromKey(lockfileFolder, specifier);
Expand All @@ -79,7 +100,16 @@ export function resolveDependencyKey(
} else if (specifier.startsWith('file:')) {
return getDescriptionFileRootFromKey(lockfileFolder, specifier, key);
} else {
return getDescriptionFileRootFromKey(lockfileFolder, `/${key}@${specifier}`);
// In v9 lockfiles, aliased dependency values use the full package key format
// (e.g., 'string-width@4.2.3' or '@types/events@3.0.0') instead of bare versions.
Comment on lines +103 to +104
Copy link
Copy Markdown
Contributor

@dmichon-msft dmichon-msft Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This statement is true in v6 as well; that's why there's the specifier.startsWith('/') branch.
I wonder if we should just try to see if the value is a specifier by checking it against the packages list?

// A bare version starts with a digit; a full key starts with a letter or @.
if (/^[a-zA-Z@]/.test(specifier)) {
return getDescriptionFileRootFromKey(lockfileFolder, specifier);
}
// Construct the full dependency key from package name and version specifier.
// v6 keys use '/' prefix; v9 keys don't.
const fullKey: string = isV9Lockfile ? `${key}@${specifier}` : `/${key}@${specifier}`;
return getDescriptionFileRootFromKey(lockfileFolder, fullKey);
}
}

Expand All @@ -91,26 +121,40 @@ export function resolveDependencyKey(
* @returns The physical path to the dependency
*/
export function getDescriptionFileRootFromKey(lockfileFolder: string, key: string, name?: string): string {
// Detect lockfile version: v6 keys start with '/', v9 keys don't
const isV9Key: boolean = !key.startsWith('/') && !key.startsWith('file:');

if (!key.startsWith('file:')) {
name = key.slice(1, key.indexOf('@', 2));
if (key.startsWith('/')) {
// v6 format: /name@version or /@scope/name@version
name = key.slice(1, key.indexOf('@', 2));
} else if (!name) {
// v9 format: name@version or @scope/name@version
const searchStart: number = key.startsWith('@') ? key.indexOf('/') + 1 : 0;
const versionIndex: number = key.indexOf('@', searchStart);
if (versionIndex !== -1) {
name = key.slice(0, versionIndex);
}
}
Comment on lines +128 to +138
Copy link
Copy Markdown
Contributor

@dmichon-msft dmichon-msft Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only difference between these branches is the presence of the leading /, which just adjusts the offsets by 1. The !key.startsWith('/') branch can just return key.slice(0, key.indexOf('@', 1)) instead of key.slice(1, key.indexOf('@', 2))

}
if (!name) {
throw new Error(`Missing package name for ${key}`);
}

const originFolder: string = `${lockfileFolder}/node_modules/.pnpm/${depPathToFilename(key)}/node_modules`;
const originFolder: string = `${lockfileFolder}/node_modules/.pnpm/${depPathToFilename(key, isV9Key)}/node_modules`;
const descriptionFileRoot: string = `${originFolder}/${name}`;
return descriptionFileRoot;
}

export function resolveDependencies(
lockfileFolder: string,
collection: Record<string, IDependencyEntry>,
context: IResolverContext
context: IResolverContext,
isV9Lockfile?: boolean
): void {
for (const [key, value] of Object.entries(collection)) {
const version: string = typeof value === 'string' ? value : value.version;
const resolved: string = resolveDependencyKey(lockfileFolder, key, version, context);
const resolved: string = resolveDependencyKey(lockfileFolder, key, version, context, isV9Lockfile);

context.deps.set(key, resolved);
}
Expand Down
Loading
Loading