Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fix-large-static-build-promise-all.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'astro': patch
---

Fixes a build error when generating projects with 100k+ static routes.
35 changes: 21 additions & 14 deletions packages/astro/src/core/build/generate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -153,22 +153,29 @@ export async function generatePages(
// Generate each path
if (config.build.concurrency > 1) {
const limit = PLimit(config.build.concurrency);
const promises: Promise<void>[] = [];
for (const { pathname, route } of filteredPaths) {
promises.push(
limit(() =>
generatePathWithPrerenderer(
prerenderer,
pathname,
route,
options,
routeToHeaders,
logger,
// Process in batches to avoid V8's Promise.all element limit, which is around ~123k items
//
// NOTE: ideally we could consider an iterator to avoid the batching limitation
const BATCH_SIZE = 100_000;
for (let i = 0; i < filteredPaths.length; i += BATCH_SIZE) {
const batch = filteredPaths.slice(i, i + BATCH_SIZE);
const promises: Promise<void>[] = [];
for (const { pathname, route } of batch) {
promises.push(
limit(() =>
generatePathWithPrerenderer(
prerenderer,
pathname,
route,
options,
routeToHeaders,
logger,
),
),
),
);
);
}
await Promise.all(promises);
}
await Promise.all(promises);
} else {
for (const { pathname, route } of filteredPaths) {
await generatePathWithPrerenderer(
Expand Down
3 changes: 2 additions & 1 deletion packages/astro/src/types/public/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -661,7 +661,8 @@ export interface AstroUserConfig<
* This feature comes with some limitations:
* - External scripts and external styles are not supported out of the box, but you can [provide your own hashes](https://v6.docs.astro.build/en/reference/configuration-reference/#hashes).
* - [Astro's view transitions](https://v6.docs.astro.build/en/guides/view-transitions/) using the `<ClientRouter />` are not supported, but you can [consider migrating to the browser native View Transition API](https://events-3bg.pages.dev/jotter/astro-view-transitions/) instead if you are not using Astro's enhancements to the native View Transitions and Navigation APIs.
* - Shiki isn't currently supported. By design, Shiki functions using inline styles.
* - Shiki isn't currently supported. By design, Shiki functions use inline styles that cannot work with Astro CSP implementation. Consider [using `<Prism />`](https://v6.docs.astro.build/en/guides/syntax-highlighting/#prism-) when your project requires both CSP and syntax highlighting.
* - `unsafe-inline` directives are incompatible with Astro's CSP implementation. By default, Astro will emit hashes for all its bundled scripts (e.g. client islands) and all modern browsers will automatically reject `unsafe-inline` when it occurs in a directive with a hash or nonce.
*
* :::note
* Due to the nature of the Vite dev server, this feature isn't supported while working in `dev` mode. Instead, you can test this in your Astro project using `build` and `preview`.
Expand Down
Loading