Skip to content

3-level nested toArray: shared buffer in createPerEntryIncludesStates drops children when correlation keys overlap across parent groups #1501

@nestarz

Description

@nestarz
  • I've validated the bug against the latest version of DB packages

Describe the bug

3-level nested toArray includes return empty arrays for some children when sibling parent groups share the same correlation key at the deepest level.

createPerEntryIncludesStates assigns nestedSetups by reference (state.nestedSetups = setup.nestedSetups), so all per-parent-group child collection entries share the same pipeline buffer. drainNestedBuffers deletes buffer entries after routing them to the first matching parent group — subsequent groups with the same nested correlation key find nothing.

To Reproduce

Minimal repro — 3 collections, 3-level nesting, overlapping regionId across products:

import { createCollection, eq, localOnlyCollectionOptions, queryOnce, toArray } from "@tanstack/db"

const products = createCollection(localOnlyCollectionOptions({
  getKey: (p: any) => p.id,
  initialData: [
    { id: 1, title: "T-Shirt" },
    { id: 2, title: "Hoodie" },
  ],
}))

const priceRanges = createCollection(localOnlyCollectionOptions({
  getKey: (r: any) => r.id,
  initialData: [
    { id: 1, productId: 1, regionId: 1 },
    { id: 2, productId: 1, regionId: 2 },
    { id: 3, productId: 2, regionId: 1 }, // same regionId as priceRange 1
  ],
}))

const regions = createCollection(localOnlyCollectionOptions({
  getKey: (r: any) => r.id,
  initialData: [
    { id: 1, name: "Europe" },
    { id: 2, name: "North America" },
  ],
}))

const anchor = createCollection(localOnlyCollectionOptions({
  getKey: (r: any) => r.id,
  initialData: [{ id: 1 }],
}))

await Promise.all([products.preload(), priceRanges.preload(), regions.preload(), anchor.preload()])

const result = await queryOnce((qb) =>
  qb
    .from({ _: anchor })
    .select(({ _ }) => ({
      products: toArray(
        qb
          .from({ p: products })
          .where(({ p }) => eq(p.id, _.id)) // dummy correlation
          .select(({ p }) => ({
            id: p.id,
            title: p.title,
            priceRanges: toArray(
              qb
                .from({ pr: priceRanges })
                .where(({ pr }) => eq(pr.productId, p.id))
                .select(({ pr }) => ({
                  id: pr.id,
                  regionId: pr.regionId,
                  region: toArray(
                    qb
                      .from({ r: regions })
                      .where(({ r }) => eq(r.id, pr.regionId))
                      .select(({ r }) => ({ id: r.id, name: r.name })),
                  ),
                })),
            ),
          })),
      ),
    }))
    .findOne(),
)

const tshirt = result!.products.find((p: any) => p.title === "T-Shirt")
const pr1 = tshirt.priceRanges.find((r: any) => r.id === 1)
console.log(pr1.region) // [] — should be [{ id: 1, name: "Europe" }]

Control: changing priceRange 3's regionId to 3 (no overlap) makes all results correct.

Expected behavior

pr1.region should contain [{ id: 1, name: "Europe" }]. Each parent group should receive its own copy of nested pipeline results regardless of whether sibling groups share the same correlation key.

Root cause

createPerEntryIncludesStates shares nestedSetups (and their buffer) by reference across all child collection entries:

// line 1309
state.nestedSetups = setup.nestedSetups

drainNestedBuffers iterates setup.buffer, routes entries via state.nestedRoutingIndex, and deletes them from the shared buffer (line 1368-1372). When a second parent group drains the same buffer looking for the same nested correlation key, the entry is already gone.

Workaround

Adding a tautological parent-referencing condition at the 3rd level (e.g. gte(pr.id, 0)) forces parentProjection, which makes routing keys unique per parent row and avoids the buffer collision:

.where(({ r }) => and(eq(r.id, pr.regionId), gte(pr.id, 0)))

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions