feat: async import loader for JS via static preload (#476)#820
feat: async import loader for JS via static preload (#476)#820stephenamar-db wants to merge 2 commits intomasterfrom
Conversation
Adds Preloader + ImportFinder to discover imports from each file's AST and drive caller-controlled (async) loading before synchronous evaluation. Exposes SjsonnetMain.interpretAsync for the JS build, accepting a Promise-returning loader so callers can use fetch / FileReader / etc. Co-authored-by: Isaac
stephenamar-db
force-pushed
the
lazyimporter
branch
from
4c3fb08 to
6758e7a
Compare
Scala 2.13/2.12 promote "private default argument never used" to an error; the binary-loader and log defaults in the test helpers were unused at every call site. Inline them since no test exercises binary imports through the async path. Co-authored-by: Isaac
| Left(new ParseError(s"$path: ${traced.msg}", offset = traced.index)) | ||
| case Parsed.Success((expr, _), _) => | ||
| ImportFinder.collect(expr).foreach { found => | ||
| parentImporter.resolve(path, found.value) match { |
There was a problem hiding this comment.
This should resolve against path.parent(), not path. During evaluation Importer.resolveAndReadOrFail calls resolve(pos.fileScope.currentFile.parent(), ...), so the preloader is passing a different docBase to the JS resolver than synchronous interpret does. A resolver that joins docBase + import name will miss nested imports: dir/a.libsonnet importing b.libsonnet gets looked up under dir/a.libsonnet/b.libsonnet instead of dir/b.libsonnet, leaving the cache incomplete before evaluation.
| else Future.sequence(batch.map(loadOne)).flatMap(_ => loop()) | ||
| } | ||
|
|
||
| val result: Future[js.Any] = loop().map { _ => |
There was a problem hiding this comment.
loop() only preloads imports reachable from the entry file AST. runInterpret still parses extVars/tlaVars lazily via Interpreter.parseVar, and those snippets can contain imports. With the cache-only importer, std.extVar('cfg') where cfg is import 'lib.libsonnet' fails with Couldn't import file unless the root file also imports that path. The async path needs to preload parsed ext/tla snippets as code roots too, or otherwise keep variable parsing from hitting a cache-only importer.
| val traced = f.trace() | ||
| Left(new ParseError(s"$path: ${traced.msg}", offset = traced.index)) | ||
| case Parsed.Success((expr, _), _) => | ||
| ImportFinder.collect(expr).foreach { found => |
There was a problem hiding this comment.
Collecting every import node here makes interpretAsync eager in places normal Jsonnet evaluation is lazy. For example, if false then import 'bad.libsonnet' else 1 currently rejects if bad.libsonnet has a parse error, while synchronous interpret returns 1 because that branch is never forced. If static preloading is the intended tradeoff, this semantic difference should be explicit in the JS docs/API contract; otherwise this needs a design that does not parse/load imports from unforced branches.
2 participants
Adds Preloader + ImportFinder to discover imports from each file's AST and drive caller-controlled (async) loading before synchronous evaluation. Exposes SjsonnetMain.interpretAsync for the JS build, accepting a Promise-returning loader so callers can use fetch / FileReader / etc.
Co-authored-by: Isaac