sqlite: optimize column name creation and text value encoding by thisalihassan · Pull Request #61954 · nodejs/node

thisalihassan · 2026-02-23T14:53:50Z

Skip the full UTF-8 decode path for text values that are pure ASCII by validating with simdutf and creating OneByte V8 strings, halving memory.Internalize column name strings so V8 shares hidden classes across row objects. Cache column names in the iterate() hot loop, invalidating on schema changes via SQLITE_STMTSTATUS_REPREPARE.

Benchmark: 30-run

Refs: nodejs/performance#181

nodejs-github-bot · 2026-02-23T14:53:56Z

Review requested:

@nodejs/sqlite

Use simdutf to detect ASCII text values and create them via NewFromOneByte for compact one-byte representation. Internalize column name strings with kInternalized so V8 shares hidden classes across row objects. Cache column names on StatementSync for iterate(), invalidated via SQLITE_STMTSTATUS_REPREPARE on schema changes. Refs: nodejs/performance#181

thisalihassan · 2026-02-23T16:37:59Z

I didn't push the changes for sqlite-prepare-select-all-options and sqlite-prepare-select-all where I tested the iterate method because it slows down the benchmark by a lot.
Adding iterate nearly ~2.3x the total benchmark runtime from ~40 minutes to ~90 minutes, but if needed I can push that as well
Benchmark with iterate: 30 runs

codecov · 2026-02-23T17:13:49Z

Codecov Report

❌ Patch coverage is 83.72093% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.77%. Comparing base (da5efc4) to head (fd4215d).
⚠️ Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
src/node_sqlite.cc	83.72%	4 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #61954      +/-   ##
==========================================
+ Coverage   88.84%   89.77%   +0.93%     
==========================================
  Files         674      674              
  Lines      204957   205640     +683     
  Branches    39309    39422     +113     
==========================================
+ Hits       182087   184608    +2521     
+ Misses      15088    13264    -1824     
+ Partials     7782     7768      -14

Files with missing lines	Coverage Δ
src/node_sqlite.h	`81.48% <ø> (+1.08%)`	⬆️
src/node_sqlite.cc	`80.92% <83.72%> (+0.02%)`	⬆️

... and 181 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

anonrig · 2026-02-24T16:26:50Z

src/node_sqlite.cc

+                                                 const char* data,
+                                                 size_t length) {


Can you just take a std::string_view input as an argument rather than this C API like function?

anonrig · 2026-02-24T16:27:13Z

src/node_sqlite.cc

+                                  NewStringType::kNormal,
+                                  len);
+  }
+  return String::NewFromUtf8(isolate, data, NewStringType::kNormal, len);


Why don't you use kInternalized here and in line 66?

Utf8StringMaybeOneByte is used for text cell values (row data), not column names, text values are typically unique user data ("Ali Hassan", "ali.n@example.com", etc.) so the lookup would almost always miss

Column names are already interned separately in ColumnNameToName

anonrig · 2026-02-24T16:28:06Z

src/node_sqlite.cc

-      if (!iter->stmt_->ColumnNameToName(i).ToLocal(&key)) return;
-      row_keys.emplace_back(key);
-    }
+    if (!iter->stmt_->GetCachedColumnNames(&row_keys)) return;


Can you add a comment to here?

anonrig · 2026-02-24T16:28:17Z

src/node_sqlite.cc

+      .As<Name>();
+}
+
+bool StatementSync::GetCachedColumnNames(LocalVector<Name>* keys) {


Can you add a comment to this function?

thisalihassan · 2026-02-26T16:04:11Z

Hi @addaleax @geeksilva97 can I please get review on this PR when you get time? Thanks

louwers · 2026-02-26T20:29:04Z

src/node_sqlite.cc

+  Isolate* isolate = env()->isolate();
+
+  int reprepare_count =
+      sqlite3_stmt_status(statement_, SQLITE_STMTSTATUS_REPREPARE, 0);


Nit

Suggested change

sqlite3_stmt_status(statement_, SQLITE_STMTSTATUS_REPREPARE, 0);

sqlite3_stmt_status(statement_, SQLITE_STMTSTATUS_REPREPARE, false);

louwers · 2026-02-26T20:29:12Z

src/node_sqlite.cc

+bool StatementSync::GetCachedColumnNames(LocalVector<Name>* keys) {
+  Isolate* isolate = env()->isolate();
+
+  int reprepare_count =


Suggested change

int reprepare_count =

const int reprepare_count =

louwers · 2026-02-26T21:15:45Z

src/node_sqlite.cc

+      sqlite3_stmt_status(statement_, SQLITE_STMTSTATUS_REPREPARE, 0);
+  if (reprepare_count != cached_column_names_reprepare_count_) {
+    cached_column_names_.clear();
+    int num_cols = sqlite3_column_count(statement_);


Suggested change

int num_cols = sqlite3_column_count(statement_);

const int num_cols = sqlite3_column_count(statement_);

louwers · 2026-02-26T21:16:38Z

src/node_sqlite.cc

        const char* v =                                                        \
            reinterpret_cast<const char*>(sqlite3_##from##_text(__VA_ARGS__)); \
-        (result) = String::NewFromUtf8((isolate), v).As<Value>();              \
+        int v_len = sqlite3_##from##_bytes(__VA_ARGS__);                       \


Suggested change

int v_len = sqlite3_##from##_bytes(__VA_ARGS__); \

const int v_len = sqlite3_##from##_bytes(__VA_ARGS__); \

louwers · 2026-02-26T21:16:58Z

src/node_sqlite.cc


+inline MaybeLocal<String> Utf8StringMaybeOneByte(Isolate* isolate,
+                                                 std::string_view input) {
+  int len = static_cast<int>(input.size());


Suggested change

int len = static_cast<int>(input.size());

const int len = static_cast<int>(input.size());

louwers · 2026-02-26T21:17:43Z

src/node_sqlite.cc

 void StatementSync::Finalize() {
  sqlite3_finalize(statement_);
  statement_ = nullptr;
+  cached_column_names_.clear();


Consider putting this in a method.

louwers · 2026-02-26T21:18:18Z

src/node_sqlite.cc

+      .As<Name>();
+}
+
+// Returns cached internalized column name strings for this statement,


This comment is incorrect, it returns a boolean.

louwers · 2026-02-26T21:20:17Z

src/node_sqlite.h

  bool allow_unknown_named_params_;
  std::optional<std::map<std::string, std::string>> bare_named_params_;
+  std::vector<v8::Global<v8::Name>> cached_column_names_;
+  int cached_column_names_reprepare_count_ = -1;


Consider using std::optional.

Thanks for the suggestion @louwers but I benchmarked on the iterate path (30 runs) saw a consistent -1% to -2% regression after making these changes:

sqlite/sqlite-prepare-select-all.js statement='SELECT text_column FROM foo LIMIT 1' method='iterate' tableSeedSize=100000 n=100000 *** -2.06 %
sqlite/sqlite-prepare-select-all.js statement='SELECT text_column FROM foo LIMIT 100' method='iterate' tableSeedSize=100000 n=100000 *** -1.42 %

The reason might be std::optional comparison generates more instructions per row than just int

Makes sense. Not worth it then.

yeah it's confirmed, but very interesting

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. sqlite Issues and PRs related to the SQLite subsystem. labels Feb 23, 2026

thisalihassan force-pushed the sqlite-one-byte-encoding branch from 6adb625 to ff6a788 Compare February 23, 2026 14:58

thisalihassan force-pushed the sqlite-one-byte-encoding branch from ff6a788 to 4fcb1ed Compare February 23, 2026 15:05

ronag requested review from addaleax, anonrig and geeksilva97 and removed request for geeksilva97 February 24, 2026 09:45

anonrig reviewed Feb 24, 2026

View reviewed changes

resolve feedback

fd4215d

thisalihassan force-pushed the sqlite-one-byte-encoding branch from ad603d4 to fd4215d Compare February 24, 2026 16:49

thisalihassan requested a review from anonrig February 25, 2026 22:38

louwers reviewed Feb 26, 2026

View reviewed changes

	sqlite3_stmt_status(statement_, SQLITE_STMTSTATUS_REPREPARE, 0);
	sqlite3_stmt_status(statement_, SQLITE_STMTSTATUS_REPREPARE, false);

	int num_cols = sqlite3_column_count(statement_);
	const int num_cols = sqlite3_column_count(statement_);

	int v_len = sqlite3_##from##_bytes(__VA_ARGS__); \
	const int v_len = sqlite3_##from##_bytes(__VA_ARGS__); \

	int len = static_cast<int>(input.size());
	const int len = static_cast<int>(input.size());

Uh oh!

Conversation

thisalihassan commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nodejs-github-bot commented Feb 23, 2026

Uh oh!

thisalihassan commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thisalihassan Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thisalihassan commented Feb 26, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

thisalihassan commented Feb 23, 2026 •

edited

Loading

thisalihassan commented Feb 23, 2026 •

edited

Loading

codecov bot commented Feb 23, 2026 •

edited

Loading

thisalihassan Feb 24, 2026 •

edited

Loading