Conversation
Specifically any C API calls that _may_ invoke a metamethod, but have not necessarily been invoked on an object with a metamethod. This intends to catch issues like the earlier `table.find()` bug in Luau where a `StkId` was being held across `lua_equal()` calls, which became invalid in cases where the input was a table with an __eq that realloc()d
This allows yields in `for i, x in string.gmatch() do ... end` loops, since `gmatch()` returns a generator closure, which may yield now.
Makes it far less annoying to write stateful C++ functions that can wind up native state and jump back into the correct point when resuming.
Now we don't have to do nearly as much stack index juggling, and everything reads a little more naturally.
JSON serialization can get pretty expensive, and with `__tojson` metamethods there's always the risk that you run over your quanta in a decently complex object graph. If everything's yieldable, we don't need to worry as much.
The big one. This opens up a lot of pre-emption opportunities in the string and table library that were not available before, and should quash a ton of existing bugs with seemingly simple string patterns causing you to go over your quanta, getting your script killed.
|
Need to figure out the 32bit crashes (probably ASAN overhead due to some test using too much memory) as well as ignore the interval delta check in coverage mode. Win64 fix should be straightforward, just don't use named designators in the struct literals. |
|
Hmmm. Progress, but still using too much memory under 32-bit ASAN and timing tests are flaky on win32. I don't love timing tests in general due to flakiness issues, but it's kind of necessary since our needs are timing-based. Maybe we can use something more accurate like number of cycles executed on the current thread, then convert to rough duration? |
|
Got 32-bit ASAN builds working by just limiting the quarantine zone size on 32-bit. Not ideal, but we only really want 32-bit ASAN to catch any obvious errors that are 32-bit specific. |
1 participant
Totally overhauls how / where native C++ functions are able to yield. Involves a rewrite of some fairly core parts of the stdlib (
string.gsub(),table.sort(), etc).Also includes a (to my knowledge) somewhat novel yieldability strategy for Lua in the lyieldable framework, which takes advantage of the fact that our functions are layered on top of the Lua VM to wind / unwind our execution state to / from a VM-managed buffer. The yieldability code is not for the faint of heart, but trust me, it's much easier than writing these things by hand, and the existing solutions I'd seen like
rcorodidn't fit our needs. This was pretty heavily inspired by theUThreadInjector/ C#async/awaittransforms.Still needs a bit of work, but should cover most of the cases where someone could intentionally or unintentionally end up doing tons of work between yield checks and getting their script killed.
I'd recommend reading the commits in order since they tell a story, and introduce the concepts in a more easy to understand way.
Fixes #40