-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Implement epoll APIs in the JS filesystem #27207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
guybedford
wants to merge
5
commits into
emscripten-core:main
Choose a base branch
from
guybedford:epoll
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
6b4741f
Implement epoll for the JS filesystem
guybedford 8bdbad7
pr feedback
guybedford 744d4a1
Deliver epoll callback on the registering thread under pthreads
guybedford f1cefff
Share epoll instance state across dup'd fds
guybedford 01ddd22
dropped pr review
guybedford File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| /* | ||
| * Copyright 2026 The Emscripten Authors. All rights reserved. | ||
| * Emscripten is available under two separate licenses, the MIT license and the | ||
| * University of Illinois/NCSA Open Source License. Both these licenses can be | ||
| * found in the LICENSE file. | ||
| */ | ||
|
|
||
| #pragma once | ||
|
|
||
| #include <sys/epoll.h> | ||
|
|
||
| #ifdef __cplusplus | ||
| extern "C" { | ||
| #endif | ||
|
|
||
| // EXPERIMENTAL. This API is new and may change (signature or semantics) over the | ||
| // next few releases; it is not yet covered by Emscripten's stability guarantees. | ||
| // | ||
| // Register a persistent readiness callback on an existing epoll fd (built with | ||
| // epoll_create1/epoll_ctl): instead of blocking in epoll_wait, the runtime calls | ||
| // `callback` every time the set makes progress, delivering up to `maxevents` | ||
| // ready events. An epoll is a long-lived readiness aggregator, so the interest is | ||
| // armed once and reused across every delivery - no per-spin re-arming. Unlike | ||
| // epoll_wait it never blocks the calling stack, so it works without ASYNCIFY/JSPI. | ||
| // The callback is delivered on the calling thread's event loop: with pthreads the | ||
| // epoll readiness is tracked on the thread that owns the filesystem (the syscalls | ||
| // are proxied there), but each delivery is dispatched back to the thread that | ||
| // registered the callback. | ||
| // | ||
| // While armed it keeps the runtime alive only as long as it can still fire - i.e. | ||
| // while the epoll has at least one open watched fd. Once every watched fd is | ||
| // closed the set is terminal (it can never become ready again) and the callback | ||
| // stops holding the runtime, so no explicit disposal is required in that case. | ||
| // To dispose while open fds remain, either pass a NULL `callback` (any | ||
| // `maxevents`) to unregister, or close the epoll fd. There is at most one | ||
| // callback per epoll: calling again replaces it (it does not stack). `events` is | ||
| // a runtime-owned buffer valid only for the duration of each callback. Returns 0, | ||
| // or a positive errno (EBADF if `epfd` is not an epoll fd, EINVAL). | ||
| // | ||
| // Each registration's trigger mode (set per-fd via epoll_ctl) controls how often | ||
| // the callback fires for it - identically to epoll_wait, so one callback can mix | ||
| // modes: | ||
| // - Level-triggered (the default): the callback fires on the next tick whenever | ||
| // the fd is ready, and keeps firing while it stays ready. The runtime - not | ||
| // the application - drives the loop, so an fd that is structurally always | ||
| // ready and never drained (notably EPOLLOUT on a writable socket) will spin | ||
| // the event loop. Use one of the modes below for such fds. | ||
| // - EPOLLET (edge-triggered): the callback fires once per readiness edge and | ||
| // not again until a fresh edge. Drain the fd fully on each delivery; this is | ||
| // the way to watch an always-writable fd without spinning. | ||
| // - EPOLLONESHOT: the callback fires once for that fd, then the registration is | ||
| // disabled until you re-arm it with epoll_ctl(EPOLL_CTL_MOD). Use it to | ||
| // handle an fd exactly once (e.g. before handing it elsewhere). | ||
| typedef void (*em_epoll_callback)(int epfd, struct epoll_event *events, int nready, void *userdata); | ||
| int emscripten_epoll_set_callback(int epfd, int maxevents, em_epoll_callback callback, void *userdata); | ||
|
|
||
| #ifdef __cplusplus | ||
| } | ||
| #endif |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| /* | ||
| * Copyright 2026 The Emscripten Authors. All rights reserved. | ||
| * Emscripten is available under two separate licenses, the MIT license and the | ||
| * University of Illinois/NCSA Open Source License. Both these licenses can be | ||
| * found in the LICENSE file. | ||
| */ | ||
|
|
||
| // Backs emscripten_epoll_set_callback under PTHREADS: the epoll readiness lives | ||
| // on the thread that owns the filesystem (the epoll syscalls are proxied | ||
| // there), but the user callback must run on the thread that registered it. This | ||
| // mirrors _emscripten_run_callback_on_thread in html5/callback.c, but carries | ||
| // the epoll callback's argument shape (epfd, events, nready, userdata) and, | ||
| // crucially, reports back to the registering thread when a delivery completes | ||
| // so it can pace the next one - a level-triggered fd stays ready until the | ||
| // callback drains it, so the FS thread must wait for that drain before | ||
| // re-deriving, or it would spin re-delivering the same readiness. | ||
|
|
||
| #include <assert.h> | ||
| #include <pthread.h> | ||
| #include <stdint.h> | ||
| #include <stdlib.h> | ||
| #include <string.h> | ||
| #include <sys/epoll.h> | ||
|
|
||
| #include <emscripten/eventloop.h> | ||
| #include <emscripten/proxying.h> | ||
|
|
||
| #include "emscripten_internal.h" | ||
|
|
||
| typedef void (*em_epoll_callback)(int epfd, | ||
| struct epoll_event* events, | ||
| int nready, | ||
| void* userdata); | ||
|
|
||
| typedef struct epoll_callback_args_t { | ||
| em_epoll_callback callback; | ||
| int epfd; | ||
| int nready; | ||
| void* userdata; | ||
| int token; | ||
| struct epoll_event events[]; | ||
| } epoll_callback_args_t; | ||
|
|
||
| // Runs on the registering thread: deliver the ready set to the user callback. | ||
| static void do_epoll_callback(void* arg) { | ||
| epoll_callback_args_t* args = (epoll_callback_args_t*)arg; | ||
| args->callback(args->epfd, args->events, args->nready, args->userdata); | ||
| } | ||
|
|
||
| // Runs back on the FS-owning thread once the delivery above has finished (or | ||
| // was cancelled because the target thread went away): let the JS layer | ||
| // re-derive. | ||
| static void do_epoll_done(void* arg) { | ||
| epoll_callback_args_t* args = (epoll_callback_args_t*)arg; | ||
| _emscripten_epoll_delivery_done(args->token); | ||
| free(arg); | ||
| } | ||
|
|
||
| void _emscripten_epoll_run_callback_on_thread(pthread_t t, | ||
| em_epoll_callback callback, | ||
| int epfd, | ||
| struct epoll_event* events, | ||
| int nready, | ||
| void* userdata, | ||
| int token) { | ||
| em_proxying_queue* q = emscripten_proxy_get_system_queue(); | ||
| // Copy the events out synchronously so the caller's (reused) buffer is free | ||
| // again the moment this returns; freed once the delivery completes. | ||
| size_t bytes = nready * sizeof(struct epoll_event); | ||
| epoll_callback_args_t* args = malloc(sizeof(epoll_callback_args_t) + bytes); | ||
| args->callback = callback; | ||
| args->epfd = epfd; | ||
| args->nready = nready; | ||
| args->userdata = userdata; | ||
| args->token = token; | ||
| memcpy(args->events, events, bytes); | ||
|
|
||
| if (!emscripten_proxy_callback( | ||
| q, t, do_epoll_callback, do_epoll_done, do_epoll_done, args)) { | ||
| assert(false && "emscripten_proxy_callback failed"); | ||
| } | ||
| } | ||
|
|
||
| // Runs on the owning thread: adjust its (thread-local) runtime keepalive so the | ||
| // epoll callback holds the thread it was registered on, not the FS thread. | ||
| static void do_epoll_keepalive(void* arg) { | ||
| if ((intptr_t)arg > 0) { | ||
| emscripten_runtime_keepalive_push(); | ||
| } else { | ||
| emscripten_runtime_keepalive_pop(); | ||
| } | ||
| } | ||
|
|
||
| void _emscripten_epoll_keepalive_on_thread(pthread_t t, int delta) { | ||
| em_proxying_queue* q = emscripten_proxy_get_system_queue(); | ||
| if (!emscripten_proxy_async( | ||
| q, t, do_epoll_keepalive, (void*)(intptr_t)delta)) { | ||
| assert(false && "emscripten_proxy_async failed"); | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we maybe re-use the existing
_emscripten_run_callback_on_threadthat we use elsewhere?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Under proxy to pthread, we need the completion on the FS thread to be able to handle the epoll completions, while also still proxying the event to to the listening thread.