feat: native socket I/O tracking via PLT hooks (PROF-10637)

[jbachorik]

What does this PR do?

Intercepts libc send, recv, write, read via PLT hooking and records
datadog.NativeSocketEvent JFR events with time-weighted inverse-transform
sampling (Poisson process, PID rate control, target ~83 events/s / ~5000/min).

Motivation

Track blocking TCP socket I/O at the libc function level to surface socket
latency and throughput in the Datadog profiler. Netty with Java NIO transport
(the primary use case) goes through libc send/recv/write/read, making
PLT patching the right interception point. Feature is explicitly opt-in via
the natsock profiler argument. Implements PROF-10637.

Configuration

Argument	Effect
natsock	Enable native socket tracking with the default 1 ms initial period
natsock=100us	Enable with an explicit initial sampling period (any parseUnits unit: ns, us, ms, s)

The interval is the initial parameter of the time-weighted Poisson process;
the PID controller adapts it at runtime to hold the total event rate near
TARGET_EVENTS_PER_SECOND (= 83 events/s) across all four hooks combined.

Architecture

flowchart LR
    A["Java code<br/>OutputStream.write()"] --> B["JDK native<br/>libnio/libnet"]
    B --> C{"PLT entry<br/>for write()"}
    C -- patched --> D["NativeSocketSampler::write_hook"]
    D --> E{"fd is TCP<br/>socket?"}
    E -- no --> F["orig write()"]
    E -- yes --> G["orig write()"]
    G --> H{"shouldSample<br/>(duration, op)"}
    H -- no --> I["return"]
    H -- yes --> J["recordEvent"]
    J --> K["JFR buffer"]
    F --> I

Loading

Sampling pipeline

flowchart TD
    A["Hook fires with<br/>duration_ticks"] --> B["PoissonSampler<br/>(thread_local)"]
    B --> C{"interval budget<br/>consumed?"}
    C -- no --> D["reject"]
    C -- yes --> E["compute weight<br/>1 / (1 - exp(-d/interval))"]
    E --> F["RateLimiter::accept()"]
    F --> G{"PID budget<br/>for this epoch?"}
    G -- no --> D
    G -- yes --> H["record event<br/>operation / fd-addr / bytes / weight"]
    H --> I["RateLimiter<br/>maybeUpdateInterval<br/>(PID: P=31 I=511 D=3)"]
    I --> J["next epoch interval"]

Loading

Two thread_local PoissonSampler instances per thread: outbound (send+write)
and inbound (recv+read). Both share one RateLimiter so the ~83 events/s
target is a single budget across all directions.

Hook installation

flowchart TD
    A["NativeSocketSampler::start"] --> B["dlsym RTLD_NEXT<br/>(RTLD_DEFAULT fallback on musl)<br/>resolve _orig_send/recv/write/read"]
    B --> C["patch_socket_functions"]
    C --> D["walk every loaded CodeCache"]
    D --> E{"lib imports<br/>send/recv/write/read?"}
    E -- yes --> F["rewrite PLT entry<br/>to *_hook"]
    E -- no --> G["skip"]
    C --> H["also patch dlopen PLT"]
    H --> I["dlopen_hook wraps<br/>subsequent loads"]
    I --> J["updateSymbols +<br/>install_socket_hooks"]
    J --> E

Loading

Late-loaded libraries (e.g. HotSpot lazily loads libnet.so on the first
java.net.Socket use) are caught by the dlopen wrapper which re-runs
patch_socket_functions after each new library is registered.

JFR event fields (datadog.NativeSocketEvent)

Field	Type	Description
startTime	long	Event start (ticks)
eventThread	Thread	Calling Java thread (constant pool)
stackTrace	StackTrace	Captured stack (constant pool)
duration	long	Syscall duration (ticks)
operation	string	SEND or RECV
remoteAddress	string	ip:port (IPv4 or [ipv6]:port); empty if unknown
bytesTransferred	long	Bytes actually transferred
weight	float	Inverse sampling probability (1 / P)
spanId	long	Datadog span id from active context
localRootSpanId	long	Datadog local root span id
context attrs	strings	User context attributes

sum(weight × duration) is an unbiased estimator of total socket I/O time;
sum(weight × bytesTransferred) estimates total bytes.

Key design notes

• Scope: TCP blocking send/recv/write/read only. UDP (sendto/recvfrom)
  and Netty's native epoll / io_uring transports are explicitly out of scope.
• Fast path: write/read hooks short-circuit on non-socket fds via a
  lock-free std::atomic<uint8_t> _fd_type_cache[65536] — one relaxed atomic
  load per call on the non-socket path.
• Thread context: hook bodies run on the calling Java thread (not a signal
  handler), so malloc and std::mutex are safe.
• PLT re-patching on dlopen: install_socket_hooks() (called from
  dlopen_hook after updateSymbols) re-runs patch_socket_functions so
  lazily-loaded libraries (libnet.so, libnio.so, native Netty transports)
  are patched the moment they appear.
• fd-to-address cache: intentionally not cleared on JFR chunk boundaries;
  stale addresses for reused fds are a known, accepted trade-off to avoid
  racing with in-flight recordings.
• Original function pointer lifecycle: _orig_send/recv/write/read are
  assigned once (when _socket_size == 0) and are intentionally not nulled in
  unpatch_socket_functions to avoid a memory-ordering race with in-flight
  hook invocations that may still be executing.
• Platform scope: #if defined(__linux__) — compiled as no-op stubs on
  macOS. Runs on both glibc and musl Linux; on musl RTLD_NEXT returns NULL
  (libc precedes the profiler DSO in the link map), so RTLD_DEFAULT is used
  as fallback to locate the real libc symbols. Feature is additionally disabled
  in Java tests on J9/Zing via Platform.isJ9() / Platform.isZing() guards.
• NativeSocketEvent lives in nativeSocketSampler.h (not the shared
  event.h) since it is only used by NativeSocketSampler.

How to test the change?

• C++ unit tests — NativeSocketSamplerHookTest in
  ddprof-lib/src/test/cpp/nativeSocketSampler_ut.cpp — verifies that
  send_hook/recv_hook/write_hook/read_hook each delegate to the
  installed _orig_* pointer and return its value.

• JUnit integration tests in
  ddprof-test/src/test/java/com/datadoghq/profiler/nativesocket/:

  Test	What it verifies
  NativeSocketEnabledTest	Events produced when feature is enabled
  NativeSocketDisabledTest	No events when feature is not enabled
  NativeSocketEventFieldsTest	All required JFR fields present and valid
  NativeSocketEventThreadTest	eventThread populated with the calling thread
  NativeSocketStackTraceTest	Stack trace captured on events
  NativeSocketRemoteAddressTest	remoteAddress in ip:port format
  NativeSocketSendRecvSeparateTest	SEND and RECV tracked independently
  NativeSocketBytesAccuracyTest	sum(weight × duration) unbiased estimator within tolerance
  NativeSocketRateLimitTest	Event count ≪ operation count (PID rate control active); weight > 1 on sampled events
  NativeSocketRestartTest	Events produced after profiler stop/restart cycle
  NativeSocketUdpExcludedTest	UDP (sendto/recvfrom) produces zero events
  NativeSocketMacOsNoOpTest	No events on macOS (no-op stub)

Spec: #487

For Datadog employees

• If this PR touches code that signs or publishes builds or packages, or handles
  credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
• This PR doesn't touch any of that.
• JIRA: PROF-10637

[dd-octo-sts]

CI Test Results

Run: #25169060411 | Commit: a87999b | Duration: 33m 48s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK	glibc-aarch64/debug	glibc-amd64/debug	musl-aarch64/debug	musl-amd64/debug
8	-	✅	-	-
8-ibm	-	✅	-	-
8-j9	✅	✅	-	-
8-librca	-	-	✅	✅
8-orcl	-	✅	-	-
11	-	✅	-	-
11-j9	✅	✅	-	-
11-librca	-	-	✅	✅
17	✅	✅	-	-
17-graal	✅	✅	-	-
17-j9	✅	✅	-	-
17-librca	-	-	✅	✅
21	✅	✅	-	-
21-graal	✅	✅	-	-
21-librca	-	-	✅	✅
25	✅	✅	-	-
25-graal	✅	✅	-	-
25-librca	-	-	✅	✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0

---

Updated: 2026-04-30 14:25:56 UTC