Add hid_set_num_input_buffers() API by auxcorelabs · Pull Request #787 · libusb/hidapi

auxcorelabs · 2026-04-17T08:55:21Z

Exposes a new public function hid_set_num_input_buffers(dev, num_buffers) to resize the per-device input report queue.

High-throughput HID devices (medical telemetry, high-poll-rate gaming peripherals, data acquisition hardware) can emit bursts of input reports that exceed the current hardcoded queue sizes (30 on macOS and Linux/libusb, 64 on Windows). When a burst exceeds the queue, reports are silently dropped with no error indication to the caller. Observed on hardware emitting ~90 reports in 200 ms bursts: both the macOS and Linux/libusb backends silently drop ~20% of frames.

Per-backend behavior

macOS: resizes the userspace input-report queue
Linux libusb: resizes the userspace report queue
Windows: wraps HidD_SetNumInputBuffers (parity with existing behavior)
Linux hidraw / NetBSD: no-op (kernel manages buffering — documented as a @note on the API declaration in hidapi.h)

Safety

Values outside [1, HID_API_MAX_NUM_INPUT_BUFFERS] return -1 and register a descriptive error on the device (via the backend's register_device_error helper where available)
Thread-safe: setter uses the same lock (dev->mutex / dev->thread_state) as the respective report callback

Design notes

Backwards compatibility: Purely additive public API. Defaults per backend are unchanged, so existing callers see no behavioral change. hid_device is opaque in hidapi.h, so struct field additions in backend-private files are not an ABI break.
Why 1024 cap: Bounds per-device memory to roughly 1 MB on USB high-speed links, preventing memory exhaustion from a caller that passes INT_MAX (malicious or buggy). We've measured bursts up to ~90 reports deep in practice; 1024 leaves room for devices ~10× more demanding. Overridable at build time via -DHID_API_MAX_NUM_INPUT_BUFFERS=N for memory-constrained targets.
No getter: hidraw and NetBSD back the queue in the kernel with sizes not portably exposed to userspace, so a cross-platform getter couldn't return a meaningful value there.
Why not hid_get_input_report() instead? That API issues a host-initiated GET_REPORT request (via the control endpoint or OS equivalent). It does not drain the interrupt-endpoint queue that streaming devices fill, and in practice many devices either don't implement GET_REPORT for input reports or respond incorrectly. The two APIs use different USB transfer mechanisms (interrupt endpoint vs. control endpoint); this PR fixes the interrupt-streaming path.

Userspace queue: linked list → ring buffer

The latest commit replaces the linked list with a fixed-size ring:

O(1) enqueue/dequeue. hid_report_callback / read_callback no longer scan a tail on every incoming report.
Exact cap semantics. Setter calls hidapi_input_ring_drop_oldest on shrink, so dev->num_input_buffers is the precise steady-state queue length (matches Copilot's "limit corresponds precisely to num_input_buffers" request).
Clean allocation-failure handling. The previous code had an unchecked allocation that would segfault on OOM. The new push returns -1 cleanly; libusb drops the report silently (no active error channel), mac calls register_device_error.
ABI unchanged. hid_device is opaque; the struct field swap is invisible to callers.
Implementation. hidapi_input_ring_* static-in-header helper, present as byte-identical copies in libusb/ and mac/.

References

when I use hidapi, how to use 'HidD_SetNumInputBuffers' to set Buffer size ? #154 — this PR also addresses the prior request to expose HidD_SetNumInputBuffers functionality to callers

Exposes a new public function to resize the per-device input report queue. High-throughput HID devices (medical telemetry, high-poll-rate gaming peripherals, data acquisition hardware) emit bursts of input reports that exceed the current hardcoded queue sizes (30 on macOS and the libusb backend, 64 on Windows). When a burst exceeds the queue, reports are silently dropped with no error indication to the caller. This adds: - hid_set_input_report_buffer_size(dev, size) in hidapi.h - HID_API_MAX_INPUT_REPORT_BUFFER_SIZE (1024) cap to prevent unbounded memory growth Per-backend behavior: - macOS: resizes the userspace IOHIDQueue-fed report queue - Linux libusb: resizes the userspace report queue - Windows: wraps HidD_SetNumInputBuffers (parity with existing behavior) - Linux hidraw: no-op (kernel manages buffering) - NetBSD: no-op (kernel manages buffering) Defaults are unchanged, so existing callers are unaffected. Values outside [1, HID_API_MAX_INPUT_REPORT_BUFFER_SIZE] are rejected with -1. Thread-safe on macOS (dev->mutex) and libusb (dev->thread_state), matching the locks used by the respective report callbacks. Addresses the same need as closed issue libusb#154 (HidD_SetNumInputBuffers exposure) and complements libusb#725 (callback-based input API).

Youw

refer to comments, otherwise looks good

Per maintainer feedback on PR libusb#787: - Remove if (!dev) validation from all 5 backends. hidapi convention is that device functions trust the caller to pass a valid handle; only hid_close is permitted to accept NULL. - Reword the inline comment in linux/hid.c and netbsd/hid.c to lead with "No-op" so the caller-visible behavior is explicit at the implementation site.

auxcorelabs · 2026-04-17T16:21:36Z

Thanks for reviewing. I have made the changes.

JoergAtGithub · 2026-04-18T11:41:56Z

I think the name of the Win32-API HidD_SetNumInputBuffers describes better what it does than hid_set_input_report_buffer_size, as it does change the number of buffers and not the size of the bufferin bytes.

… controls the number of input report buffers, not their byte size. - Function: hid_set_input_report_buffer_size -> hid_set_num_input_buffers - Macro: HID_API_MAX_INPUT_REPORT_BUFFER_SIZE -> HID_API_MAX_NUM_INPUT_BUFFERS - Parameter: buffer_size -> num_buffers - Error string: "buffer_size out of range" -> "num_buffers out of range"

auxcorelabs · 2026-04-19T05:59:20Z

Good callout @JoergAtGithub. Have renamed the function to hid_set_num_input_buffers() and made the associated changes.

@Youw - Please let me know if any further changes required.

Thanks both.

mcuee · 2026-04-19T08:01:47Z

@auxcorelabs
Just wondering if you can enhance the hidtest application to include this new API.

If not, do you have a simple test to share? Thanks.

@note

…ile commentary, add hidtest coverage - hidapi/hidapi.h: replace the Defaults per backend list with an @note Per-backend behavior block covering macOS / Windows / libusb / hidraw / uhid semantics, ranges, and defaults. Per @Youw, the public header is the canonical place for the cross-backend contract. - linux/hid.c, netbsd/hid.c: drop the comment that cross-referenced other backends. The (void)num_buffers; idiom and the header contract speak for themselves. - libusb/hid.c: drop the self-scoped no-error-registration note for the same reason. - hidtest/test.c: add a compile-time symbol reference and a runtime call hid_set_num_input_buffers(handle, 500) right after hid_open() succeeds, per @mcuee. Both guarded on HID_API_VERSION >= 0.16.0 so they activate in the 0.16 release cycle, matching the precedent of hid_send_output_report at 0.15.0.

Youw

So far so good.

Copilot

Pull request overview

Adds a new public HIDAPI entry point to let callers increase the per-device input report queue depth, addressing silent report drops during bursty/high-throughput input on backends that queue reports in userspace (macOS + Linux/libusb) and exposing the existing Windows kernel buffering control.

Changes:

Introduces hid_set_num_input_buffers(dev, num_buffers) and documents it in hidapi.h (with a max-cap macro).
Updates macOS and Linux/libusb backends to enforce queue limits via a per-device num_input_buffers value (default 30).
Adds no-op stub implementations for Linux hidraw and NetBSD uhid; updates hidtest to exercise the new API when available.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`hidapi/hidapi.h`	Declares and documents the new public API and its max-cap macro.
`mac/hid.c`	Adds per-device buffer cap field, uses it in the report callback, and provides a setter guarded by the device mutex.
`libusb/hid.c`	Adds per-device buffer cap field, uses it in the read callback, and provides a setter guarded by the thread-state mutex.
`windows/hid.c`	Implements the API by forwarding to `HidD_SetNumInputBuffers`.
`linux/hid.c`	Adds a validated no-op stub for the hidraw backend.
`netbsd/hid.c`	Adds a validated no-op stub for the uhid backend.
`hidtest/test.c`	Calls the new API under a version guard to ensure all backends export it.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-21T16:49:37Z

 		/* Pop one off if we've reached 30 in the queue. This
 		   way we don't grow forever if the user never reads
 		   anything from the device. */
-		if (num_queued > 30) {
+		if (num_queued > dev->num_input_buffers) {
 			return_data(dev, NULL, 0);
 		}


The queue-length enforcement here becomes both expensive and semantically fuzzy now that callers can raise num_input_buffers up to 1024. Each report traverses the entire linked list to find the tail (O(n)), which can become a hot path at high report rates, and the current num_queued accounting means the steady-state queue length is not exactly dev->num_input_buffers. Consider tracking a tail pointer and an explicit queued-count (or switching to a ring buffer) so enqueue/drop is O(1) and the limit corresponds precisely to num_input_buffers.

Someone needs to double-check this, but I'm pretty sure Copilot is right here.
We might even need to consider one step further and switch from linked list to a ring buffer as an optimisation

Updated to ring buffer.

Per-backend helper consistency: * linux/hid.c, netbsd/hid.c, mac/hid.c: setter uses register_device_error() instead of register_error_str() directly. Build-time override: * hidapi/hidapi.h: wrap HID_API_MAX_NUM_INPUT_BUFFERS in #ifndef so downstreams can set the cap via -DHID_API_MAX_NUM_INPUT_BUFFERS=<value>. Ring buffer input queue: * New static-in-header helper hidapi_input_ring_*, present in libusb/ and mac/ as byte-identical copies. * libusb/hid.c, mac/hid.c: replace struct input_report * linked list with fixed-size ring. Enqueue is O(1); eviction is inline in push; the setter shrinks via drop_oldest so dev->num_input_buffers is the exact steady-state cap. * ABI unchanged (hid_device is opaque in hidapi.h). * Allocation failure in the read callback is now handled — the previous code had an unchecked malloc() that would segfault. libusb has no active error channel so the drop is silent there; mac calls register_device_error.

Youw · 2026-04-22T21:15:06Z

is this a known implementation for such a ring buffer?
I'd love to at least basic unit-testing for such kind implementation.

We're still having a dynamic allocation each time we try to push/pop, which kind of defeats the purpose of a ring buffer in a first place. Since we know the maximum size of an input report (from a HID Report descriptor), we can pre-allocate a single continues buffer to fit at most num_input_buffers.

Having two idential implementations copy-pasted in two subdirs is not great. Since we already have it in the separate file, lets place it at common place: ../core/hidapi_input_ring.h

Youw

This is an impressive amount of new code contributed in a short amount of time.
What AI tool has been used to generate it? I will not believe it was hand-written.

mcuee added API API change, Version 1 stuff enhancement New feature or request labels Apr 17, 2026