Modernize for current Linux/Python (BlueZ 5.83, Python 3.12) by exkuretrol · Pull Request #39 · Poohl/joycontrol

exkuretrol · 2026-05-01T11:17:41Z

Summary

Brings joycontrol back to a working state on modern Linux distributions and Python releases. Verified
end-to-end on Oracle Linux 10.1 (BlueZ 5.83 / Python 3.12), should still work fine on Raspbian / Debian /
Ubuntu.

The branch covers three buckets of work: (1) replace deprecated/removed BlueZ tooling, (2) fix Python
3.10–3.12 asyncio API breakages that surfaced during runtime testing, (3) overhaul the interactive CLI so
it's usable for actual play sessions.

No behavioral change for the protocol layer itself — Switch handshake, NFC/amiibo, controller emulation are
unchanged. All fixes are below the protocol or in the CLI/host layer.

Modernization

hciconfig / hcitool → btmgmt. The legacy bluez CLI tools are no longer shipped on recent
Debian/Ubuntu/RHEL/Oracle Linux. joycontrol/device.py now prefers btmgmt (bluez-tools) for adapter
reset, device-class setting, and BD-address changes; falls back to hciconfig / hcitool / bdaddr only
when present, and to a raw HCI socket (socket.AF_BLUETOOTH, BTPROTO_HCI) for vendor-specific commands
as a last resort. scripts/change_btaddr.sh and the README follow the same precedence.
pkg_resources → importlib.resources for the bundled SDP record XML.
Asyncio cleanup — asyncio.get_event_loop() at module/import time replaced with get_running_loop()
inside coroutines and asyncio.run() at entry points. The loop=asyncio.get_event_loop() mutable default
in AsyncHID.__init__ is gone.
Packaging — setup.py bumped to 0.16, python_requires>=3.9, drops the unused aioconsole, adds
crc8 and prompt-toolkit to install_requires.

Python 3.10/3.12 runtime fixes

These were not theoretical — each one reproduced in real pairing sessions on the test host before the fix.

File	Issue
`joycontrol/protocol.py`	`connection_lost` called `Task.set_exception()` on a Task created via `asyncio.ensure_future`. Disallowed since 3.10 ("Task does not support set_exception operation"). Switched to `loop.create_future()` driven by a helper waiter task.
`joycontrol/protocol.py`	`_writer` had a bare `except: break` that silently swallowed `asyncio.CancelledError`. On 3.12 (where `CancelledError` derives from `BaseException`) the writer died after handshake → input-report stream stopped → Switch dropped the controller. Replaced with targeted handlers + traceback log. This was the actual root cause of "pair didn't succeed" against Python 3.12.
`joycontrol/my_semaphore.py`	`_Request` used `loop.create_future()` from `self._loop`, but `asyncio.Semaphore` no longer exposes `_loop` on 3.10+. Every transport write raised `AttributeError: 'NoneType' object has no attribute 'create_future'`, killing the writer. Now resolved via `asyncio.get_running_loop()`.
`joycontrol/protocol.py`	Writer task started via `start_asyncio_thread(self._writer())` without `ignore=asyncio.CancelledError`, so clean shutdown logged "Exception in callback" tracebacks. Matches the convention used elsewhere in `transport.py`.
`joycontrol/protocol.py` / `joycontrol/transport.py`	`connection_lost` always logged at ERROR — even on a user-initiated `exit`. Now logs INFO when `exc is None` (clean), ERROR when an exception is provided; transport's write-error path passes the exception.
`joycontrol/protocol.py`	"Code is running X s too slow!" warnings spammed during normal flow-control pauses. Now only fires for genuine stalls (>100 ms).
`joycontrol/server.py`	`-r auto` picker called `paths.items()` on a list, crashing with `AttributeError` whenever multiple Switches were paired.

CLI overhaul

prompt_toolkit-based prompt replaces aioconsole.ainput:
- Tab completion — registered commands + button names + stick keywords.
- ↑ / ↓ history, persisted at ~/.local/state/joycontrol/cli_history (overridable via
  $JOYCONTROL_STATE_DIR or $XDG_STATE_HOME).
- Logs render above the prompt via patch_stdout; the cmd >> line stays clean while logs render on
  lines above. Logging is reconfigured to write to stdout for this to take effect.
- mash / test_buttons stop-on-Enter fixed. They had been stuck on aioconsole.ainput after the
  prompt switched, racing prompt_toolkit on stdin. Now use a single wait_for_enter helper, and the
  inter-press sleep races against the stop signal so Enter takes effect immediately.
Ctrl-C / Ctrl-D at the prompt now exit cleanly.

Picker UX (when `-r auto` is in effect)

controller arg is now optional — defaults to PRO_CONTROLLER. Uses argparse choices= so
--help always shows the valid set.
-r defaults to 'auto', falling through to initial pairing if no Switch is bonded. -r "" or -r none opts out for users who want a forced pair.
Bare sudo run_controller_cli.py now does what you almost always want: emulate Pro Controller, reconnect
to the most-recently-bonded Switch.
Interactive picker when ≥1 Switch is paired:
```
found the following paired switches, please choose one:
 1: /org/bluez/hci0/dev_AA_..  (last bond: 2026-05-01 16:36:48)
 2: /org/bluez/hci0/dev_BB_..  (last bond: 2026-05-01 14:31:04)
 n: pair a new Switch
 u: unpair a Switch
 0: abort
number 1 - 2, n to pair new, u to unpair, 0 to abort [1]:
```
- Sorted by last-bond mtime — most recent is option 1.
- n falls through to initial pairing without disturbing existing bonds (the old "do you want to unpair
  every paired Switch?" gauntlet is gone).
- u does an org.bluez.Adapter1.RemoveDevice with a y/N confirm; menu reflows.
- Bad input no longer aborts — picker reprints with a hint.
scripts/list_paired_switches.sh — shell helper that lists paired devices for the default adapter
sorted by /var/lib/bluetooth/<adapter>/<dev>/info mtime. Useful for finding the address to pass to -r.

README / setup

Rewrote Installation and Bluetooth service setup:

Fedora / RHEL / Oracle Linux dnf line alongside the existing apt line, with a CodeReady Builder
enable command for bluez-libs-devel.
Pivoted Python install instructions to pip install . (or a project-local venv).
Replaced "edit /lib/systemd/system/bluetooth.service" with a systemd drop-in override at
/etc/systemd/system/bluetooth.service.d/override.conf (won't be clobbered by package upgrades).
Documents the bluetoothd path difference between Debian-likes (/usr/lib/bluetooth/bluetoothd) and
RHEL-likes (/usr/libexec/bluetooth/bluetoothd).

Testing

Verified on:

Oracle Linux 10.1 (kernel 6.12, BlueZ 5.83, Python 3.12.12, btmgmt only — no
hcitool/hciconfig/bdaddr available)
- Initial pairing with two different Switch consoles ✅
- Reconnect via -r auto (single + multiple-Switch picker) ✅
- Buttons / stick / NFC amiibo read ✅
- Clean exit (exit, Ctrl-C, Ctrl-D) without tracebacks ✅
- mash with decimal interval — Enter stops within one interval ✅

Compile-tested all touched Python files with python3 -m py_compile.

Migration notes

Existing users using -r auto see the new picker (with n / u options) instead of the old
behavior. If you don't want the picker, pass an explicit -r <bdaddr>.
Existing users running on a host without bluez-tools: install it (apt install bluez-tools / dnf install bluez-tools) — the project warns and falls back to hciconfig/hcitool if those exist, but the
recommended path is btmgmt.
The aioconsole Python dependency is removed; it's safe to pip uninstall aioconsole.
prompt-toolkit is a new runtime dep.

Test plan

Install on a clean target (Debian/Ubuntu and Fedora/RHEL family)
Initial pair via n in the picker, with another Switch already bonded
Reconnect to specific BD addr via -r <addr>
u/unpair from the picker, then immediate re-pair
Run mash a 0.5 and confirm Enter stops it promptly
Tab completion / ↑↓ history work in the prompt
Clean exit shows Connection closed. at INFO, no traceback

hciconfig and hcitool are deprecated on modern distros (e.g. recent Debian/Ubuntu, Oracle Linux 10) where bluez no longer ships them. The project now prefers btmgmt (bluez-tools) for adapter reset, class setting, and BD address changes, falling back to the legacy tools when present, and to raw HCI sockets via socket.AF_BLUETOOTH as a last resort. Also drop deprecated Python idioms: - pkg_resources -> importlib.resources for the bundled SDP record - asyncio.get_event_loop() -> asyncio.get_running_loop() inside coros, asyncio.run() at entry points; remove the mutable-default loop=asyncio.get_event_loop() arg evaluated at import time - bump python_requires to >=3.9, add crc8 to install_requires Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Two upstream bugs surfaced when running under Python 3.12: 1) `_writer` had a bare `except: break` that silently swallowed every exception, including asyncio.CancelledError (which derives from BaseException since 3.8). The result on 3.12: the writer task exits immediately after starting, the input-report stream stops, and the Switch refuses to accept the controller as a player. Replace with targeted handlers: re-raise CancelledError, log NotConnectedError at info, log every other exception with a traceback before breaking. 2) `connection_lost` called `set_exception()` on `self._controller_state_sender`, which was created via `asyncio.ensure_future(...)` and is therefore a Task. Python 3.10+ disallows `Task.set_exception()` ("Task does not support set_exception operation"). Switch the sender to a plain Future created via `loop.create_future()`, driven by a helper waiter task that mirrors the sig_is_send event onto the future. The Future still supports set_exception(), so disconnect handling works again. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

asyncio.Semaphore stopped accepting/storing a `loop` parameter in Python 3.10, and on 3.12 `self._loop` is None on the subclass. The writer task crashed with `AttributeError: 'NoneType' object has no attribute 'create_future'` on every input report, so the input-report stream never started and the Switch wouldn't accept the controller. Resolve the loop the supported way — `asyncio.get_running_loop()` from inside the coroutine — when constructing the per-request Future. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

When the CLI exits, transport.close() cancels the writer task. The writer correctly propagates CancelledError, but the task's done callback (create_error_check_callback) re-raised it, which asyncio's default handler then logged twice as "Exception in callback". All the other start_asyncio_thread() callers in transport.py already pass ignore=asyncio.CancelledError. The writer thread had been missing this. Match the others. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

When the user types `exit`, transport.close() calls connection_lost(None). Per asyncio convention, exc=None means clean shutdown — but the protocol was logging it as ERROR ("Connection lost."), which made every clean exit look like a failure. - protocol.connection_lost(): log INFO when exc is None, ERROR when an exception is provided. - transport.write(): when an OSError/ConnectionResetError forces us into connection_lost(), pass the exception so the log path knows it's a real loss and not a clean close. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

bluez already tracks per-device bond state in /var/lib/bluetooth/<adapter>/ <addr>/info, and the file's mtime gets bumped whenever the bond is touched (connect / disconnect / link-key update). That makes it a reasonable proxy for "most recently connected" without joycontrol having to track its own connection log. The script lists paired devices for the default adapter, sorted by mtime descending, with the device name and bond timestamp. Useful for finding the address to pass to `run_controller_cli.py -r <addr>`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The writer loop targets a 1/60 s cadence in input-report modes 0x30/0x31. Routine sub-100 ms overruns happen on every connection — Linux scheduler jitter and L2CAP flow-control waits where the writer correctly blocks on _write_lock but active_time still accounts for the wait. Logging every one of those at WARNING level was just noise, and drowned out actual stalls worth investigating. Only warn for overruns greater than 100 ms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

server.py iterated `paths.items()` for the interactive picker, but `HidDevice.get_paired_switches()` returns a list, not a dict — so running `run_controller_cli.py -r auto` against a host with more than one paired/cached Switch crashed with AttributeError. Use `enumerate(paths, start=1)` to match the 1-based prompt that follows ("number 1 - N [1]:"). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

When `-r auto` finds multiple paired Switches, the picker now shows the last bond mtime alongside each DBus path so you can tell which Switch is the most recent. It also adds a "0: abort" option (or 'q'/'Q') so you can back out of the prompt without picking blindly. Bond mtime is read from /var/lib/bluetooth/<adapter>/<addr>/info — the same file timestamp used by scripts/list_paired_switches.sh. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The old aioconsole.ainput-based prompt had three usability problems: - log lines printed mid-input garbled the `cmd >>` prompt and any text the user had typed - no tab completion - no command history / arrow-key recall across runs Replace it with prompt_toolkit's PromptSession: - patch_stdout(raw=True) buffers writes during the prompt and replays them on the line above, so logs no longer collide with input - WordCompleter built from registered cmd_* methods, dynamic commands, button names (for ControllerCLI), and stick keywords gives Tab completion for the things you actually type - FileHistory at $JOYCONTROL_STATE_DIR (or XDG_STATE_HOME or ~/.local/state)/joycontrol/cli_history persists Up/Down history between runs logging_default now writes to sys.stdout instead of the default stderr, so patch_stdout intercepts log output. Ctrl-D / Ctrl-C at the prompt exit cleanly via the existing return path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Two bugs in run_controller_cli.py's mash/test_buttons commands: 1) They still used aioconsole.ainput, which competed for stdin against prompt_toolkit's terminal-mode handling left over from the parent CLI session. Result: pressing Enter never resolved the input future and the loop ran forever until Ctrl-C. 2) Even if Enter had been received, the loop awaited `asyncio.sleep(interval)` between button pushes, so a stop only took effect after the next full interval elapsed — visibly broken for `mash x 0.5` and worse for longer intervals. Add joycontrol.command_line_interface.wait_for_enter() that uses a fresh PromptSession (single source of truth for stdin in this process) and call it from both spots. Race the inter-button sleep against the stop signal via asyncio.wait_for(shield(user_input), timeout=...) so Enter ends the loop immediately. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Most invocations of run_controller_cli.py needed -r auto to reconnect to an already-paired Switch. Make that the default so a bare invocation "just works": - argparse default for --reconnect_bt_addr is now 'auto'. - create_hid_server() normalizes 'auto' to None when no Switch is paired, falling through to the initial-pairing flow instead of fatal-exiting. - '' or 'none' (case-insensitive) are accepted as an explicit opt-out for users who really want to force initial pairing even with bonds present. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Make the positional `controller` arg optional with a PRO_CONTROLLER default. The bare invocation `run_controller_cli.py` now sets up a Pro Controller, which combined with -r defaulting to 'auto' lets you reconnect to your last Switch with zero arguments. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Replace the manually-typed "JOYCON_R, JOYCON_L or PRO_CONTROLLER (default: PRO_CONTROLLER)" help string with `choices=[c.name for c in Controller]`, which makes argparse: - show the valid options in the auto-generated usage line - reject invalid values with a clear "invalid choice" error before Controller.from_arg gets called - stay in sync if a new Controller variant is ever added argparse appends the default to the help automatically when "%(default)s" is used, but here `choices=` plus the existing default already produce the right rendering, so the help text can just describe purpose. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Sort `paths` from get_paired_switches() by the bond file's mtime descending before showing the picker. Effects: - Interactive picker: option 1 is always the most-recently-bonded Switch — what you almost always want when reconnecting. - Non-interactive `-r auto` with multiple paired Switches: previously picked an arbitrary entry (DBus iteration order); now picks the most recent one. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Extract the auto-resolution logic into _resolve_auto() and run it as part of reconnect_bt_addr normalization, before the if/else split. This lets the picker fall through to initial pairing by simply returning None — no special-case plumbing needed in the caller. User-visible changes: - The picker now shows even when only one Switch is paired (so you can pair a new one without first unpairing what you have). - New 'n' (or 'new') option pairs a fresh Switch via the existing initial-pairing flow. - Existing '0' / 'q' (abort) and Enter (most-recent default) still work. - Numeric choices outside the valid range now abort cleanly with a message instead of crashing with IndexError. Sample prompt: found the following paired switches, please choose one: 1: /org/bluez/hci0/dev_AA_.. (last bond: 2026-05-01 16:36:48) 2: /org/bluez/hci0/dev_BB_.. (last bond: 2026-05-01 14:31:04) n: pair a new Switch 0: abort number 1 - 2, n to pair new, 0 to abort [1]: Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The 'Warning: a switch was found paired, do you want to unpair it?' prompt fired on every initial-pairing attempt for every existing bond, forcing the user through a y/n cycle just to pair an additional Switch. Now that the auto picker has explicit 'pair new' / 'reconnect' / 'abort' options, the user has already made the connect-vs-pair choice upstream. The unpair prompt offered no useful new information — it just disturbed unrelated bonds. Drop the prompt and the matching non-interactive 'switches are paired' warning. The SDP-record-count warning is preserved because it's still informational about Switch compatibility. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Extract the picker into a reusable read-eval loop so the user can: - 1..N: reconnect to that paired Switch - Enter: most recent (default) - n / new: fall through to initial pairing - u / unpair: pick a Switch to forget, with y/N confirm; menu re-prints - 0 / q: abort Bad/out-of-range input no longer aborts the script — the menu just reprints with a hint, since the user is now in an interactive loop. The unpair path uses HidDevice.unpair_path (already implemented) which calls org.bluez.Adapter1.RemoveDevice — same effect as `bluetoothctl remove <addr>` — and reflows the menu on success. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Installation: - Add Fedora/RHEL/Oracle Linux dnf line alongside the apt line. - Pivot to `pip install .` (or local venv) since setup.py now lists the full runtime dep set (hid, aioconsole, crc8, prompt-toolkit, dbus-python). - One-line verification command for installed deps. Bluetooth service setup: - Replace the "edit /lib/systemd/system/bluetooth.service" instructions with a systemd drop-in override at /etc/systemd/system/bluetooth. service.d/override.conf, which survives package upgrades. - Document the bluetoothd path difference between Debian-likes (/usr/lib/bluetooth/bluetoothd) and RHEL-likes (/usr/libexec/bluetooth/bluetoothd) and how to check. - Spell out which services break (input/sap/avrcp) so users know what they're trading away. CLI section: - Refresh argparse usage block (controller arg now optional, -r defaults to "auto", `-r ""`/`none` opt-out documented). - Document the new picker (1..N / n / u / 0) with a cheat-sheet table. - Mention scripts/list_paired_switches.sh. - Document tab completion, history, patch_stdout behavior in the prompt. Issues section: refresh the "reconnect spins" entry to point at the new in-picker unpair option instead of dropping out to bluetoothctl. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

On Oracle Linux / RHEL / AlmaLinux / Rocky, bluez-libs-devel ships in the CodeReady Builder (CRB) repo, which isn't enabled by default. Add the per-distro enable command so first-time installers don't get a "package not found" error from the dnf line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

aioconsole.ainput was the original stdin reader; the prompt_toolkit refactor replaced every call site (CLI prompt, mash, test_buttons, the various 'press <enter> to stop' blockers). The package isn't imported anywhere now — the only remaining references are dead install_requires + verification command lines. Keep the historical mention in command_line_interface.wait_for_enter's docstring since it explains why we route stdin through prompt_toolkit sessions exclusively (avoiding the two-readers-on-stdin race that broke mash's stop-on-enter under terminal raw mode). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…m-site-packages Two real issues hit while dogfooding the README install on a clean Oracle Linux 10 host: 1. `pip install .` failed with `metadata-generation-failed` for dbus-python. Listing it in install_requires triggers a build from source (no compiler in venv → fails). Drop it from install_requires with an in-tree comment explaining why; treat it as a system package only. 2. `bluez-tools` doesn't exist on RHEL family — `btmgmt` ships in the main `bluez` package there. Adjust the dnf line and document the distro split. README now uses `python3 -m venv --system-site-packages .venv` so the distro python3-dbus stays visible inside the project venv. The verify command runs against the venv interpreter. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Walks through the full setup from a clean clone, written for readers who've never set up a Python venv or BlueZ before. Each step explains *why* it's needed, not just what to type. Installation: 1. git clone (start from zero) 2. distro-package install (apt vs dnf, with CRB enable for RHEL) 3. python3 -m venv --system-site-packages .venv (with explanation of why the flag matters — system dbus visibility) 4. sudo .venv/bin/pip install . (with a note on why sudo) 5. import-test verification with the failure-mode hint Bluetooth setup also reformatted as numbered steps, with a small table mapping distro family to bluetoothd binary path so users can fill their own value into the drop-in override correctly. Adds an explicit "how to revert" line and an adapter-existence sanity check. No behavior change — README only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

`sudo python3 run_controller_cli.py` would run the script under the system Python, which doesn't see the venv's prompt-toolkit / hid / crc8 / etc., producing "ModuleNotFoundError: No module named 'prompt_toolkit'" right at startup. Using `.venv/bin/python` invokes the venv interpreter directly without needing source-activate. Add an upfront `cd /path/to/joycontrol` and explain the .venv/bin/ prefix for first-time readers. Also call out that list_paired_switches.sh is run from the project directory. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Chiawei Chen and others added 21 commits May 1, 2026 14:27

exkuretrol marked this pull request as ready for review May 1, 2026 11:18

Chiawei Chen and others added 3 commits May 1, 2026 19:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Modernize for current Linux/Python (BlueZ 5.83, Python 3.12)#39

Modernize for current Linux/Python (BlueZ 5.83, Python 3.12)#39
exkuretrol wants to merge 24 commits intoPoohl:amiibo_editsfrom
exkuretrol:modify-the-project-to-modern-pythonos-capatible

exkuretrol commented May 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

exkuretrol commented May 1, 2026

Summary

Modernization

Python 3.10/3.12 runtime fixes

CLI overhaul

Picker UX (when -r auto is in effect)

README / setup

Testing

Migration notes

Test plan

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Picker UX (when `-r auto` is in effect)