Detecting xruns in process callback

[dj-foxxy]

Hi and thanks for the great library, been using it for years!

What's the best way to detect if the last process callback lead to an xrun? Here's my use case: I'm sending MIDI, which works 99% of the time. But when an xrun occurs, the MIDI events aren't sent to connected inputs and on the next process callback I clear the buffer and they're lost forever. Here's how I'm currently doing it:

jack_midi_event_write = jack._lib.jack_midi_event_write

jack_port_get_buffer = jack._lib.jack_port_get_buffer


class Process:
    __slots__ = ('_before', '_buffer', '_client', '_port', '_xrun')

    def __init__(self, client: jack.Client, port: jack.OwnMidiPort) -> None:
        self._before = 0;
        self._buffer = jack.RingBuffer(2 ** 8)
        self._xrun = threading.Event()
        self._client = client
        self._port = port

    def process(self, frames: int) -> None:
        xrun = self._xrun
        xrun_set = xrun.is_set()
        port = self._port
        client = port._client
        last_frame_time = client.last_frame_time
        blocksize = client.blocksize

        if not xrun_set and last_frame_time - blocksize == self._before:
            port.clear_buffer()
        if xrun_set:
            xrun.clear()

        self._before = last_frame_time
        src_buffer = self._buffer
        space = src_buffer.read_space

        if space != 0:
            data = src_buffer.read(space)
            dst_buffer = jack_port_get_buffer(port._ptr, blocksize)
            i = 0
            while i != space:
                jack_midi_event_write(dst_buffer, 0, data[i : i + 3], 3)
                i += 3

    def handle_xrun(self, delay_usec: float) -> None:
        self._xrun.set()

    def _append(self, event: Tuple[int, int, int]) -> None:
        self._buffer.write(bytes(event))

    def note_on(self, note: int) -> None:
        self._append((0x99, note, 100))

    def note_off(self, note: int) -> None:
        self._append((0x89, note, 0))

I'm doing two things:

1. I'm comparing last_frame_time - blocksize to the last_frame_time on the previous process callback.

2. Using the set_xrun_callback to set a threading.Event()

I'm not sure if I'm doing any of this correctly, just experimenting to see what works. I know you're not meant to use Python for realtime, but it's working too well to justify moving to C++.

Thanks.

[mgeier]

Why are you using stuff from jack._lib?
This is considered a "hidden" implementation detail.
You can of course use it if there is no other way, but it shouldn't be your first choice.

Why not use write_midi_event() and get_buffer()?

If there is no reason, can you please update your code example?

[dj-foxxy]

Hi @mgeier. Sorry I should have posted a simplified example. I'm just calling jack._lib stuff directly to shave off a few microseonds (it's just the content of write_midi_event) without the buffer support. Really I just want to know the best way of detecting if an xrun happened in the last process callback, meaning that the midi notes did not get sent to connected inputs. Basically this bit of the code:

        if not xrun_set and last_frame_time - blocksize == self._before:
            port.clear_buffer()

At the moment, if the xrun callback is called or the frame times do not add up, I guess the last process callback resulted in an xrun and I don't clear the buffer. I want to know if that's the best way or if there's something simpler/more robust.

Here's a standard version of the process callback:

    def process(self, frames: int) -> None:
        expected_frame_time = self._client._last_frame_time - self._client.blocksize \
                == self._before

        if expected_frame_time and self._xrun.set():
            self._port.clear_buffer()

        if self._xrun.set():
            self._xrun.clear()

        self._before = self._client.last_frame_time

        if self._buffer.read_space == 0:
            return

        data = self._buffer.read(self._buffer.read_space)
        for i in range(0, self._buffer.read_space, 3):
            self._port.write_midi_event(0, data[i : i + 3])

[mgeier]

I'm just calling jack._lib stuff directly to shave off a few microseonds (it's just the content of write_midi_event) without the buffer support.

Ah, I see. You mainly want to avoid calling jack_port_get_buffer() multiple times per callback (plus avoid a few Python function calls and name lookups).

I'm not really familiar with the JACK MIDI API, but isn't this the reason why reserve_midi_event() exists?

[mgeier]

Coming back to your original question ...

Since the xrun callback seems to run in a separate non-realtime thread, I guess using a threading.Event makes sense in a Python context. This isn't great, as you mentioned, because it involves some kind of a lock, but we are not worrying about this now.

I guess theoretically multiple process callbacks could be called before the xrun callback is invoked, but I don't know if this can happen in practice.

It would be much better to have an API to get the xrun information directly from the process callback.
I don't know if such an API exists.

I guess you would have the exact same question if you were implementing everything in C, right?

I think you should ask at the JACK mailing list: https://jackaudio.org/community.html

The problem might be that there are currently few subscribers because the list was moved and the subscriber list wasn't transferred: https://jackaudio.org/news/2019/12/16/mailing-list-and-rss-feed.html

[dj-foxxy]

Thanks again @mgeier. Yep, I guess I'd have the same question regardless of language. I'll have a go at doing directly with the JACK API and asking the JACK people directly. then I'll report back.

We've been using the code above for a couple of days and we haven't noticed dropped or duplicate MIDI events so maybe it's sufficient, if not optimal.