Implementation of the asyncio consumer#228
Conversation
BewareMyPower
left a comment
There was a problem hiding this comment.
Please remove the binaries from the git commits
Could you also make the tests pass for Python 3.8? Though Python 3.8 will reach EOL on the next month (2024-10) |
|
I will try to fix the issues this week. |
|
I found another issue with the schema system in my asyncio implementation, i will fix this too before the next commit. |
|
Let me fix the broken CI first |
|
Could you rebase to master to resolve the conflicts and have the CI fixed? |
|
@BewareMyPower I removed the binary files |
BewareMyPower
left a comment
There was a problem hiding this comment.
When you wrap an xxxAsync API from pulsar-client-cpp, the correct way is:
// release the GIL because xxxAsync does not access any Python objects
py::gil_scoped_release release;
// Pass the callback directly because pybind11 acquires the GIL automatically when the callback is executed
xxx.xxxAsync(yyy, callback);
BewareMyPower
left a comment
There was a problem hiding this comment.
Please also fix the wrapper in consumer.cc
|
|
||
| void Consumer_unsubscribeAsync(Consumer& consumer, ResultCallback callback) { | ||
| consumer.unsubscribeAsync([callback] (Result result) { | ||
| py::gil_scoped_acquire acquire; |
There was a problem hiding this comment.
This GIL acquire is not necessary
| } | ||
|
|
||
| void Consumer_receiveAsync(Consumer& consumer, ReceiveCallback callback) { | ||
| py::gil_scoped_acquire acquire; |
There was a problem hiding this comment.
It should release the GIL rather than acquire the GIL
|
I will continue the work in #277 |

Hi,
I implemented the asyncio consumer in a similar way as the producer. All missing async functions provided by the C++ library are now implemented in the pybind11 classes and I use futures in python to convert the functions with callbacks to async functions. The reason I started with this implementation is because I need the async consumer in a project with FastAPI myself.
Tests could be a little bit patchy, for that reason i would appreciate help if i missed something