feat: Add camera livestream and voice call support - #764
Conversation
Adds a standalone camera module for streaming video/audio from Roborock vacuums with cameras. This addresses part of Python-roborock#738. Features: - Live video streaming (HD/SD quality) - Bidirectional voice calls (speak to and hear from robot) - Snapshot capture - Video recording - Audio sending for intercom functionality The implementation uses: - MQTT for WebRTC signaling (via existing protocol encoders) - aiortc for WebRTC peer connection - paho-mqtt for MQTT connection (standalone, not using aiomqtt) This is provided as a standalone module that could be integrated more tightly with the library's session management in future work. See docs/CAMERA_PROTOCOL.md for full protocol documentation. New dependencies (optional): - aiortc: WebRTC implementation - paho-mqtt: MQTT client - pyaudio: Voice call audio (optional) - opencv-python: Video display (optional)
|
Impressive result and use of AI. I know you said this is a standalone module, but not sure we should necessarily add a standalone module that doesn't really fit into the code base structure? Happy to discuss with you on discord if you do intend to make this fit in with the project. |
Hey! I'm the bot owner. clawd139 is my OpenClaw instance. This was a couple hrs of back-and-forth but it did 99% of the work. I just helped with grabbing mitm packets. I know this wasn't built with python-roborock compatibility in mind but hoping the protocol documentation helps :) Edit: the decompiled apk was also used to help debug |
|
Yes, the spec is helpful. The interesting next part is getting the API right. Likely what needs to happen to make this usable is to define a python roborock trait that implements an interface similar to the WebRTC interface in home assistant: https://developers.home-assistant.io/docs/core/entity/camera/#webrtc-streams - Namely, the signaling path such that given an offer it returns an answer, and a way to forward on ice candidates. From your last reply though, it sounds like the intent here is not necessarily to get this shipped? |
|
I'm happy to tell your openclaw instance what to do if it'll listen haha |
My intent was to share progress not necessarily to ship, but sure I'll paste in your prompt :P |
|
👍🏼 have it also review the docs in |
Summary
This PR adds a standalone camera module for streaming video/audio from Roborock vacuums with cameras. Addresses part of #738 (Camera Livestream, Remote Control).
Features
Implementation
The module uses:
roborock.protocolencodersThis is provided as a standalone module (
roborock/camera.py) that manages its own MQTT connection. Future work could integrate it more tightly with the library's session management and aiomqtt usage.Protocol Documentation
Full protocol documentation is included in
docs/CAMERA_PROTOCOL.md, covering:Dependencies
New optional dependencies:
aiortc- WebRTC implementationpaho-mqtt- MQTT clientpyaudio- Voice call audio playback (optional)opencv-python- Video display (optional)Usage Example
Tested On
Other camera-equipped models likely work but are untested.
Notes
start_voice_chatcommand must be called to receive audio framesHappy to iterate on this if you'd prefer different structuring or tighter integration with the existing codebase!
Closes part of #738