Fix livestream reliability and add two-way audio support - #1303
Open
neoalarrode wants to merge 2 commits into
Open
Fix livestream reliability and add two-way audio support#1303neoalarrode wants to merge 2 commits into
neoalarrode wants to merge 2 commits into
Conversation
- Use the real liveview_token in the auth header instead of null bytes, and use readexactly() instead of read() for header/payload reads in recv(), fixing frequent spurious disconnects caused by partial reads being misread as corrupt packets. - Add two-way audio (talk) support to BlinkLiveStream: request_audio(), stop_audio() and send_audio() send SESSION_COMMAND/AUDIO messages, and audio_format_received/audio_format expose the AAC-LC format the camera announces via the AUDIO_CONFIG message. Outgoing audio frames use a sequence range disjoint from the keep-alive counter, since the server tracks sequence numbers per-session rather than per-msgtype and resets the connection when they collide. - Update tests/test_livestream.py: mock readexactly() instead of read() wherever target_reader is simulated (several tests were silently passing for the wrong reason after the read()->readexactly() change), and add coverage for the AUDIO_CONFIG message and the new audio methods. 100% coverage on the changed file.
The camera's video payloads don't reliably align to 188-byte TS packet boundaries. Dropping any payload that didn't start with 0x47 discarded otherwise-valid packets and desynced downstream demuxers for the rest of the session. Buffer across payloads and resync on the sync byte instead, confirming it recurs one packet length later before trusting it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
Two related fixes/additions to
BlinkLiveStream, both derived from reverse-engineering the official Android app's nativelibwalnut.soand cross-checking against real device traffic:Reliability fixes
liveview_tokenreturned by the liveview API response. Sending the real token is required for the two-way audio work below, and also appears to make the relay session more well-behaved in general.recv()usedreader.read(n)for the 9-byte header and the payload, which can legitimately return fewer bytes than requested without that being an error condition. Any partial read was being misinterpreted as a corrupt/incomplete packet, causing frequent spurious disconnects. Switched toreader.readexactly(n)with explicitIncompleteReadErrorhandling for the two cases that are genuinely EOF.Two-way audio (talk) support
Adds the minimal building blocks for sending audio to a camera's speaker over the existing
immisconnection, mirroring how video already works (blinkpy moves bytes, it doesn't capture/encode audio):request_audio()/stop_audio(): send theSESSION_COMMANDthat arms/disarms two-way audio on the camera.audio_format_received(anasyncio.Event) andaudio_format/audio_format_flags: the camera announces the AAC-LC format it expects (sample rate, channel count) via a previously-unhandledAUDIO_CONFIGmessage type, which turned out to be a fixed 9-byte self-contained message rather than following the generic[type][sequence][length]framing every other message type uses.send_audio(payload): sends one ADTS-framed AAC-LC audio frame. Callers are responsible for encoding, exactly as they already are for anything downstream of the raw video bytes.One correctness note baked into the implementation: outgoing audio frames use a sequence-number range disjoint from the keep-alive counter (
AUDIO_SEQUENCE_START). The relay server appears to track sequence numbers per-session rather than per-message-type, and reusing small overlapping sequence numbers between the keep-alive and audio channels caused it to reset the connection after only a handful of audio frames.Also factored the repeated 9-byte header construction (previously duplicated for the keep-alive and latency-stats packets) into a small
build_packet()helper, now shared by every outgoing message type.Related issue (if applicable):
None - found while building two-way audio support on top of blinkpy for a personal integration.
Checklist:
ruff check/ruff format --check/pytestrun successfully (didn't havetoxset up locally, but ran the equivalent lint + test steps directly)tests/test_livestream.pyhas 100% coverage on the changed file; also fixed several existing tests that were mockingreader.readinstead ofreader.readexactlyand were silently passing for the wrong reason after the reliability fix above