Skip to content

Fix livestream reliability and add two-way audio support - #1303

Open
neoalarrode wants to merge 2 commits into
fronzbot:devfrom
neoalarrode:two-way-audio-support
Open

Fix livestream reliability and add two-way audio support#1303
neoalarrode wants to merge 2 commits into
fronzbot:devfrom
neoalarrode:two-way-audio-support

Conversation

@neoalarrode

Copy link
Copy Markdown

Description:

Two related fixes/additions to BlinkLiveStream, both derived from reverse-engineering the official Android app's native libwalnut.so and cross-checking against real device traffic:

Reliability fixes

  • The auth header sent 64 null bytes instead of the real liveview_token returned by the liveview API response. Sending the real token is required for the two-way audio work below, and also appears to make the relay session more well-behaved in general.
  • recv() used reader.read(n) for the 9-byte header and the payload, which can legitimately return fewer bytes than requested without that being an error condition. Any partial read was being misinterpreted as a corrupt/incomplete packet, causing frequent spurious disconnects. Switched to reader.readexactly(n) with explicit IncompleteReadError handling for the two cases that are genuinely EOF.

Two-way audio (talk) support
Adds the minimal building blocks for sending audio to a camera's speaker over the existing immis connection, mirroring how video already works (blinkpy moves bytes, it doesn't capture/encode audio):

  • request_audio() / stop_audio(): send the SESSION_COMMAND that arms/disarms two-way audio on the camera.
  • audio_format_received (an asyncio.Event) and audio_format / audio_format_flags: the camera announces the AAC-LC format it expects (sample rate, channel count) via a previously-unhandled AUDIO_CONFIG message type, which turned out to be a fixed 9-byte self-contained message rather than following the generic [type][sequence][length] framing every other message type uses.
  • send_audio(payload): sends one ADTS-framed AAC-LC audio frame. Callers are responsible for encoding, exactly as they already are for anything downstream of the raw video bytes.

One correctness note baked into the implementation: outgoing audio frames use a sequence-number range disjoint from the keep-alive counter (AUDIO_SEQUENCE_START). The relay server appears to track sequence numbers per-session rather than per-message-type, and reusing small overlapping sequence numbers between the keep-alive and audio channels caused it to reset the connection after only a handful of audio frames.

Also factored the repeated 9-byte header construction (previously duplicated for the keep-alive and latency-stats packets) into a small build_packet() helper, now shared by every outgoing message type.

Related issue (if applicable):

None - found while building two-way audio support on top of blinkpy for a personal integration.

Checklist:

  • Local tests with ruff check / ruff format --check / pytest run successfully (didn't have tox set up locally, but ran the equivalent lint + test steps directly)
  • Changes tested locally against a real Blink doorbell to ensure platform still works as intended (video playback, and two-way audio frames sent without the connection dropping)
  • Tests added to verify new code works - tests/test_livestream.py has 100% coverage on the changed file; also fixed several existing tests that were mocking reader.read instead of reader.readexactly and were silently passing for the wrong reason after the reliability fix above

neoalarrode and others added 2 commits September 8, 2026 15:27
- Use the real liveview_token in the auth header instead of null bytes,
  and use readexactly() instead of read() for header/payload reads in
  recv(), fixing frequent spurious disconnects caused by partial reads
  being misread as corrupt packets.
- Add two-way audio (talk) support to BlinkLiveStream: request_audio(),
  stop_audio() and send_audio() send SESSION_COMMAND/AUDIO messages, and
  audio_format_received/audio_format expose the AAC-LC format the camera
  announces via the AUDIO_CONFIG message. Outgoing audio frames use a
  sequence range disjoint from the keep-alive counter, since the server
  tracks sequence numbers per-session rather than per-msgtype and resets
  the connection when they collide.
- Update tests/test_livestream.py: mock readexactly() instead of read()
  wherever target_reader is simulated (several tests were silently
  passing for the wrong reason after the read()->readexactly() change),
  and add coverage for the AUDIO_CONFIG message and the new audio
  methods. 100% coverage on the changed file.
The camera's video payloads don't reliably align to 188-byte TS packet
boundaries. Dropping any payload that didn't start with 0x47 discarded
otherwise-valid packets and desynced downstream demuxers for the rest
of the session. Buffer across payloads and resync on the sync byte
instead, confirming it recurs one packet length later before trusting it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant