Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,64 @@ go2rtc:

Replace `CAMERA_SERIAL` with your camera serial number and `192.168.1.10` with the IP address of your go2rtc or Frigate server.

### Repairing packet loss

WHIP media runs over UDP by default, and go2rtc does not negotiate
retransmission. `RegisterDefaultCodecs` in `pkg/webrtc/api.go` registers no RTX
(RFC 4588) codec, so go2rtc strips `rtx/90000` out of its SDP answer while still
advertising `a=rtcp-fb:96 nack`. Harbor cameras offer RTX correctly, but the
answer deletes the channel those retransmissions would travel on — go2rtc asks
the camera to retransmit over something it just removed, so nothing is ever
repaired. This is true as of go2rtc 1.9.10 and current master.

The symptom is occasional blocky or smeared frames that clear on the next
keyframe. Each lost packet damages the frame it belonged to, so even a very low
loss rate is visible — 0.03% loss on a 20 fps stream is a handful of damaged
frames every few minutes.

Running the WHIP session over ICE-TCP sidesteps this. The kernel retransmits
lost segments and delivers them in order, with no SDP negotiation involved:

```yaml
webrtc:
listen: ":8555/tcp"
candidates:
- 192.168.1.10:8555
filters:
networks: [tcp4]
```

Nest that block under `go2rtc:` if you are configuring Frigate.

Both settings are required. `filters` on its own is not enough: with a bare
`listen: ":8555"`, go2rtc registers every entry in `candidates` as **both** a TCP
and a UDP candidate, so it would keep advertising a UDP candidate that the filter
had stopped anything from listening on.

Confirm it took effect by checking the producer's transport, which should read
`http+tcp`:

```bash
curl -s "http://192.168.1.10:1984/api/streams?src=CAMERA_SERIAL" | grep protocol
```

Frigate does not expose port 1984 by default — use
`http://FRIGATE_HOST:5000/api/go2rtc/api/streams?src=CAMERA_SERIAL` instead.

What this costs:

- Loss becomes latency instead of corruption. A sustained wifi dropout shows up
as a stall rather than a glitch.
- `filters` applies to all inbound WebRTC, so browser live view moves to TCP as
well. If you watch streams from outside your network, forward **8555/tcp**, not
only UDP.
- The camera's congestion control stops having an effect, since TCP handles
pacing. That is not a concern at the camera's bitrate on a local network, but
it does mean the camera will not lower quality if the link saturates.

Measured on one Harbor camera pushing 1728x1080 HEVC over wifi, video packet loss
over a 3 minute sample went from 0.039% to 0.000%.

## Development

```bash
Expand Down
4 changes: 4 additions & 0 deletions harbor/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
HeartbeatEvent,
LocalLivekitHeartbeatEvent,
MotionDetectedEvent,
NoiseDetectedEvent,
Settings,
SettingsEvent,
SettingsState,
Expand All @@ -29,6 +30,7 @@
HeartbeatUpdate,
LocalLivekitHeartbeatUpdate,
MotionDetectedUpdate,
NoiseDetectedUpdate,
RawEventUpdate,
SettingsUpdate,
ViewerInfo,
Expand Down Expand Up @@ -72,6 +74,7 @@
"SettingsUpdate",
"CameraEventUpdate",
"MotionDetectedUpdate",
"NoiseDetectedUpdate",
"ViewerInfo",
"parse_message",
"GetCameraSettingsRequest",
Expand All @@ -84,6 +87,7 @@
"ViewerJoinedEvent",
"ViewerLeftEvent",
"MotionDetectedEvent",
"NoiseDetectedEvent",
"HarborSourceType",
"HarborViewer",
"HarborEventState",
Expand Down
44 changes: 42 additions & 2 deletions harbor/data/mqtt_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -157,13 +157,53 @@ class ViewerLeftEvent(HarborMQTTPayload):


class MotionDetectedEvent(HarborMQTTPayload):
"""Payload for a motion detection event."""
"""Payload for a ``motion-detected`` event.

Keys arrive in snake_case. ``duration`` is a unit-suffixed string such as
``"10s"`` -- never parse it as a bare number, and never treat it as a hold
time; see :class:`NoiseDetectedEvent` for why.
"""

active_config: str | None = None
duration: str | None = None
file_duration: str | None = None
filename: str | None = None
level: str | None = None
sensitivity: str | None = None
threshold: str | None = None
thumbnail: str | None = None
timestamp: str | None = None


class NoiseDetectedEvent(HarborMQTTPayload):
"""Payload for a ``sound-anomaly-detected`` event.

The firmware topic calls this a sound anomaly; the app presents it as an
alert for "sudden, loud sounds" and "sustained noises", hence the noise
naming used here.

Field set verified against a live camera. The audio fields are dB strings
(``"-36.401137dB"``) and pair with the ``volume_baseline_current`` /
``volume_baseline_reference`` / ``volume_threshold_effective`` values on
:class:`SettingsState`.

``duration`` is a unit-suffixed string (``"10s"``) that matches
``file_duration``, the length of the recorded clip named by ``filename``.
It describes a detection window that has already closed by the time this
message is published, so it must not be used to decide how long the
detection "stays on" -- the camera never publishes a cleared counterpart at
all.
"""

active_config: str | None = None
duration: str | float | int | None = None
baseline: str | None = None
baseline_reference: str | None = None
duration: str | None = None
file_duration: str | None = None
filename: str | None = None
level: str | None = None
sensitivity: str | None = None
threshold: str | None = None
thumbnail: str | None = None
timestamp: str | None = None
user_offset: str | None = None
54 changes: 17 additions & 37 deletions harbor/devices/camera.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from __future__ import annotations

import asyncio
import logging

from ..config import HarborCameraConfig
Expand All @@ -19,8 +18,10 @@

_LOGGER = logging.getLogger(__name__)

DEFAULT_CAMERA_EVENT_KEYS = ("motion_detection", "cry_detection", "noise_detection")
DEFAULT_EVENT_ACTIVE_SECONDS = 5.0
#: Detection events the camera can report. The firmware publishes exactly two
#: detection topics (``motion-detected`` and ``sound-anomaly-detected``), so
#: seeding anything else would create an entity that can never fire.
DEFAULT_CAMERA_EVENT_KEYS = ("motion_detection", "noise_detection")

# Known values for enumerated state fields. The device reports these in
# mixed/upper case (e.g. "PLAYING", "GOOD"); they are normalized to the
Expand All @@ -47,7 +48,6 @@ def __init__(self, config: HarborCameraConfig) -> None:
"""Initialize the camera device."""
super().__init__(config.serial, "camera")
self.config = config
self._event_reset_handles: dict[str, asyncio.TimerHandle] = {}
self._unexpected_enum_values: set[tuple[str, str]] = set()

for event_key in DEFAULT_CAMERA_EVENT_KEYS:
Expand Down Expand Up @@ -221,49 +221,31 @@ def _apply_viewer_left(self, viewer_id: str | None) -> None:
self.state.values["num_viewers"] = len(self.state.viewers)

def _apply_camera_event(self, event: CameraEventUpdate) -> None:
"""Apply a transient camera event update."""
"""Record a camera detection event.

Detections are edge triggers with no cleared counterpart, so this only
stamps when the event last fired.

Earlier versions synthesized an ``is_on`` flag and held it open for a
duration parsed from the payload. That never worked: the firmware
reports ``"10s"``, which failed to parse and silently fell back to a
fixed 5s, so a 10-second detection was reported as a 5-second one and
every event looked identical. Holding the real value would not have
helped either -- the window has already closed by the time the message
is published.
"""
event_state = self._ensure_camera_event(event.event_key, topic=event.topic)
event_state.topic = event.topic
event_state.last_seen = event.timestamp
event_state.last_payload = event.raw_payload

if handle := self._event_reset_handles.pop(event.event_key, None):
handle.cancel()

if event.explicit_state is False:
event_state.is_on = False
return

event_state.is_on = True
active_seconds = event.active_seconds
if active_seconds <= 0:
active_seconds = DEFAULT_EVENT_ACTIVE_SECONDS

loop = asyncio.get_running_loop()
self._event_reset_handles[event.event_key] = loop.call_later(
active_seconds,
lambda: asyncio.create_task(self._async_reset_event(event.event_key)),
)
_LOGGER.debug(
"Camera %s received event on topic %s: %s",
self.serial,
event.topic,
event.raw_payload,
)

def shutdown(self) -> None:
"""Release camera resources."""
for handle in self._event_reset_handles.values():
handle.cancel()
self._event_reset_handles.clear()

async def _async_reset_event(self, event_key: str) -> None:
"""Reset a transient event to off after its active period."""
self._event_reset_handles.pop(event_key, None)
if event_state := self.state.events.get(event_key):
event_state.is_on = False
await self._emit_update()

def _ensure_camera_event(
self,
event_key: str,
Expand All @@ -287,8 +269,6 @@ def _ensure_camera_event(

def _event_name_from_key(event_key: str) -> str:
"""Return a user-facing event name for an event key."""
if event_key == "cry_detection":
return "Cry detected"
if event_key == "motion_detection":
return "Motion detected"
if event_key == "noise_detection":
Expand Down
Loading
Loading