Skip to content

fix(orchestrator): accept job events only from the job's own workers - #76

Merged
mjunek merged 2 commits into
masterfrom
fix/scope-job-events
Nov 15, 2026
Merged

mjunek merged 2 commits into
masterfrom
fix/scope-job-events

Conversation

@mjunek

@mjunek mjunek commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Stacked on #75 (targets its branch): both change the job event handler. Retarget to master once #75 merges.

What and why

A job event was accepted from any approved worker. The signature proved who sent it, not that the job was theirs, so a collector of another group, a storage worker of another target or a receiver could:

  • mark a device's active job stored or failed;
  • set its storage_ref;
  • append to any job's log, finished or not.

Events are now accepted only from the job's own workers (backups.jobs._not_its_worker):

  • Storing step (stored, or any event with stage store): a storage worker of the job's target.
  • Everything else: a collector of the device's collection group.
  • Pushed backups: no worker reports on their collection, since the orchestrator creates them already collected. Their storing step is still the storage worker's.

A refused event changes nothing on the job and is logged as job_event_refused with the worker and reason. Deletion events already had their own check and are unchanged.

Behaviour to be aware of:

  • Membership is read when the event arrives, not when the job was dispatched. Taking a collector off a group, moving a device to another collection group, or taking a storage worker off a target while a job runs means that job's remaining events are refused and it times out.
  • Refusals are logged, not audited.

Host key pinning (#75) no longer checks the collector itself, since the collected event is scoped before it gets there.

How it was tested

  • make check passes.
  • New tests cover outsiders (another group's collector, another target's storage worker, a receiver) across every event type, a job's own workers reporting the wrong step, a finished job's log, a worker removed mid-job, and pushed backups.

Checklist

  • make check passes locally
  • Tests added or updated for the change
  • New DV_* variables are documented in docs/DEPLOYMENT.md and the relevant *.env.example (none added)
  • Model changes have generated migrations (makemigrations) (no model changes)
  • API changes keep api/serializers.py, api/views.py and frontend/src/api/types.ts / endpoints.ts in step (no API changes)
  • Docs (README.md, docs/) updated where behaviour changed

A job event was accepted from any approved worker, so a signature
proved who sent it but not that the job was theirs. A collector of
another group, a storage worker of another target or a receiver could
mark a device's active job stored or failed, set its storage reference
and append to any job's log.

The storing step is now accepted only from a storage worker of the
job's target, and everything else only from a collector of the device's
collection group. Events about a pushed backup's collection are refused,
since no worker collects it. A refused event changes nothing and is
logged as job_event_refused.

Membership is read when the event arrives, so a worker taken off a
group or target mid-job loses its say in that job.

Host key pinning no longer checks the collector itself: the collected
event is scoped before it gets there.
@mjunek
mjunek requested a review from a team as a code owner November 15, 2026 04:25
Base automatically changed from fix/host-keys-client-address to master November 15, 2026 04:31
@mjunek
mjunek merged commit c48909e into master Nov 15, 2026
1 check passed
@mjunek
mjunek deleted the fix/scope-job-events branch November 15, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant