extract Execution and close the job lifecycle (#2360, goal 2)

[ebuzerdrmz44]

Description

Follows #2558, which split out Scheduling and State. This takes out the third piece: create_backup, notify and post_backup_tasks move into scheduler/execution.py as SchedulerExecution, same collaborator shape as the other two.

It also makes the scheduler record runs that actually happen. Until now only record_skip wrote JobModel rows, so the table held runs that didn't run and the Jobs page never showed a completed backup. record_start and record_finish sit next to it in State. A run is written as running, then settled as completed or failed with its EventLogModel row linked, which is the intent/result link from goal 1 of the issue. Failed runs store the worst message Borg logged as the reason, so the Reason column is filled the way it already is for skips.

Screenshot

An example of 'running' row

Submitting a run also drops the timer entry that fired it, otherwise the page listed the same run twice while it was in flight, once as a stale scheduled row and once as running. notify re-arms the timer as before.

recover_interrupted_jobs marks rows left at running as interrupted. I put it in init_db next to the existing JobModel purge rather than in the scheduler: it has to run once per process, and hanging it off VortaScheduler.__init__ meant a second scheduler could flip a live run. That answers open question 3 in the issue. Let me know if you would rather it retried those runs instead of just recording them.

Manual backups still bypass the scheduler, so they stay off the table. (not sure if they supposed to be in this tab , let me know if you want it )

Jobs page filtering is the last open bullet of goal 3 and is coming as its own PR next.

[ebuzerdrmz44]

Hey @m3nu, I do not feel strong for the type column , but would be there any new job type rather than backup in the future ? It also runs prune , check and compact after a backup but these only go to event log .