R5 systemd as PID 1: cgroup v2 controllers, namespaces and pidfds

Part of #14.

cgroup2 has the cpu, cpuset, io, memory and pids controllers, cgroup.freeze, cgroup.kill, PSI, the thread mode, nsdelegate and CLONE_INTO_CGROUP (core/rk-cgroup, fs/rk-fs-cgroup). rukernel has the mount, cgroup, network, UTS and IPC namespaces, but CLONE_NEWPID, CLONE_NEWUSER and CLONE_NEWTIME give EINVAL (core/rk-syscall/src/process.rs). pidfd_open, pidfd_send_signal, pidfd_getfd, CLONE_PIDFD and waitid(P_PIDFD) work, but there is no pidfs, no PIDFD_GET_INFO and no SO_PEERPIDFD. The debian boot test runs a shell script as init. No boot test runs systemd.

LTP now: pidfd_* 8 pass, unshare 3 pass, clone3 2 pass. ioctl_ns01, ioctl_ns06 and setns02 fail, and clone301, clone304, unshare05 and ioctl_pidfd02 to ioctl_pidfd06 do not apply, because there is no PID or user namespace.

Scope

pidfs: give each pidfd an inode of a pidfs mount, with one inode number for each process that stays the same while the process lives. fstat, name_to_handle_at and open_by_handle_at on a pidfd work, and /proc/PID/fdinfo of a pidfd has the Pid: and NSpid: lines. Test: add pidfd to tools/kselftest/build.sh and run pidfd_fdinfo_test and pidfd_file_handle_test. Exit: both pass, and LTP pidfd_* stays at 8 pass. Namespace owners for the setns checks, and pidfs handles in the PID namespace #612: pidfd_fdinfo_test (2 of 2) and pidfd_file_handle_test (11 of 11) pass and run in the kselftest boot test. LTP pidfd_open01 to 04, pidfd_send_signal01 to 03 and pidfd_getfd01 and 02 pass (9 of 9).
The PIDFD_GET_INFO and PIDFD_GET_*_NAMESPACE ioctls, a real PIDFD_THREAD (the pidfd refers to the thread, not to its process) and process_mrelease. Test: kselftest pidfd_info_test, pidfd_test and pidfd_setns_test, LTP ioctl_pidfd01 and process_mrelease01. Exit: these pass, apart from the cases that need a PID or user namespace. pidfd: add PIDFD_GET_INFO and the namespace ioctls #557, pidfd: PIDFD_GET_INFO gives EREMOTE for a process out of the PID namespace #608, pidfd: PIDFD_THREAD of a thread, POLLHUP after a reap, the signal scopes, and process_mrelease #620, execve of a thread takes the ID of the leader, as de_thread #655, Owners of the mount and cgroup namespaces, setns of a mount namespace with a pidfd #658: kselftest pidfd_info_test (8 of 8) and pidfd_setns_test (8 of 8) pass in the kselftest boot test. LTP ioctl_pidfd01 to 06 pass. pidfd_test is not in tools/kselftest/build.sh, because its test 8 takes about an hour on QEMU TCG. LTP process_mrelease01 is not in the LTP tree of the test host (20260930), and the init check of process_mrelease passes. The init check -- only pidfd passes on x86_64 and arm64.
SO_PEERPIDFD and SCM_PIDFD on AF_UNIX sockets, as systemd and dbus-broker use them. Test: kselftest net/af_unix/scm_pidfd, and the gvisor socket_unix tests. Exit: scm_pidfd passes, and the gvisor results do not change. net: add SO_PEERPIDFD and SCM_PIDFD to Unix sockets #560: scm_pidfd passes 4 of 4, and the gVisor socket_unix tests still pass (20 of 20 binaries).
PID namespaces: a PID number in each namespace, the init of a namespace, the kill of the namespace when its init exits, /proc that shows the PIDs of the namespace of the mount, /proc/sys/kernel/ns_last_pid and clone3 with set_tid. Test: LTP clone301, unshare05, ioctl_ns01, ioctl_ns05, pidfd_send_signal03 and the pidns* tests of runtest/containers. Exit: these reach the pass set of the reference. pid namespaces: CLONE_NEWPID, setns, and the IDs of each namespace #581, procfs: show the PID namespace of the mount #584, clone3: set_tid for each level of PID namespace #588: on main 2fbcee3, LTP clone301, unshare05, ioctl_ns01, ioctl_ns05, pidfd_send_signal03 and the pidns* tests of runtest/containers pass (x86-64, release kernel, server2 load 11 to 21). pidns05 scans all PIDs to 4194304 and needs --timeout 1800 on TCG; with the default time limit it is TBROK after all its checks pass. The init check -- only pidns passes on arm64 (606 s, load 13.6).
User namespaces: uid_map, gid_map and setgroups with the write rules of Linux, capabilities that apply only in the owner namespace, setns with CLONE_NEWUSER, NS_GET_USERNS, NS_GET_PARENT and NS_GET_OWNER_UID, and MOUNT_ATTR_IDMAP. Test: LTP userns01 to userns08, ioctl_ns06, setns02, clone304 and ioctl_pidfd02 to ioctl_pidfd06. Exit: these reach the pass set of the reference. pidfd: PIDFD_GET_INFO gives EREMOTE for a process out of the PID namespace #608: on main 2fbcee3, LTP userns01 to userns08, ioctl_ns06, setns02, clone304 and ioctl_pidfd02 to ioctl_pidfd06 pass, 17 of 17 (x86-64, release kernel, server2 load 16). MOUNT_ATTR_IDMAP is not done: mount_setattr refuses it with EBADF or EINVAL, so the item still needs idmapped mounts.
Time namespaces: offsets for CLOCK_MONOTONIC and CLOCK_BOOTTIME, /proc/PID/timens_offsets, and a vDSO data page for each time namespace. Test: LTP timens01, clock_gettime03, clock_nanosleep03, timerfd04 and sysinfo03, and kselftest timens. Exit: these pass. proc: add time namespaces #565: LTP timens01, clock_gettime03, clock_nanosleep03, timerfd04 and sysinfo03 and kselftest timens pass.
cgroup2: the hugetlb, rdma and misc controllers, the mount option memory_recursiveprot, and the user.* and trusted.* xattrs on cgroup directories, which systemd and systemd-oomd write. Test: run LTP runtest/controllers with tools/ltp/run.py and add tools/ltp/controllers.md. Add cgroup to tools/kselftest/build.sh and run test_core, test_memcontrol, test_freezer, test_kill, test_cpu and test_cpuset. Exit: cgroup.controllers of the root lists the same controllers as the reference, and the tests reach the pass set of the reference.
A systemd boot test: the Debian 13 image with the systemd and systemd-sysv packages, with systemd as PID 1, to multi-user.target. Exit: systemctl is-system-running gives running, or gives degraded with only the units that wait for the other R5 issues in the failed list. The test lists these units. xtask: add a systemd boot test #572, xtask: run the systemd test with -icount without KVM #619, xtask: check IP accounting and DevicePolicy=closed in the systemd boot test #632, xtask: check systemd-detect-virt in the systemd boot test #640: on main 2fbcee3, cargo xtask boot-test systemd (debug kernel, -icount) gives rk-systemd: state running and rk-systemd: pass, and QEMU powers off, in 233 s on server1 (load 0.1).
Run the systemd integration tests that need only the parts of this issue: TEST-07-PID1, TEST-19-CGROUP and TEST-55-OOMD. Add the harness to tools and a report with the results of rukernel and of the reference. Exit: the three tests reach the pass set of the reference, apart from the listed exceptions.

Done when

systemd boots as PID 1 and starts its default units. This is the exit of the first item of #14.

Test each task on x86_64 and on arm64. Give the result of both in the PR body.

Linux reference: kernel/cgroup/cgroup.c, kernel/nsproxy.c, kernel/pid_namespace.c, kernel/user_namespace.c, kernel/time/namespace.c, fs/pidfs.c, kernel/pid.c.

#552 (not an item of this issue): the arm64 boot and smp boot tests now run with 8 CPUs. 3 runs of the arm64 smp test on server1 pass in 61, 63 and 73 s at a load of 0.2 to 4.7.

#555 merged: pidfs. Each pidfd of a process has one inode in the internal mount of pidfs, with PID_FS_MAGIC. name_to_handle_at gives a FILEID_KERNFS handle and open_by_handle_at opens a new pidfd from it. /proc/PID/fdinfo of a pidfd has Pid: and NSpid:. The init and smp boot tests pass on x86_64 and arm64 (server1, load 0.4 to 4.6), with 15 new checks.

The item stays open. pidfd_fdinfo_test and pidfd_file_handle_test need CLONE_NEWUSER and CLONE_NEWPID in their fixtures, so pidfd goes into tools/kselftest/build.sh with the PID and user namespaces.

#557 merged: the pidfd ioctls. PIDFD_GET_INFO gives the IDs, the credentials, the cgroup ID, the core dump bits, and the exit status after the reap. PIDFD_GET_*_NAMESPACE open the namespace files of the process, and setns takes them. The init and smp boot tests pass on x86_64 and arm64 (server1, load 0.03 to 3.8), with 11 new checks.

The item stays open: a real PIDFD_THREAD and process_mrelease are not done, and pidfd_info_test, pidfd_test and pidfd_setns_test need user and PID namespaces. PIDFD_GET_USER_NAMESPACE gives EOPNOTSUPP until rukernel has user namespaces.

Item 3 is done with #560. kselftest net/af_unix/scm_pidfd passes 4 of 4. The gVisor Unix socket tests (20 binaries, socket_unix_* and the related socket_*_local tests) all pass on x86_64 with QEMU TCG, with a kernel that has #560 (server1, load average 0.63 at start, 0.94 at end), so the results do not change.

#565 merged: time namespaces. unshare and clone3 with CLONE_NEWTIME, /proc/PID/timens_offsets, setns with /proc/PID/ns/time or a pidfd, and a vvar page with the offsets for the vDSO. On x86_64 with QEMU TCG (server1), LTP timens01 (8 of 8), clock_gettime03 (24 of 24), clock_nanosleep03 (2 of 2) and sysinfo03 (6 of 6) pass, and kselftest timens passes 8 with 2 skips. timerfd04 passed 3 of 4 in one run (the first case slept 747 ms, more than the limit of 250 ms), and passed in three later runs. The sysinfo uptime now rounds up as on Linux. init and smp pass on x86_64 and arm64. I ticked the item.

#567 merged: two fixes that the systemd boot found. mount with MS_BIND | MS_REC now copies the mounts under the source (copy_tree), and /proc/PID/mountinfo works with epoll and has an event when the mounts change. In the Debian 13 systemd boot on x86_64, systemd-logind no longer fails at the step NAMESPACE, "Failed to drain libmount events" is gone, and tmp.mount, dev-hugepages.mount, dev-mqueue.mount and run-lock.mount no longer fail. This is part of the systemd boot test item, which stays open.

#572 merged: cargo xtask boot-test systemd (Debian 13, systemd as PID 1, archive from tools/systemd/build.sh, skipped without RK_SYSTEMD) and one /dev/kmsg record for each writev. The item stays open, because the test does not pass yet:

systemd-logind fails again and again with "Failed to create udev watchers: Protocol not available", so the system stays in starting. The probable cause is SO_ATTACH_FILTER: the udev monitor of systemd puts a classic BPF filter on its NETLINK_KOBJECT_UEVENT socket, and rukernel gives ENOPROTOOPT for that option on all sockets (net/rk-net, the area of R5 netlink: rules, nexthops, generic netlink and ethtool for networkd, resolved and udevd #538 and R5 uevents, the keyring, fanotify and inotify #539).
systemd-modules-load fails with "Failed to initialize libkmod context: Operation not supported". This is not analysed yet.

All the mount units start now (after #567). When these two are fixed, fill allowed in the test and check the state again.

#575 adds SO_ATTACH_FILTER, SO_DETACH_FILTER and SO_LOCK_FILTER to netlink sockets. This was the probable cause of the logind failure of #572 ("Failed to create udev watchers: Protocol not available"): the udev monitor of systemd attaches a classic BPF filter to its NETLINK_KOBJECT_UEVENT socket.

I could not confirm the logind fix with the systemd boot test. On server2, with the Debian 13 archive of tools/systemd/build.sh, systemd now stops before any unit starts:

systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error
[!!!!!!] Failed to start up manager.
systemd[1]: Freezing execution.

The kernel of main (a60e447, without #575) stops in the same place with the same archive, so #575 does not cause it. The run of #572 on server1 got past the generators. In systemd, "Protocol error" (EPROTO) here means that the child of safe_fork for the generators (new mount namespace, MS_SLAVE propagation) exited with an error. The cause is not traced yet. It is probably the mount namespace setup, or something that changed after #572.

#578 adds the kmod package to the systemd boot test image. systemd 257 loads libkmod with dlopen, and without it systemd-modules-load failed with "Failed to initialize libkmod context: Operation not supported". Now systemd-modules-load.service finds no modules and ends with status 0/SUCCESS, as on Linux without loadable modules (systemd boot on server1, QEMU TCG). The init boot test now checks that the module calls give ENOSYS and that /proc/modules does not exist.

About "Failed to fork off sandboxing environment for executing generators: Protocol error": I saw it in one boot on server2 at a load average of 30 to 46, with the kernel of main a60e447 and the image with kmod. Two boots on server1 (load 0.05) with the same kernel and image did not show it. In those boots (sd-gens) succeeded and the boot went on to the systemd-logind failure. Thus the error is not a fixed result of a change on main. It occurs on a busy host, and its cause is not known yet.

#581 adds the first part of item 12: PID namespaces. clone, clone3 and unshare take CLONE_NEWPID, and setns takes /proc/PID/ns/pid or a pidfd. A thread has an ID at each level, the system calls use the IDs of the caller namespace, the init of a namespace reaps its orphans and is protected from signals of its namespace, and its exit ends the namespace. /proc/PID/ns/pid_for_children, NS_GET_PARENT, the NS_GET_*_PIDNS ioctls and NSpid in the pidfd fdinfo work. On x86_64, LTP pidns01, 02, 04, 06, 10, 12, 13, 16, 17, 20, 30, 31, 32, clone301, ioctl_ns01, ioctl_ns05, pidfd_send_signal03 and unshare05 pass, and the init check passes on x86_64 and arm64. Not done yet: procfs for each namespace (pidns03 fails), NSpid and NStgid in /proc/PID/status, ns_last_pid for each namespace, set_tid of clone3, and pidns05 is too slow under TCG. The item stays open.

#663: FSCONFIG_CMD_RECONFIGURE on the context of fsopen after fsmount changes the file system, as in Linux, and a file open for a write on a detached mount stops the change to read only. With it, the credentials units of systemd 261 (Ubuntu 26.10) and 262 (Arch) no longer fail with 243/CREDENTIALS. Tested with the other fixes of this work: x86_64 init passes (load 6 to 10), arm64 init has these checks ok, and the Ubuntu 26.10 tree gets to multi-user.target. The item of the L4 traces stays open.

#665: the console and the pseudo terminals take TCGETS2, TCSETS2, TCSETSW2 and TCSETSF2 with struct termios2. glibc 2.42 and later use TCGETS2 in isatty and tcgetattr, so rk-check.service of the systemd boot test no longer fails with 209/STDOUT (ENOTTY on /dev/console) on Ubuntu 26.10 and Arch. Tested with the other fixes of this work: x86_64 init and arm64 init pass, and the Ubuntu 26.10 tree gets to multi-user.target. The item of the L4 traces stays open.

#667: ioprio_set and ioprio_get, as block/ioprio.c of Linux, with the checks of the class, the level and the permissions, and the inheritance to a new child. The units of systemd with IOSchedulingClass= no longer fail with 211/IOPRIO on Ubuntu 26.10. Tested with the other fixes of this work: x86_64 init and arm64 init pass, and the Ubuntu 26.10 tree gets to multi-user.target. The item of the L4 traces stays open.

#670: each file of /proc/sys can go in an epoll set, as proc_sys_poll of Linux, and a change of the host name or the NIS domain name gives POLLPRI and POLLERR on its file. systemd-journald of systemd 261 (Ubuntu 26.10) and 262 (Arch) no longer stops with "Failed to register hostname fd in event loop". The item of the L4 traces stays open.

#671: fcntl record locks, lockf and flock work on a socket. systemd locks the storage socket of PrivateNetwork= with lockf, so e2scrub_reap.service of Ubuntu 26.10 no longer fails with 225/NETWORK. With this and the earlier fixes, the systemd boot test with the Ubuntu 26.10 tree (systemd 261) passes on x86_64: rk-systemd: state running and rk-systemd: pass. The item of the L4 traces stays open.

#675: the systemd boot test and the tests of the systemd tree boot with systemd.firstboot=no, so systemd-firstboot.service does not ask for the time zone on the console. With it and the kernel fixes #663, #665, #667, #670, #671 and #538, the systemd boot test passes with the Arch tree (systemd 262) on x86_64: rk-systemd: state running and rk-systemd: pass, in 466 s. The Ubuntu 26.10 tree (systemd 261) also passes (#671). The item of the L4 traces stays open.

cgroup2: #676 is merged. test_cgcore_destroy passes now. The kernel runs with interrupts off, so the timer ticks did not come in a long read, and a task that slept in usleep did not wake. should_yield now lets in the waiting interrupts and wakes the expired timers, and a long read yields between its steps and ends at a fatal signal. x86_64 test_core: 10 pass, 1 fail (test_cgcore_thread_migration, next PR), 1 skip, in 23 s. The -- only cgroup init test passes on x86_64 and arm64.

cgroup2: #681 is merged. /proc/PID/task/TID/cgroup shows the cgroup of the thread, so test_cgcore_thread_migration passes. The kselftest boot test now builds and runs test_core and test_cpuset, and mounts cgroup2 at /sys/fs/cgroup with nsdelegate. x86_64 test_core: 12 pass. test_cpuset: 3 pass. The -- only cgroup init test passes on x86_64 and arm64, except the check cpu.max limits a busy loop in some boots, which is sensitive to the load of the host (#504) at a load average of 10 to 18.

cgroup2: #682 is merged. A stopped, traced or vfork thread counts as frozen in its cgroup, and cgroup.events sends inotify IN_MODIFY. The kselftest boot test now builds and runs test_freezer and test_kill too. On x86_64 TCG, test_core, test_cpuset, test_freezer and test_kill pass (666 s for the boot test at a load average of 14). The -- only cgroup init test passes on x86_64. On arm64 only the load sensitive check cpu.max limits a busy loop (#504) failed, at a load average of 30 to 45.

cgroup2: #685 is merged. A write of memory.peak now resets the peak of the open file only, as peak_write of Linux, so the checks of test_memcg_current_peak on the open files of memory.peak pass. The test still fails later, because a read of a hole of a file does not put a page in the page cache. The x86_64 and arm64 -- only cgroup init tests pass (load average 16 to 20 on x86_64, 26 on arm64).

cgroup2: #687 is merged. A long populate (mlock, MAP_POPULATE, MAP_LOCKED) now waits for memory.high after each batch of 64 pages, as try_charge_memcg of Linux, and test_memcg_high_sync passes. The x86_64 and arm64 -- only cgroup init tests pass.

cgroup2: #688 is merged. tools/kselftest/build.sh builds the cgroup selftests without optimization, as their Makefile in Linux, so alloc_anon of test_memcontrol uses its memory again. test_memcontrol and test_cpu are not in TESTS yet, because some of their tests still fail (page cache of reads of holes, sock accounting, cpu.stat against the CPU clock of the process, the weights of nested cgroups, and cpu.max on QEMU TCG).

cgroup2: #691 is merged. A kill of a memory.oom.group now counts the victim again in oom_kill, as oom_kill_memcg_member of Linux, and test_memcg_oom_group_score_events passes. The x86_64 and arm64 -- only cgroup init tests pass.

cgroup2: LTP runtest/controllers ran once with tools/ltp/run.py on server2 (x86_64, QEMU TCG without KVM, release kernel of the cgroup2 branches up to cg-8, LTP_TIMEOUT_MUL=4). Load average 23.8 at the start and 10.5 at the end. The run took about 4 hours.

Of the 347 tests: 96 pass, 206 do not apply (TCONF), 40 time out, and 5 fail.

Pass: cgroup_core01 to cgroup_core03, memcontrol01, memcontrol02, memcg_regression, memcg_control, io_control01, cgroup_fj_function for cpu, cpuset, memory and hugetlb, 47 of the cgroup_fj_stress tests, and 37 of the 39 pids tests.
Do not apply: the tests of the cgroup v1 controllers that cgroup2 does not have (memcg_* v1 files, cpuset_*, cpuacct_*, cgroup_xattr, and the cgroup_fj_* tests of debug, cpuacct, devices, freezer, blkio, net_cls, net_prio and perf_event).
Time out: memcg_stress, memcontrol03, memcontrol04, and 37 cgroup_fj_stress tests of cpuset, cpu, memory and hugetlb with many cgroups (2_9, 10_3, 4_4, 200_1 and 1_200). These tests are long on QEMU without KVM.
Fail: cgroup (cgroup_regression_test.sh mounts the v1 cgroup file system, and rukernel has only cgroup2), memcg_test_3 (the test timer killed it), controllers (test_controllers.sh is not in the LTP build), and pids_9_2 and pids_9_10 (TWARN: the cleanup rmdir of a test cgroup gave EBUSY right after its processes ended).

tools/ltp/controllers.md is not made yet, because report.py knows only the groups syscalls, mm, sched, net and pty.

cgroup2: #692 is merged. A cgroup that ran past its cpu.max quota now stays throttled until the next periods pay for the difference, as distribute_cfs_runtime of Linux. cpu.max limits a busy loop of the init test passed in the first try at a load average of 16 to 20. test_cpucg_max and test_cpucg_max_nested still fail on QEMU TCG, because each wake costs about 24 ms against a quota of 1 ms, and the main thread of the test does not run first after a throttle.

Navigation Menu

Repository navigation

R5 systemd as PID 1: cgroup v2 controllers, namespaces and pidfds #536

Scope

Done when

15 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

R5 systemd as PID 1: cgroup v2 controllers, namespaces and pidfds #536

Description

Scope

Done when

Activity

tamnd commented on Nov 2, 2026

tamnd commented on Nov 3, 2026

tamnd commented on Nov 3, 2026

tamnd commented on Nov 3, 2026

tamnd commented on Nov 3, 2026

tamnd commented on Nov 3, 2026

tamnd commented on Nov 4, 2026

tamnd commented on Nov 4, 2026

tamnd commented on Nov 4, 2026

tamnd commented on Nov 4, 2026

15 remaining items

tamnd commented on Nov 8, 2026

tamnd commented on Nov 8, 2026

tamnd commented on Nov 8, 2026

tamnd commented on Nov 9, 2026

tamnd commented on Nov 9, 2026

tamnd commented on Nov 9, 2026

tamnd commented on Nov 9, 2026

tamnd commented on Nov 9, 2026

tamnd commented on Nov 10, 2026

tamnd commented on Nov 10, 2026

tamnd commented on Nov 10, 2026

tamnd commented on Nov 11, 2026

tamnd commented on Nov 11, 2026

tamnd commented on Nov 11, 2026

tamnd commented on Nov 11, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions