Improve tests for execution in sandboxed environments - #4
Merged
Conversation
haxtibal
force-pushed
the
tdmg/test_improvements
branch
from
September 7, 2026 08:15
4a03cf5 to
d5082db
Compare
koalo
approved these changes
Sep 7, 2026
| ) | ||
| monkeypatch.setenv("SOFTHSM2_CONF", str(conf_file)) | ||
|
|
||
| openssl_conf = tmp_path / "openssl.cnf" |
Collaborator
There was a problem hiding this comment.
That is great! The test should not rely on systemwide settings.
haxtibal
force-pushed
the
tdmg/test_improvements
branch
from
September 7, 2026 13:41
d5082db to
9e03fa5
Compare
c47cc10 ("signers: protect from in-place overwrite") added a new attribute force_overwrite, but test_uefi_variable_sign_cli was not updated to initialize it. The test crashes since then. Initialize the attribute now to fix the test. Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
This prevents using systemwide /etc/softhsm2.conf and /etc/ssl/openssl.conf in tests to limit side effects of system configuration to test execution and vice versa. It comes with a workaround for SoftHSMv2 (exclude digest calculation from pkcs11 operations), because SoftHSMv2 is known to cause deadlock if used along with pkcs11-provider. The workaround mitigates it. See for example see openssl-projects/pkcs11-provider#634 for details. Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
Test environments should have the ability to control where build artifacts are generated. Enable this by evaluating make BUILD_DIR=... and pytest --build-dir=... Artifact paths are communicated to tests through fixtures that also assert existence of the required input artifacts. Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
debsign was only tested against debian.org, which is good as compatibility evidence. But it's not suitable for frequent integration test suite and the referenced packages may vanish from the external archive (replaced by newer versions). Now tests create their own deterministic input apt archive consisting of a minimal Debian package, containing minimal signables and a signed template package. Integration tests can now run without external dependencies. Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
The previous unit / integration grouping was not sufficient when distributions want to replay the tests in their restricted environment. Let's categorize as follows: - Unit tests: Fast, no system dependencies. Suitable to run for TDD and on every build. - Integration tests: Slower, system dependencies. Suitable to run as regression tests. - Live tests: Test against external services for compatibility. Not suited for isolated environments, maybe unstable. The tests can be started with invoke tu / ti / tl. Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
pkcs11-tool (opensc) and p11tool (gnutls-bin) have comparable functionality. p11tool is already used by the production code and support PKCS11 URIs. Thus replace pkcs11-tool entirely. Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
The path to sign-file was as a fallback derived from the running kernel version. However, on sandboxed systems installed kernel and running kernel may differ. It is more reliable to require explicit pytest --sign-file-path=... Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
The path to the module build infrastructure was derived from the running kernel version. However, on sandboxed systems installed kernel and running kernel may differ. Searching in the well known directory is more reliable. Where tighter control is needed, users can explicitly define the path with make -C KDIR=... Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
test_swu_sign relied on a committed .swu file as test input. We generally want to avoid committed blobs. Build the artifact with swugenerator from textual input instead. Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
haxtibal
force-pushed
the
tdmg/test_improvements
branch
from
September 7, 2026 15:00
9e03fa5 to
923c029
Compare
test_cert_from_file doesn't rely on real certificates from the system bundle. It just needs *some* file to test lookup when the URI ist not pkcs11 scheme. Signed-off-by: Tobias Deiminger <tobias.deiminger@linutronix.de> Reviewed-by: Florian Kauer <florian.kauer@linutronix.de>
haxtibal
force-pushed
the
tdmg/test_improvements
branch
from
September 8, 2026 07:33
6156e39 to
a097827
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The main driver for this series is to enable idiomatic testing with Debian pybuild / autopkgtest.