-
Notifications
You must be signed in to change notification settings - Fork 10
feat: add TimeDaemon CIT with pip hub infrastructure #122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
gordon9901
wants to merge
14
commits into
eclipse-score:main
Choose a base branch
from
gordon9901:ecarx_time_daemon_cit
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
8f9ae18
feat: add TimeDaemon CIT with pip hub infrastructure
gordon9901 2647308
Update Bazel lockfile
gordon9901 057fa74
feat: add TimeDaemon CIT with pip hub infrastructure
gordon9901 4266671
style: apply clang-format to test scenario sources
gordon9901 6a25ee4
drop premature MIT visibility from shm_ptp_engine
gordon9901 21d57ff
fix: address PR review comments on TimeDaemon CIT
gordon9901 d765fee
update lock file
gordon9901 4bf3136
Merge branch 'main' into ecarx_time_daemon_cit
gordon9901 d72e98b
Merge branch 'main' into ecarx_time_daemon_cit
gordon9901 2dbb8bd
merge: resolve MODULE.bazel.lock conflict with main
gordon9901 bc6a669
Merge branch 'main' into ecarx_time_daemon_cit
gordon9901 05aafa5
chore: add local clang-tidy config to silence noise
gordon9901 321294a
test(time_daemon/cit): narrow to multi-module integration
gordon9901 b0aa357
Merge branch 'main' into ecarx_time_daemon_cit
gordon9901 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
| # Local clang-tidy overrides for the score_time module. | ||
| # | ||
| # The S-CORE baseline from @score_cpp_policies//clang_tidy:.clang-tidy is | ||
| # applied first; settings here are layered on top per clang-tidy's standard | ||
| # merge rules. Only module-specific suppressions belong here — do not remove | ||
| # baseline checks. | ||
|
|
||
| # Only analyze headers inside the score/ source tree. Without this, | ||
| # clang-tidy picks up warnings from externally provided headers | ||
| # (e.g. toolchain redacted_dates.h) that we cannot fix. | ||
| HeaderFilterRegex: 'score/' | ||
|
|
||
| # Suppress the builtin-macro-redefined diagnostic emitted by the LLVM | ||
| # toolchain's redacted_dates.h (which redefines __DATE__/__TIME__ for | ||
| # deterministic builds). This is injected by the toolchain via -include | ||
| # and is not user code, so suppressing it here is appropriate. | ||
| Checks: >- | ||
| -clang-diagnostic-builtin-macro-redefined |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
|
|
||
| load("@pip_time_daemon_venv//:requirements.bzl", "all_requirements") | ||
| load("@rules_python//python:pip.bzl", "compile_pip_requirements") | ||
| load("@score_tooling//python_basics:defs.bzl", "score_py_pytest", "score_virtualenv") | ||
|
|
||
| # To regenerate requirements.txt.lock after editing requirements.txt, run: | ||
| # bazel run //score/time_daemon/tests/component_tests:requirements.update | ||
| compile_pip_requirements( | ||
| name = "requirements", | ||
| srcs = [ | ||
| "requirements.txt", | ||
| "@score_tooling//python_basics:requirements.txt", | ||
| ], | ||
| requirements_txt = "requirements.txt.lock", | ||
| tags = ["manual"], | ||
| ) | ||
|
|
||
| score_virtualenv( | ||
| name = "python_tc_venv", | ||
| reqs = all_requirements, | ||
| venv_name = ".python_tc_venv", | ||
| ) | ||
|
|
||
| score_py_pytest( | ||
| name = "time_daemon_cit", | ||
| srcs = glob(["tests/**/*.py"]) + ["conftest.py"], | ||
| args = [ | ||
| "--cpp-target-path=$(rootpath //score/time_daemon/tests/test_scenarios/cpp:test_scenarios)", | ||
| ], | ||
| data = [ | ||
| ":python_tc_venv", | ||
| "//score/time_daemon/tests/test_scenarios/cpp:test_scenarios", | ||
| ], | ||
| pytest_config = ":pytest.ini", | ||
| tags = ["integration"], | ||
| deps = all_requirements, | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| <!-- ---------------------------------------------------------------------------- | ||
| Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
|
|
||
| See the NOTICE file(s) distributed with this work for additional | ||
| information regarding copyright ownership. | ||
|
|
||
| This program and the accompanying materials are made available under the | ||
| terms of the Apache License Version 2.0 which is available at | ||
| https://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| SPDX-License-Identifier: Apache-2.0 | ||
| ----------------------------------------------------------------------------- --> | ||
|
|
||
| # TimeDaemon Component Integration Tests (CIT) | ||
|
|
||
| ## Scope | ||
|
|
||
| Component integration tests verify **behaviour across multiple modules inside the | ||
| TimeDaemon component** — wiring, data flow, IPC, and lifecycle — from the | ||
| component's public surface. They are NOT a replacement for unit tests. | ||
|
|
||
| | Layer | Tool | Where it lives | What it covers | | ||
| |-------|------|----------------|----------------| | ||
| | Unit | gtest / gmock | `score/time_daemon/src/**/*_test.cpp` | Individual classes and small modules, white-box | | ||
| | **CIT (this layer)** | **pytest + test_scenarios binary** | `score/time_daemon/tests/component_tests/` | **Multi-module integration, cross-module data flow, IPC, lifecycle** | | ||
| | Module integration | pytest + ITF | `score/tests/module_integration_tests/` | TimeSlave ↔ TimeDaemon across components | | ||
| | System | pytest + ITF + QEMU / Docker | `score/tests/gptp_pipeline/` | Multi-binary end-to-end across OS targets | | ||
|
|
||
| **Rule of thumb** — if a scenario only instantiates a single production class and | ||
| exercises its public API, it belongs in the gtest unit layer under `src/`, not | ||
| here. A CIT scenario should exercise **two or more production objects wired | ||
| together** (callback chaining, message-bus topics, shared-memory IPC, full daemon | ||
| startup, etc.). | ||
|
|
||
| ## Architecture | ||
|
|
||
| ``` | ||
| pytest (Python) test runner, assertions, reporting | ||
| │ | ||
| ▼ subprocess + structured-log parsing | ||
| test_scenarios (C++ binary) scenario runner, builds real production objects | ||
| │ | ||
| ▼ factory + public API | ||
| TimeDaemon production code classes under test | ||
| ``` | ||
|
|
||
| - Python side lives under `tests/` and subclasses `testing_utils.Scenario`. | ||
| - C++ side lives under `../test_scenarios/cpp/` and uses the | ||
| `@score_test_scenarios` framework (`Scenario`, `TRACING_INFO`, CLI runner). | ||
| - Each pytest class declares a `scenario_name` fixture that selects which | ||
| C++ scenario to run; results come back as exit code + structured JSON log lines | ||
| parsed by `testing_utils.LogContainer`. | ||
|
|
||
| ## Scenarios | ||
|
|
||
| | Scenario name | Modules involved | What it verifies | | ||
| |---------------|-----------------|------------------| | ||
| | `verification.pipeline` | `GPTPStubMachine` → `SvtVerificationMachine` (3 validators) | PTP data flows correctly through the full verification pipeline | | ||
| | `ipc.shm_roundtrip` | `SvtPublisher` ↔ `SvtReceiver` via shared memory | Shared-memory IPC round-trip preserves data integrity | | ||
| | `daemon.lifecycle` | `SvtHandler` + all subsystems (MessageBroker, machines, IPC) | Daemon initialises and shuts down cleanly through all lifecycle states | | ||
|
|
||
| ## Running | ||
|
|
||
| ```bash | ||
| bazel test //score/time_daemon/tests/component_tests:time_daemon_cit | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
| import json | ||
| import os | ||
| from pathlib import Path | ||
|
|
||
| import pytest | ||
| from testing_utils import BazelTools | ||
|
|
||
| FAILED_CONFIGS = [] | ||
|
|
||
|
|
||
| def pytest_addoption(parser): | ||
| parser.addoption( | ||
| "--traces", | ||
| choices=["none", "target", "all"], | ||
| default="none", | ||
| help=( | ||
| "Verbosity of traces in output. " | ||
| '"none" - show no traces, ' | ||
| '"target" - show traces from test code, ' | ||
| '"all" - show all traces.' | ||
| ), | ||
| ) | ||
| parser.addoption( | ||
| "--cpp-target-name", | ||
| type=str, | ||
| default="//score/time_daemon/tests/test_scenarios/cpp:test_scenarios", | ||
| help="C++ test scenario executable Bazel target.", | ||
| ) | ||
| parser.addoption( | ||
| "--cpp-target-path", | ||
| type=Path, | ||
| help="Path to the pre-built C++ test scenario executable.", | ||
| ) | ||
| parser.addoption( | ||
| "--build-scenarios", | ||
| action="store_true", | ||
| help="Build test scenario executables before running tests.", | ||
| ) | ||
| parser.addoption( | ||
| "--build-scenarios-timeout", | ||
| type=float, | ||
| default=180.0, | ||
| help="Build command timeout in seconds. Default: %(default)s", | ||
| ) | ||
| parser.addoption( | ||
| "--default-execution-timeout", | ||
| type=float, | ||
| default=10.0, | ||
| help="Default scenario execution timeout in seconds. Default: %(default)s", | ||
| ) | ||
| parser.addoption( | ||
| "--bazel-config", | ||
| type=str, | ||
| default="time-x86_64-linux", | ||
| help="Bazel config to use when building C++ test scenarios. Default: %(default)s", | ||
| ) | ||
|
|
||
|
|
||
| @pytest.hookimpl(tryfirst=True) | ||
| def pytest_sessionstart(session): | ||
| try: | ||
| if session.config.getoption("--build-scenarios"): | ||
| build_timeout = session.config.getoption("--build-scenarios-timeout") | ||
| print("Building C++ test scenarios executable...") | ||
| bazel_config = session.config.getoption("--bazel-config") | ||
| bazel_tools = BazelTools(option_prefix="cpp", build_timeout=build_timeout) | ||
| cpp_target_name = session.config.getoption("--cpp-target-name") | ||
| bazel_tools.build(cpp_target_name, f"--config={bazel_config}") | ||
| except Exception as e: | ||
| pytest.exit(str(e), returncode=1) | ||
|
|
||
|
|
||
| def pytest_html_report_title(report): | ||
| report.title = "Time Daemon Component Integration Tests Report" | ||
|
|
||
|
|
||
| def pytest_html_results_table_header(cells): | ||
| cells.insert(1, "<th>Test Input</th>") | ||
| cells.insert(2, "<th>Description</th>") | ||
| cells.insert(3, "<th>Test Scenario Name</th>") | ||
| cells.insert(4, "<th>Test Scenario Command</th>") | ||
|
|
||
|
|
||
| def pytest_html_results_table_row(report, cells): | ||
| cells.insert( | ||
| 1, | ||
| f'<td><pre style="white-space:pre-wrap;word-wrap:break-word">{json.dumps(report.input)}</pre></td>', | ||
| ) | ||
| cells.insert(2, f"<td><pre>{report.description}</pre></td>") | ||
| cells.insert(3, f"<td><pre>{report.scenario}</pre></td>") | ||
| cells.insert(4, f"<td><pre>{report.command}</pre></td>") | ||
|
|
||
|
|
||
| @pytest.hookimpl(hookwrapper=True) | ||
| def pytest_runtest_makereport(item, call): | ||
| outcome = yield | ||
| report = outcome.get_result() | ||
| report.description = str(item.function.__doc__) | ||
| report.scenario = item.funcargs.get("scenario_name", "") | ||
| report.input = item.funcargs.get("test_config", "") | ||
|
|
||
| command = [] | ||
| for token in item.funcargs.get("command", ""): | ||
| if " " in token: | ||
| command.append(f"'{token}'") | ||
| else: | ||
| command.append(token) | ||
| report.command = " ".join(command) | ||
|
|
||
| if report.failed: | ||
| FAILED_CONFIGS.append( | ||
| { | ||
| "nodeid": report.nodeid, | ||
| "command": report.command, | ||
| } | ||
| ) | ||
|
|
||
|
|
||
| def pytest_terminal_summary(terminalreporter): | ||
| if not FAILED_CONFIGS: | ||
| return | ||
| terminalreporter.write_sep("=", "Failed tests reproduction info") | ||
| terminalreporter.write_line( | ||
| "Run failed scenarios from the repo root working directory\n" | ||
| ) | ||
| for entry in FAILED_CONFIGS: | ||
| terminalreporter.write_line( | ||
| f"{entry['nodeid']} | Run command:\n{entry['command']}\n" | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
| [pytest] | ||
| addopts = -v | ||
| testpaths = tests | ||
| pythonpath = | ||
| . | ||
| tests | ||
| junit_family = xunit1 | ||
| filterwarnings = | ||
| ignore:record_property is incompatible with junit_family:pytest.PytestWarning | ||
| ignore:record_xml_attribute is an experimental feature:pytest.PytestExperimentalApiWarning |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| psutil | ||
| pytest-metadata | ||
| pytest-env | ||
| testing-utils @ git+https://github.com/eclipse-score/testing_tools.git@a2f9cded3deb636f5dc800bf7a47131487119721 |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.