Add SerialProgramsTests, a unit test runner without the GUI by dayfine · Pull Request #207 · dayfine/Arduino-Source

dayfine · 2026-11-03T08:03:22Z

Depends on #206

Adds SerialProgramsTests, a small executable that runs the unit tests without the GUI. It is for two things:

Testing while the app runs: it can run next to an open SerialPrograms (e.g. one driving a console) without touching it.
Running only what you touched: --filter PokemonFRLG takes about 1 s instead of about 12 s for every test.

It does not make builds faster: both executables link the same SerialProgramsLib, so rebuild time is the same (measured below).

Why: SerialPrograms --command-line-test-mode first reads the settings and scans serial ports and cameras, logs into the app's own SerialPrograms.log, and always runs every test. Its camera scan has made an open SerialPrograms on the same Mac switch its program panel mid-run.
How: Source/Tests/TestRunner_Main.cpp links SerialProgramsLib with only a QCoreApplication, defines use_qt_ui() (from Replace the USE_QT_UI global with use_qt_ui() #206) as false, and logs to its own folder (~/Library/Application Support/SerialProgramsTests/ on macOS). Built by default; -DTEST_RUNNER_EXE=OFF turns it off.
Flags: --filter <text> (repeatable, name substring), --list, --augment-color-distortions. The exit code is the number of failed tests (capped at 254), or 255 for a bad argument or no match. CommandLineUnitTestRunner::run() gained the filter; the GUI test mode still runs everything and returns 0 or 1.
Build and run: the CMake target is SerialProgramsTests, built by default with SerialPrograms (configure with -DTEST_RUNNER_EXE=OFF to skip it). ninja -C <build> SerialProgramsTests builds only it, and the binary lands at <build>/SerialProgramsTests. Run <build>/SerialProgramsTests --filter PokemonFRLG, or --list to see what a filter selects. Image tests need the CommandLineTests images in a UnitTestResources/ folder in or above <build>, as for the GUI test mode. The in-repo AGENTS.md now says the same.
Impact: the build is not faster. Both targets link the same static SerialProgramsLib, and after a one-file edit re-archiving it takes 5–18 s while linking either executable takes 0.6–0.9 s. The gain is at run time: --filter PokemonFRLG finishes in 0.6–1.2 s, against about 12 s for the GUI mode's whole suite on the same Mac, and it leaves an open app alone.
When to use it: the edit-test loop on logic, sim and detector-image tests. Build the full SerialPrograms for anything touching UI or the program panels, and before a console run.

Testing

macOS, one configure with ccache, -j4, no other build running: touch or edit one FRLG .cpp, then ninja SerialPrograms took 6.5–20 s and ninja SerialProgramsTests 5.9–22 s over six alternating runs each, so no difference beyond noise.
--list (20,339 tests on main) takes 0.4–0.5 s; the GUI mode reaches its first test about 0.2 s after starting, so startup is not different either.
Ran it while the lab SerialPrograms was running a program; nothing went to SerialPrograms.log. Image tests need UnitTestResources/ in the build folder, as in the GUI mode; without it they fail with FileException.
Known issue on current main: Sockets (Qt) - close() cancels an in-flight send (Sockets: make the Qt client's close() cancel in-flight sends PokemonAutomation/Arduino-Source#1831) hung in 4 of 4 runs here under heavy load, so an unfiltered run did not finish. It looks like a race in ClientSocket_Qt::send() rather than this runner, but I did not try it in the GUI mode.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E

SerialProgramsTests runs the same unit tests as --command-line-test-mode, but opens no window, reads and writes no settings, and does not scan serial ports or cameras, so it can run while SerialPrograms is open. --filter <text> (repeatable) selects tests by name substring, --list prints the selected names, and the exit code is the number of failed tests (255 for a bad argument or no matching test). CommandLineUnitTestRunner::run() takes the filters and returns the failure count; --command-line-test-mode still returns 0 or 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E

dayfine · 2026-11-03T19:07:49Z

do we need to explain / document how to build this thing? (the CMake changes)?

dayfine · 2026-11-03T19:03:23Z

+namespace{
+bool matches_any(const std::string& test_name, const std::vector<std::string>& filters){
+    if (filters.empty()){
+        return true;
+    }
+    for (const std::string& filter : filters){
+        if (test_name.contains(filter)){
+            return true;
+        }
+    }
+    return false;
+}
+}


google C++ style, put this unnamed namespace on the top, between line 45-50

ad6aa40: moved the unnamed namespace (matches_any) to the top of the file, right after namespace ComputerPrograms{ opens.

dayfine · 2026-11-03T19:03:54Z



-bool CommandLineUnitTestRunner::run(bool augment_color_distortions){
+size_t CommandLineUnitTestRunner::run(bool augment_color_distortions, const std::vector<std::string>& filters){


is there a span / view type in C++ 23 that can be used?

ad6aa40: the project is C++23; filters are now std::span in run(), unit_test_names() and run_unit_tests(). The runner keeps string_views of argv, so nothing is copied.

dayfine · 2026-11-03T19:04:58Z

+}
+
+
+namespace{


ad6aa40: read it as comment 1: the unnamed namespace is now the first thing inside namespace PokemonAutomation, before use_qt_ui().

dayfine · 2026-11-03T19:05:14Z

+#include "ComputerPrograms/ComputerPrograms_UnitTestRunner.h"
+#include "Tests/CommandLineTests.h"
+
+using namespace PokemonAutomation;


don't do this -- check google C++ style / ToTw on relevant recommendations

ad6aa40: removed the using-directive. Everything now lives in namespace PokemonAutomation, and main() just calls PokemonAutomation::test_runner_main(argc, argv).

dayfine · 2026-11-03T19:06:22Z

+    GlobalThreadPools::qt_event_threadpool().stop();
+}
+
+}


nit: } // namespace

ad6aa40: closed with } // namespace, which is what the repo uses most (Common/ and Source/).

- Pass test-name filters as std::span<const std::string_view>; the runner keeps views of argv instead of copying strings. - Put the unnamed namespaces at the top, inside namespace PokemonAutomation, closed with "} // namespace"; drop the using-directive and let main() call PokemonAutomation::test_runner_main(). - AGENTS.md: how to build and run SerialProgramsTests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E

dayfine · 2026-11-03T19:12:56Z

Yes. ad6aa40 adds a Build and run part to the description: the SerialProgramsTests target, built by default and turned off with -DTEST_RUNNER_EXE=OFF, ninja SerialProgramsTests, the binary at <build>/SerialProgramsTests, --filter/--list, and UnitTestResources/ for the image tests. The in-repo AGENTS.md, which already described --command-line-test-mode, gets the same note. The repo's README and docs/ don't list build targets. For the developer docs site, the matching section for Developer/Tests.md is on the fork branch dayfine/pokemonautomation.github.io:docs/serialprograms-tests (9261d58). It is not opened against the site.

dayfine · 2026-11-03T20:50:43Z

    static UnitTestDatabase database = make_UNIT_TESTS_ALL();
    return database;
 }
+std::vector<std::string> unit_test_names(std::span<const std::string_view> filters){


blank line above and below this function

c213cac: added a blank line above and below unit_test_names().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E

dayfine and others added 2 commits November 3, 2026 00:01

Merge origin/build/use-qt-ui-accessor into build/headless-tests

43edf27

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E

dayfine commented Nov 3, 2026

View reviewed changes

dayfine and others added 2 commits November 3, 2026 13:43

Unit test runner: blank lines around unit_test_names()

c213cac

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E

Merge origin/build/use-qt-ui-accessor into build/headless-tests

e8d97ba

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E



		bool CommandLineUnitTestRunner::run(bool augment_color_distortions){
		size_t CommandLineUnitTestRunner::run(bool augment_color_distortions, const std::vector<std::string>& filters){

		}


		namespace{

Conversation

dayfine commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dayfine commented Nov 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dayfine commented Nov 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

dayfine commented Nov 3, 2026 •

edited

Loading