feat(autotune): menu to add/edit presets

.github/workflows/autotune_tests.yml

@@ -0,0 +1,35 @@
+name: autotune_tests
+on:
+  pull_request:
+  push:
+    branches: [master]
+
+jobs:
+  autotune_tests:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: autotune
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      # Qt's "offscreen" platform plugin still needs these shared libraries.
+      - name: Install Qt offscreen system libraries
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y \
+            libegl1 libgl1 libxkbcommon0 libdbus-1-3
+
+      - name: Install dependencies
+        run: |
+          pip install poetry
+          poetry install
+
+      - name: Run tests
+        env:
+          QT_QPA_PLATFORM: offscreen
+        run: poetry run pytest -v

autotune/CLAUDE.md

@@ -0,0 +1,94 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+This is the `autotune` module within the Flight Control Prototyping Scripts — a PyQt5 GUI application for PX4 fixed-wing/multirotor rate controller tuning via system identification from flight logs.
+
+## Setup and Running
+
+**Install with Poetry (recommended):**
+```bash
+cd /home/mathieu/src/Flight_Control_Prototyping_Scripts/autotune
+poetry install
+poetry run python3 autotune.py
+```
+
+**Or with venv:**
+```bash
+python3 -m venv virtualenv-test
+source virtualenv-test/bin/activate
+pip3 install numpy scipy pyulog control pyqt5 pyyaml
+python3 autotune.py
+```
+
+**Run simulation-based test (no GUI data needed):**
+```bash
+poetry run python3 simulated_autotune.py
+```
+
+**Run the unit/functional tests:**
+```bash
+# pytest is in the dev group; the GUI tests need Qt's offscreen platform
+QT_QPA_PLATFORM=offscreen poetry run pytest -v
+```
+`test_presets.py` covers preset load/save (seed, round-trip, order, corrupt-file
+fallback); `test_preset_dialogs.py` drives `PresetEditDialog` headlessly to verify
+the add/edit/rename/create/delete behavior. These run in CI via
+`.github/workflows/autotune_tests.yml`.
+
+## Code Quality
+
+The root repo uses pre-commit hooks with **black** (formatting) and **isort** (imports, Black profile). Run manually:
+```bash
+cd /home/mathieu/src/Flight_Control_Prototyping_Scripts
+poetry run black autotune/
+poetry run isort autotune/
+```
+
+## Architecture
+
+### Data flow
+```
+ULog Flight Log → data_extractor.py → data_selection_window.py →
+system_identification.py (ARX/RLS) → pid_design.py (GMVC) →
+autotune.py (Bode/step response validation) → PX4 gains
+```
+
+### Module responsibilities
+
+- **`autotune.py`** — Main PyQt5 GUI window (~900 lines). Orchestrates the full workflow: log loading, axis/vehicle-type selection, model parameter configuration, gain computation, and result visualization via matplotlib figures embedded in Qt.
+
+- **`data_extractor.py`** — Parses PX4 ULog binary logs using `pyulog`. Extracts and interpolates signals (rates, setpoints, actuator outputs, airspeed). Returns numpy arrays aligned to a common time base.
+
+- **`data_selection_window.py`** — Interactive matplotlib window for selecting the maneuver time window and inspecting signal quality/coherence before running identification. Loads input/output presets via `presets.py` and lets the user add/edit/delete them through `preset_dialogs.py`.
+
+- **`presets.py`** — Loads/saves input/output presets from the user-editable `presets.yaml` (next to the code). Seeds the file with `DEFAULT_PRESETS` on first run; falls back to defaults if the file is missing or unparseable.
+
+- **`preset_dialogs.py`** — `PresetEditDialog`: a Qt dialog to add, edit, or delete a preset, with an old→new signal diff that switches to "create" mode when the preset is renamed.
+
+- **`system_identification.py`** — Preprocesses signals (bias removal, filtering) and runs weighted RLS to fit an ARX model. Returns numerator/denominator polynomial coefficients.
+
+- **`arx_rls.py`** — Core recursive least-squares implementation. Assumes ARX model: `A(q⁻¹)y(k) = q⁻ᵈ B(q⁻¹)u(k) + A(q⁻¹)e(k)`. Uses matrix-inversion-free update for efficiency.
+
+- **`pid_design.py`** — Computes PX4-compatible P/I/D gains from identified model polynomials using **General Minimum Variance Control (GMVC)**. Inputs: ARX coefficients, sample time, rise time, damping ratio.
+
+- **`pid_analyse_window.py`** — Secondary PyQt5 window with pole-zero plots, Bode diagrams, stability margins, and disturbance response for the tuned controller.
+
+- **`closed_loop_sim.py`** — Simulates closed-loop behavior with the identified model and designed gains to validate stability before applying to the aircraft.
+
+- **`simulated_autotune.py`** — End-to-end test on a synthetic 2nd-order system (no flight log needed).
+
+### Key dependencies
+
+| Package | Purpose |
+|---------|---------|
+| `control` | Transfer functions, Bode plots, pole-zero, simulation |
+| `pyulog` | PX4 ULog binary flight log parsing |
+| `pyqt5` | GUI framework |
+| `numpy` / `scipy` | Numerical computing, signal processing |
+| `pyyaml` | Read/write the user-editable `presets.yaml` |
+
+### Sample logs for testing
+Located in `logs/`: `quadrotor_sitl_jmavsim.ulg`, `quadrotor_x500.ulg`, `vtol_standard.ulg`.

autotune/data_selection_window.py

@@ -4,6 +4,9 @@
 from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
 from matplotlib.widgets import SpanSelector
 from pid_analyse_window import PIDAnalyseWindow
+from preset_dialogs import PresetEditDialog
+from presets import load_presets, save_presets
+from PyQt5.QtGui import QBrush, QColor
 from PyQt5.QtWidgets import (
     QComboBox,
     QDialog,
@@ -13,7 +16,6 @@
     QLabel,
     QMessageBox,
     QPushButton,
-    QRadioButton,
     QVBoxLayout,
 )
 from scipy import signal
@@ -24,76 +26,10 @@ class DataSelectionWindow(QDialog):
     def __init__(self, filename):
         QDialog.__init__(self)
 
-        self.preset_candidates = {
-            "Rollrate": {
-                "input": "vehicle_torque_setpoint/xyz[0].0",
-                "output": "vehicle_angular_velocity/xyz[0].0",
-                "input_legacy": "actuator_controls_0/control[0].0",
-            },
-            "Pitchrate": {
-                "input": "vehicle_torque_setpoint/xyz[1].0",
-                "output": "vehicle_angular_velocity/xyz[1].0",
-                "input_legacy": "actuator_controls_0/control[1].0",
-            },
-            "Yawrate": {
-                "input": "vehicle_torque_setpoint/xyz[2].0",
-                "output": "vehicle_angular_velocity/xyz[2].0",
-                "input_legacy": "actuator_controls_0/control[2].0",
-            },
-            "Rollrate(FW)": {
-                "input": "vehicle_torque_setpoint/xyz[0].1",
-                "output": "vehicle_angular_velocity/xyz[0].0",
-                "input_legacy": "actuator_controls_1/control[0].0",
-            },
-            "Pitchrate(FW)": {
-                "input": "vehicle_torque_setpoint/xyz[1].1",
-                "output": "vehicle_angular_velocity/xyz[1].0",
-                "input_legacy": "actuator_controls_1/control[1].0",
-            },
-            "Yawrate(FW)": {
-                "input": "vehicle_torque_setpoint/xyz[2].1",
-                "output": "vehicle_angular_velocity/xyz[2].0",
-                "input_legacy": "actuator_controls_1/control[2].0",
-            },
-            "Rollrate(closed-loop)": {
-                "input": "vehicle_rates_setpoint/roll.0",
-                "output": "vehicle_angular_velocity/xyz[0].0",
-            },
-            "Pitchrate(closed-loop)": {
-                "input": "vehicle_rates_setpoint/pitch.0",
-                "output": "vehicle_angular_velocity/xyz[1].0",
-            },
-            "Yawrate(closed-loop)": {
-                "input": "vehicle_rates_setpoint/yaw.0",
-                "output": "vehicle_angular_velocity/xyz[2].0",
-            },
-            "GimbalRollRate": {
-                "input": "motor_state/roll.effort_cmd.0",
-                "output": "motor_angular_rates/roll.angular_rate.0",
-            },
-            "GimbalPitchRate": {
-                "input": "motor_state/pitch.effort_cmd.0",
-                "output": "motor_angular_rates/pitch.angular_rate.0",
-            },
-            "GimbalYawRate": {
-                "input": "motor_state/yaw.effort_cmd.0",
-                "output": "motor_angular_rates/yaw.angular_rate.0",
-            },
-            "GimbalRollAtt": {
-                "input": "motor_control/motor_commands.roll.angular_rate_setpoint.0",
-                "output": "attitude_info/roll.0",
-            },
-            "GimbalPitchAtt": {
-                "input": "motor_control/motor_commands.pitch.angular_rate_setpoint.0",
-                "output": "attitude_info/pitch.0",
-            },
-            "GimbalYawAtt": {
-                "input": "motor_control/motor_commands.yaw.angular_rate_setpoint.0",
-                "output": "attitude_info/yaw.0",
-            },
-        }
+        self.preset_candidates = load_presets()
 
         self.presets = {}
+        self.topic_names = []
 
         self.t = []
         self.u = []
@@ -121,7 +57,15 @@ def __init__(self, filename):
         self.combo_preset.setEditable(False)
 
         self.combo_preset.currentIndexChanged.connect(self.selectPreset)
-        in_out_group.addRow(QLabel("Preset:"), self.combo_preset)
+        preset_row = QHBoxLayout()
+        preset_row.addWidget(self.combo_preset)
+        self.btn_add_preset = QPushButton("Add")
+        self.btn_add_preset.clicked.connect(self.addPreset)
+        preset_row.addWidget(self.btn_add_preset)
+        self.btn_edit_preset = QPushButton("Edit")
+        self.btn_edit_preset.clicked.connect(self.editPreset)
+        preset_row.addWidget(self.btn_edit_preset)
+        in_out_group.addRow(QLabel("Preset:"), preset_row)
 
         self.combo_u = SearchableComboBox()
         self.combo_u.currentIndexChanged.connect(self.selectUData)
@@ -187,30 +131,43 @@ def openFile(self):
                 f"{topic.topic_name}/{topic.variable_name}.{topic.instance}"
                 for topic in self.topics
             ]
+            self.topic_names = list_names
             self.combo_u.clear()
             self.combo_u.addItems(list_names)
             self.combo_y.clear()
             self.combo_y.addItems(list_names)
 
-            # Trigger preset selection. If no preset matches the available
-            # topics, leave the input/output combos for manual configuration.
+            # Show all presets, then select the first one that matches the
+            # available topics. Presets that don't match are still listed (and
+            # editable) but greyed out and won't auto-fill the input/output.
             self.fillPresets()
-            if self.presets:
-                self.combo_preset.setCurrentIndex(0)
-                self.selectPreset(0)
+            for i in range(self.combo_preset.count()):
+                if self.combo_preset.itemText(i) in self.presets:
+                    self.combo_preset.setCurrentIndex(i)
+                    self.selectPreset(i)
+                    break
 
     def fillPresets(self):
+        self.combo_preset.blockSignals(True)
         self.combo_preset.clear()
         self.presets = {}
 
-        for candidate in self.preset_candidates:
+        # List every preset so the user can see and edit all of them. Track
+        # which ones match the loaded log's topics in self.presets.
+        for index, candidate in enumerate(self.preset_candidates):
+            self.combo_preset.addItem(candidate)
             (index_u, index_y) = self.findInputOutputIndex(
                 self.preset_candidates[candidate]
             )
             if index_u > -1 and index_y > -1:
                 self.presets[candidate] = self.preset_candidates[candidate]
+            else:
+                # Grey out presets that don't match the current log.
+                item = self.combo_preset.model().item(index)
+                if item is not None:
+                    item.setForeground(QBrush(QColor("gray")))
 
-        self.combo_preset.addItems(list(self.presets.keys()))
+        self.combo_preset.blockSignals(False)
 
     def printRangeError(self):
         msg = QMessageBox()
@@ -220,11 +177,10 @@ def printRangeError(self):
         msg.exec_()
 
     def selectPreset(self, index):
-        preset_keys = list(self.presets.keys())
-        if index < 0 or index >= len(preset_keys):
+        name = self.combo_preset.itemText(index)
+        if name not in self.preset_candidates:
             return
-        preset_key = preset_keys[index]
-        preset = self.presets[preset_key]
+        preset = self.preset_candidates[name]
         (index_u, index_y) = self.findInputOutputIndex(preset)
 
         if index_u > -1:
@@ -246,6 +202,62 @@ def findInputOutputIndex(self, preset):
 
         return (index_u, index_y)
 
+    def addPreset(self):
+        self._openPresetDialog(create=True)
+
+    def editPreset(self):
+        self._openPresetDialog(create=False)
+
+    def _openPresetDialog(self, create):
+        if not self.topic_names:
+            QMessageBox.information(
+                self, "No log loaded", "Open a log file before editing presets."
+            )
+            return
+
+        # The currently selected preset (from the topic-matched subset shown in
+        # the combo); may be empty if no preset matched the log.
+        selected_name = self.combo_preset.currentText()
+        if selected_name not in self.preset_candidates:
+            selected_name = None
+
+        if not create and selected_name is None:
+            QMessageBox.information(
+                self, "No preset selected", "Select a preset to edit first."
+            )
+            return
+
+        dialog = PresetEditDialog(
+            self,
+            create,
+            selected_name,
+            self.preset_candidates,
+            self.topic_names,
+            self.combo_u.currentText(),
+            self.combo_y.currentText(),
+        )
+        dialog.exec_()
+
+        if dialog.result_action == "save":
+            if dialog.remove_name and dialog.remove_name != dialog.name:
+                self.preset_candidates.pop(dialog.remove_name, None)
+            self.preset_candidates[dialog.name] = dialog.preset
+            target = dialog.name
+        elif dialog.result_action == "delete":
+            self.preset_candidates.pop(dialog.remove_name, None)
+            target = None
+        else:
+            return
+
+        save_presets(self.preset_candidates)
+        self.fillPresets()
+
+        # Reselect the affected preset if it is still present and matches.
+        if target is not None:
+            index = self.combo_preset.findText(target)
+            if index > -1:
+                self.combo_preset.setCurrentIndex(index)
+
     def selectUData(self, index):
         self.index_u = index
         (self.t, self.u) = self.data_extractor.getPreview(self.topics[index])