Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,10 +154,22 @@ The file is written to your Downloads folder as `iMotions.xml` (`%USERPROFILE%\D
The `Id="PilotDataSync"` is what ties the definition to the live stream. Every packet the relay sends carries the same source name:

```
E;1;PilotDataSync;;;;;AltitudeSync;1250.5;1250.5
E;1;PilotDataSync;;;;;AltitudeSync;1248.2;1250.5
```

Packets are semicolon-delimited and CRLF-terminated. The value appears twice because each sample carries both a `FlightModel*` and a `Pilot*` field.
Packets are semicolon-delimited and CRLF-terminated. Two values appear because
each sample carries both a `FlightModel*` and a `Pilot*` field, in that order:
the aircraft's true state and the pilot's instrument reading, which differ by
the simulated sensor error. Anything reading the stream for pilot performance
wants the second value.

`yaw` and `g_force` are exceptions. Neither has a pilot-side DataRef, so the
plugin sends the flightmodel value in both halves and the two are always equal.

Before #184 the plugin sent the pilot half only and the relay copied it into
both fields, so the two values were always identical. A relay running against
an older plugin still behaves that way, since it has nothing else to put
there.

### Which port iMotions listens on

Expand Down
34 changes: 34 additions & 0 deletions docs/telemetry_schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,40 @@ error), not the aircraft's raw physical state — see note 2 in
`key_datarefs.md`. `yaw` and `g_force` are exceptions: both come from
`sim/flightmodel/*` DataRefs rather than a pilot-side instrument.

### The paired wire payload (#184)

The plugin sends 16 values per sample, not 8: both halves of every value,
flightmodel first, in the order the relay turns into iMotions entries.

| position | 0, 1 | 2, 3 | 4, 5 | 6, 7 | 8, 9 | 10, 11 | 12, 13 | 14, 15 |
|---|---|---|---|---|---|---|---|---|
| value | altitude | airspeed | vertical speed | heading | roll | pitch | yaw | g force |

Each pair becomes one iMotions entry carrying two values, `FlightModel*` then
`Pilot*`, for example `E;1;PilotDataSync;;;;;AltitudeSync;1248.2;1250.5`.
`data_logger.py` records the second value, because the dataset is defined as
pilot-side readings (see GLOSSARY.md, "Pilot reading").

Note the order: vertical speed comes before heading. The older pilot-only
payload sent heading third and the relay reordered it on the way out; the
paired payload is in entry order so the relay pairs straight through.

The flightmodel half is converted into the pilot half's units in the plugin,
since `sim/flightmodel/position/elevation` is metres and
`sim/flightmodel/position/true_airspeed` is metres per second while the pilot
DataRefs are already feet and knots. Airspeed also differs in kind, not only in
sensor error: the flightmodel half is true airspeed and the pilot half is
indicated, so the gap between them grows with altitude.

`yaw` and `g_force` have no pilot-side DataRef, so both halves carry the
flightmodel value and are always equal.

**Older plugins send 8 values.** The relay still accepts 8 (and 4) and fills
the flightmodel half with a copy of the pilot value, because it has nothing
else to put there. Data captured that way cannot be used to compare the two
halves, in the same way that pre-#196 data cannot be used at face value for
`altitude` and `velocity`.

**Older data has mis-scaled `altitude` and `velocity` (fixed in #196):** before
#196, the plugin multiplied `altitude_ft_pilot` (already feet) by 3.28084 and
`airspeed_kts_pilot` (already knots) by 1.94384 before sending. Data collected
Expand Down
8 changes: 7 additions & 1 deletion inference/pilot_inference/capture/data_logger.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,14 @@ def parse_packet(line: str) -> tuple[str, float] | None:
if event_name not in EVENT_TO_COLUMN:
return None

# Each entry carries two values, FlightModel* then Pilot*. The dataset is
# defined as pilot-side instrument readings, carrying simulated sensor
# error rather than the aircraft's true state (see GLOSSARY.md, "Pilot
# reading"), so the second one is the one to record. Before #184 the relay
# sent the pilot value in both slots, which made the first slot look
# correct; it carries the flightmodel value now.
try:
value = float(parts[8])
value = float(parts[9])
except ValueError:
return None

Expand Down
17 changes: 14 additions & 3 deletions inference/pilot_inference/capture/replay.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,22 @@


def packets_for_row(row: dict) -> list[str]:
"""One sample as the relay's iMotions packets, with each value sent twice as the relay does."""
"""
One sample as the relay's iMotions packets.

Each entry carries two values, FlightModel* then Pilot*. A recorded CSV
holds pilot readings only, so the value goes in the pilot slot and is
repeated in the flightmodel slot as a stand-in: there is no flightmodel
half to replay. Consumers that want the pilot reading must take the second
value, as data_logger.parse_packet does.
"""
packets = []
for event, column in RELAY_EVENTS:
value = row.get(column, 0.0)
packets.append(f"E;1;PilotDataSync;;;;;{event};{value:.6f};{value:.6f}\r\n")
pilot = row.get(column, 0.0)
flightmodel = pilot
packets.append(
f"E;1;PilotDataSync;;;;;{event};{flightmodel:.6f};{pilot:.6f}\r\n"
)
return packets


Expand Down
16 changes: 16 additions & 0 deletions inference/tests/capture/test_data_logger.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,22 @@ def packet(event: str, value: float) -> str:
return f"E;1;PilotDataSync;;;;;{event};{value};{value}\r\n"


def test_parse_packet_takes_the_pilot_half_not_the_flightmodel_half():
"""
#184: each entry carries FlightModel* then Pilot*. The dataset is defined
as pilot-side instrument readings, so the second value is the one logged.
The helper above sends the same number twice, as a pre-#184 relay did, so
it cannot tell the two apart -- this uses values that differ.
"""
line = "E;1;PilotDataSync;;;;;AltitudeSync;1000.0;1234.5\r\n"
assert data_logger.parse_packet(line) == ("AltitudeSync", 1234.5)


def test_parse_packet_rejects_an_entry_missing_the_pilot_half():
"""A truncated entry is dropped rather than logged as flightmodel data."""
assert data_logger.parse_packet("E;1;PilotDataSync;;;;;AltitudeSync;1000.0\r\n") is None


def test_csv_fields_start_with_identity_columns():
assert data_logger.CSV_FIELDS[:3] == ["timestamp", "pilot_id", "flight_id"]
assert data_logger.CSV_FIELDS[3:] == data_logger.TELEMETRY_FIELDS
Expand Down
112 changes: 100 additions & 12 deletions relay/src/state.rs
Original file line number Diff line number Diff line change
Expand Up @@ -179,14 +179,49 @@ fn normalize_baton_payload(raw: &str) -> Vec<String> {
.collect()
}

/// One iMotions entry per flight value, in the order the relay sends them.
///
/// "Event" here is one iMotions entry, not a Sample in the GLOSSARY sense,
/// which is a whole set of telemetry values.
const IMOTIONS_EVENTS: [&str; 8] = [
"AltitudeSync",
"AirspeedSync",
"VerticalVelocitySync",
"HeadingSync",
"RollSync",
"PitchSync",
"YawSync",
"GForceSync",
];

/// The iMotions packets one baton sample turns into, in send order.
///
/// Pulled out of the TCP thread so the mapping can be tested without a socket,
/// and so both destinations are written the same bytes from one list rather
/// than each building its own.
///
/// Every iMotions entry carries two values, `FlightModel*` then `Pilot*`. A
/// 16-field sample has both, paired. The 4- and 8-field shapes are what older
/// plugins send: they carry the pilot half only, so the flightmodel half is
/// filled with the same number and the two cannot be compared (#184).
fn packets_for_fields(fields: &[String]) -> Vec<String> {
// Legacy duplication: one value standing in for both halves. Only for the
// field counts that predate the paired payload.
let pair = |value: &String| vec![value.clone(), value.clone()];

// The paired payload: both halves of every value, flightmodel first, in
// iMotions event order. Spelled out rather than left to the generic branch
// at the end so a test pins the order and the halves cannot quietly swap.
if fields.len() == IMOTIONS_EVENTS.len() * 2 {
return IMOTIONS_EVENTS
.iter()
.enumerate()
.map(|(i, name)| {
build_imotions_packet(name, &[fields[i * 2].clone(), fields[i * 2 + 1].clone()])
})
.collect();
}

if fields.len() == 4 {
let (alt, air, head, vv) = (&fields[0], &fields[1], &fields[2], &fields[3]);
return vec![
Expand All @@ -213,18 +248,7 @@ fn packets_for_fields(fields: &[String]) -> Vec<String> {
}

// Anything else: take the fields two at a time, in event order, and stop
// when they run out. "Event" here is one iMotions entry, not a Sample in
// the GLOSSARY sense, which is a whole set of telemetry values.
const IMOTIONS_EVENTS: [&str; 8] = [
"AltitudeSync",
"AirspeedSync",
"VerticalVelocitySync",
"HeadingSync",
"RollSync",
"PitchSync",
"YawSync",
"GForceSync",
];
// when they run out.
let mut packets = Vec::new();
let mut idx = 0usize;
for name in IMOTIONS_EVENTS {
Expand Down Expand Up @@ -1034,6 +1058,70 @@ mod tests {
assert_eq!(packets[3], "E;1;PilotDataSync;;;;;HeadingSync;90;90\r\n");
}

// The point of #184: both halves reach iMotions, so the instrument reading
// and the aircraft's true state can be compared.
#[test]
fn sixteen_fields_pair_flightmodel_with_pilot() {
let packets = packets_for_fields(&fields(&[
"1000", "1001", // altitude: flightmodel, pilot
"120", "121", // airspeed
"500", "501", // vertical velocity
"90", "91", // heading
"2", "3", // roll
"4", "5", // pitch
"91", "91", // yaw: no pilot DataRef exists, so both are flightmodel
"1.0", "1.1", // g force
]));

assert_eq!(packets.len(), 8);
assert_eq!(
packets[0],
"E;1;PilotDataSync;;;;;AltitudeSync;1000;1001\r\n"
);
assert_eq!(
packets[1],
"E;1;PilotDataSync;;;;;AirspeedSync;120;121\r\n"
);
// Vertical velocity before heading, as in the 8-field shape. The pairs
// are in event order, which is not baton's old field order.
assert_eq!(
packets[2],
"E;1;PilotDataSync;;;;;VerticalVelocitySync;500;501\r\n"
);
assert_eq!(packets[3], "E;1;PilotDataSync;;;;;HeadingSync;90;91\r\n");
assert_eq!(
packets[7],
"E;1;PilotDataSync;;;;;GForceSync;1.0;1.1\r\n"
);
}

// The regression #184 describes: with only the pilot half, the flightmodel
// field is a copy, so nothing can be compared. Pinned so the legacy shape
// is not mistaken for correct behaviour.
#[test]
fn eight_fields_still_duplicate_because_the_flightmodel_half_is_absent() {
let packets = packets_for_fields(&fields(&[
"1000", "120", "90", "500", "2", "3", "91", "1.0",
]));

for packet in &packets {
let values: Vec<&str> = packet
.trim_end()
.split(';')
.skip(8)
.collect();
assert_eq!(
values.len(),
2,
"every entry carries two values: {packet:?}"
);
assert_eq!(
values[0], values[1],
"the 8-field shape has no flightmodel half to send: {packet:?}"
);
}
}

#[test]
fn four_fields_send_only_the_flight_dynamics_factors() {
let packets = packets_for_fields(&fields(&["1000", "120", "90", "500"]));
Expand Down
Loading
Loading