diff --git a/GLOSSARY.md b/GLOSSARY.md index f16c904..b8d45bb 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -38,6 +38,10 @@ _Avoid_: Fake iMotions A second destination the **Relay** copies every packet to, so the Python side can read a flight while iMotions records it. The copy is best effort: packets are dropped if the mirror falls behind, and the iMotions stream is never held up for it (`docs/adr/0003`). _Avoid_: Python stream, fan-out, secondary connection, sink +**Listener**: +The Python program that reads a **Flight**'s **Telemetry** from the **Relay**, through the **Mirror** or directly when there is no iMotions: the capture logger or live groups. +_Avoid_: Consumer, sink. "Sidecar" in `docs/adr/0004` means the live-groups listener only. + ### Performance measurement **Airspeed**: @@ -98,6 +102,10 @@ _Avoid_: User, subject, participant One continuous simulator session by one pilot. _Avoid_: Session, run, trip +**Instructor**: +The person who gives the **Instructions** during a **Flight** and runs the software that records it. +_Avoid_: Host, operator, experimenter + **Pilot metadata**: Facts about a pilot that are not telemetry: certification level, flight hours and pilot rating. _Avoid_: Pilot profile, demographics @@ -132,6 +140,7 @@ _Avoid_: Fake data, dummy data, mock data - A **Pilot** flies many **Flights**; a **Flight** produces many **Samples**. - A **Sample** has one value per telemetry field, and one **Deviation** per **Flight dynamics factor** once a **Target** exists. +- An **Instructor** gives a **Flight**'s **Instructions** in the order its **Brief** lists them. - A **Flight** is divided into **Segments**, and is also flown as a sequence of **Legs**. A **Brief** lists its **Instructions** in order. - An **Instruction** sets new **Targets** from the moment it is given. The targets it changes are not scored during its **Settle window**. - A **Pilot** falls into exactly one **Performance group**, decided by their **Deviation features** alone. A pilot outside the **Reference cohort** is placed in the nearest group without changing the groups. diff --git a/README.md b/README.md index d38e681..e9c223e 100644 --- a/README.md +++ b/README.md @@ -109,7 +109,7 @@ Then, in order: - A literal IP address and port are required. A host name such as `localhost:5050` is rejected. - Addresses that connect successfully are remembered in the `Saved IPs` dropdown (stored on Windows at `%APPDATA%\PilotDataSynchronization\relay_ips.json`). 4. Press `Connect TCP`. -5. Press `Check TCP Connection Status` to refresh the status line. It does not update by itself. This button is the only thing that changes it. +5. Press `Check TCP Connection Status` to refresh the status line. It is also updated when the connection is made or lost, but this button is the way to refresh it. 6. Start X-Plane. Flight data now flows automatically. > The `Send Packet` button sends one test packet of zeros, for example `E;1;PilotDataSync;;;;;AltitudeSync;0;0`, and shows the time it was sent next to the button. The sample is the first dataref toggle that is enabled, so the packet always matches a `` in the generated `iMotions.xml`. It is a connection check and needs TCP connected, not X-Plane. Avoid pressing it during a recording, since iMotions receives the zeros as a real sample. The live data stream is automatic and does not need this button. @@ -117,7 +117,8 @@ Then, in order: Two more behaviors that catch people out: - The eight dataref toggles do not affect what is sent over TCP. All eight values are always transmitted. The toggles control only which `` entries appear in the generated `iMotions.xml`. -- Losing X-Plane also tears down the TCP connection. After restarting the simulator, press `Connect TCP` again. +- Losing X-Plane also tears down the TCP connection. After restarting the simulator, press `Connect TCP` again, unless the relay was started with `--tcp` (see below), which reconnects by itself. +- A TCP connection made by clicking now shows a drop too. If iMotions or the listener goes away, the event log says `TCP disconnected` and `Connect TCP` works again. Only outputs given on the command line are retried. ### Sending to iMotions and Python at the same time @@ -137,6 +138,23 @@ With no iMotions to record to, skip the mirror and point `Connect TCP` straight `Send Packet` goes to both destinations, which makes it a check of both links. It also means the logger writes a row of zeros, so avoid it mid-recording. +### Starting with the outputs already connected + +Two optional options connect the relay's outputs when it starts, so nobody has to find the right box and button: + +```bash +cd relay +cargo run -- --tcp --mirror 127.0.0.1:5001 # iMotions, and the Python side as the mirror +cargo run -- --tcp 127.0.0.1:5001 # no iMotions: straight to the Python listener +``` + +(With a built relay, the same options follow `relay`.) Addresses are a literal IP and port, as in the boxes. `--mirror` needs `--tcp`, because the mirror copies the TCP stream. `relay --help` prints the usage. + +- Each output given is retried until it connects, and again after it drops, a connection that was up at once. A listener that starts late, or is restarted mid-Flight, is picked up with no clicks, and the relay window says it is waiting for the address and retrying. The relay itself keeps running, because the plugin cannot reconnect to a relay that has restarted (`docs/adr/0005`). +- While the mirror is down the iMotions stream is not held up, and the event log gets one line for the whole run of failed attempts, not one per attempt. +- `Disconnect TCP` stops the retrying, and the mirror's with it. `Disconnect mirror` stops only the mirror's. Pressing `Connect TCP` or `Connect mirror` yourself takes that output over: it is connected as the button always did, and is not retried to the command-line address afterwards. +- Without options nothing changes: the relay waits for you to click. + ## 3. iMotions With the relay running, generate the event-source definition that iMotions needs: diff --git a/docs/adr/0003-the-relay-mirrors-its-stream-rather-than-holding-two.md b/docs/adr/0003-the-relay-mirrors-its-stream-rather-than-holding-two.md index f09f906..3f81f34 100644 --- a/docs/adr/0003-the-relay-mirrors-its-stream-rather-than-holding-two.md +++ b/docs/adr/0003-the-relay-mirrors-its-stream-rather-than-holding-two.md @@ -15,9 +15,9 @@ What is guaranteed: the TCP thread never waits on the mirror. Connecting, writin A mirror falling behind is reported once when it starts dropping and once when it catches up, not per packet, because the event log is capped at 200 entries and 160 packets a second would empty it. -The Python side has to read its socket continuously, on a thread of its own if it also does slow work. A clustering refit that stops reading for long enough will first lose copies and then, once the write timeout expires, lose the mirror, which does not reconnect by itself. +The Python side has to read its socket continuously, on a thread of its own if it also does slow work. A clustering refit that stops reading for long enough will first lose copies and then, once the write timeout expires, lose the mirror. -Nothing reconnects the mirror automatically. If Python restarts, press `Connect mirror` again. +A mirror connected by clicking is not reconnected: if Python restarts, press `Connect mirror` again. A mirror given with `--mirror` on the command line is retried until it connects and again after it drops (#276, docs/adr/0005). `Send Packet` reaches both destinations, so it checks both links at once, and it writes a row of zeros to the logger as well as to iMotions. diff --git a/docs/adr/0004-live-clustering-runs-in-a-python-sidecar.md b/docs/adr/0004-live-clustering-runs-in-a-python-sidecar.md index efaf6fe..9e73a6f 100644 --- a/docs/adr/0004-live-clustering-runs-in-a-python-sidecar.md +++ b/docs/adr/0004-live-clustering-runs-in-a-python-sidecar.md @@ -13,5 +13,5 @@ To update **Performance groups** while a **Flight** is under way, the relay forw - The sidecar reads the relay's **Mirror** (docs/adr/0003), so the relay already forwards to iMotions unchanged when the sidecar is absent or slow. The copy is best effort, so the sidecar must tolerate dropped samples and must keep reading its socket on a thread separate from the slower work, or it loses the mirror. - The return channel (how the sidecar tells the relay a pilot's group) is not decided here. - The groups are fitted once and the live pilot is assigned to the nearest one, rather than refitting with the live pilot included. A refit would renumber the groups between updates, and one live pilot cannot form groups alone (`candidate_ks` needs 6). Until real pilots have **Targets**, the reference cohort is **Synthetic pilots**, so the live group is a demonstration rather than a measurement. -- The client's **Targets** are not available (`docs/telemetry_schema.md` section 3). For our own X-Plane flights we are the instructor: we write a **Brief** of **Instructions**, and the operator marks each one live as it is spoken, so the sidecar knows when every target starts. The targets an instruction changes are not scored during its **Settle window**, so the turn or level-off is not scored, since **Synthetic pilots** fly only steady holds. Targets it leaves alone are scored throughout. +- The client's **Targets** are not available (`docs/telemetry_schema.md` section 3). For our own X-Plane flights we are the **Instructor**: we write a **Brief** of **Instructions** and mark each one live as it is spoken, so the sidecar knows when every target starts. The targets an instruction changes are not scored during its **Settle window**, so the turn or level-off is not scored, since **Synthetic pilots** fly only steady holds. Targets it leaves alone are scored throughout. - If the sidecar is later judged too heavy to run beside the relay, this decision should be revisited, since porting to Rust then means re-validating the clustering against ADR 0002's evidence. diff --git a/docs/adr/0005-the-relay-outlives-the-listener.md b/docs/adr/0005-the-relay-outlives-the-listener.md new file mode 100644 index 0000000..3691fc0 --- /dev/null +++ b/docs/adr/0005-the-relay-outlives-the-listener.md @@ -0,0 +1,16 @@ +# The relay outlives the listener and retries its outputs + +Status: accepted. The relay's side is #276; the launchers and the listeners' resume are planned in #278 and #277. + +The launchers start the **Relay** as its own process, apart from the Python listener (`data_logger` or `live_groups`). They leave it running when the listener stops, and reuse it if it is already running. Given `--tcp` or `--mirror`, the relay keeps retrying that address until it connects, and again after a drop, so a listener started again mid-**Flight** is picked up with no clicks. We chose this because the **Plugin** cannot reconnect to a relay that has restarted (#256, #255): a relay that closed with its listener would cost the **Instructor** an X-Plane restart mid-**Flight**. In the lab, closing the console window is how people stop a listener. + +## Considered Options + +- **One launch is one Flight** (the relay closes when the listener stops). Rejected. It is simpler and every Flight starts clean, but a listener that crashes or has its window closed takes the relay with it, and recovering then means restarting X-Plane or re-enabling the plugin in Plugin Admin. +- **Relay outlives the listener (chosen).** A restarted listener costs nothing in X-Plane. The relay is closed by hand, for example at the end of the day. + +## Consequences + +- A relay that is already running keeps the addresses it started with. The launcher does not ask for an iMotions address in that case; it says so, and changing the address means closing the relay. +- Restarting a listener mid-flight must keep the same `flight_id`, or one **Flight** splits into two. The launcher offers to resume when the last Flight did not end with `q`, and `live_groups` then continues the **Brief** from the last **Instruction** given. +- A crash of the relay itself still forces a plugin restart until #256 and #255 are fixed. Once they are, this decision still stands, since it avoids reconnecting at all. diff --git a/inference/pilot_inference/performance/live.py b/inference/pilot_inference/performance/live.py index 938434b..ff73c69 100644 --- a/inference/pilot_inference/performance/live.py +++ b/inference/pilot_inference/performance/live.py @@ -1,9 +1,9 @@ """ Deviation features for one flight while it is being flown (docs/adr/0004). -For flights we fly ourselves we are the instructor, so we know the targets: a -brief lists the instructions with the targets each one sets, and the operator -marks each one as it is spoken. Targets an instruction does not name carry +For flights we fly ourselves we are the Instructor, so we know the targets: a +brief lists the instructions with the targets each one sets, and we mark each +one as it is spoken. Targets an instruction does not name carry over. LiveFlight attaches the targets in force to every sample, leaving the ones an instruction changed blank for a settle window so the turn or level-off is not scored, and summarises the scored samples with the same functions the diff --git a/inference/pilot_inference/performance/live_groups.py b/inference/pilot_inference/performance/live_groups.py index 0f914f4..28a262f 100644 --- a/inference/pilot_inference/performance/live_groups.py +++ b/inference/pilot_inference/performance/live_groups.py @@ -2,7 +2,7 @@ Show a pilot's performance group while they fly (docs/adr/0004). Listens for the relay's mirror (or Connect TCP), assembles samples the way the -capture logger does, attaches the brief's targets as the operator marks each +capture logger does, attaches the brief's targets as the Instructor marks each instruction, and writes every sample to CSV. Every few seconds it prints the pilot's deviation features and the nearest group of a reference cohort. The cohort is synthetic until real pilots have targets, so the group demonstrates @@ -184,15 +184,18 @@ def _serve(self) -> None: pass except (OSError, ValueError) as error: # Garbled bytes or a socket error end this connection, not the - # listener: the mirror never reconnects by itself (docs/adr/0003), - # so the next Connect mirror must still find someone listening. + # listener: the relay may reconnect (with --mirror it does so by + # itself, docs/adr/0005), and must still find someone listening. if not self._closed: print(f"Relay connection failed: {error}") with self._lock: self._conn = None if self._closed: return - print("Relay disconnected. Press Connect mirror again to resume.") + print( + "Relay disconnected. A relay started with --mirror reconnects by itself; " + "otherwise press Connect mirror again." + ) def _cell(value) -> str: diff --git a/relay/src/cli.rs b/relay/src/cli.rs new file mode 100644 index 0000000..5bdf29b --- /dev/null +++ b/relay/src/cli.rs @@ -0,0 +1,178 @@ +//! Command-line options (#276). Without any, the relay starts exactly as it +//! always has and waits for clicks. + +use anyhow::{anyhow, bail, Result}; + +use crate::state::{State, IMOTIONS_ADDR_HINT, MIRROR_ADDR_HINT}; + +/// What `relay --help` prints, and what follows a rejected command line. +pub(crate) const USAGE: &str = "Usage: relay [--tcp ] [--mirror ] + +Without options the relay waits for you to connect from its window. + +Options: + --tcp Connect to iMotions (or, with no iMotions, to the Python + listener) on start-up, and keep retrying until it + connects and again after it drops. + --mirror Also copy every packet to the Python listener, with the + same retrying. Needs --tcp. + -h, --help Show this text. + +Addresses are a literal IP and port, for example 127.0.0.1:5001."; + +/// Returned by `parse_args` when the command line asks for the usage text. It +/// is not a mistake: the caller prints `USAGE` and exits successfully. +#[derive(Debug)] +pub(crate) struct HelpRequested; + +impl std::fmt::Display for HelpRequested { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "help requested") + } +} + +impl std::error::Error for HelpRequested {} + +/// Outputs to connect on start-up and keep connected, retrying until each one +/// connects and again after it drops (docs/adr/0005). +#[derive(Debug, Default, Clone, PartialEq, Eq)] +pub(crate) struct StartupConnections { + /// `--tcp `: iMotions, or the Python listener when there is no + /// iMotions. + pub tcp: Option, + /// `--mirror `: the Python listener, while `--tcp` goes to + /// iMotions. + pub mirror: Option, +} + +/// Parse the arguments after the program name. +/// +/// `--help` and `-h` give `Err(HelpRequested)`, which the caller recognises +/// with `downcast_ref`. +pub(crate) fn parse_args(args: impl IntoIterator) -> Result { + let mut connections = StartupConnections::default(); + let mut args = args.into_iter(); + while let Some(arg) = args.next() { + let (slot, hint) = match arg.as_str() { + "--help" | "-h" => return Err(HelpRequested.into()), + "--tcp" => (&mut connections.tcp, IMOTIONS_ADDR_HINT), + "--mirror" => (&mut connections.mirror, MIRROR_ADDR_HINT), + other if other.starts_with('-') => bail!("Unknown option {other}"), + other => bail!("Unexpected argument {other:?}. Addresses go after --tcp or --mirror"), + }; + if slot.is_some() { + bail!("{arg} was given more than once"); + } + // An address never starts with a dash, so another option here means + // the address was left out. + let value = match args.next() { + Some(value) if !value.trim_start().starts_with('-') => value, + _ => bail!("{arg} needs an address, for example {hint}"), + }; + let value = value.trim(); + State::validate_tcp_addr(value, hint).map_err(|e| anyhow!("{arg}: {e}"))?; + *slot = Some(value.to_string()); + } + if connections.mirror.is_some() && connections.tcp.is_none() { + bail!("--mirror needs --tcp too: the mirror copies the TCP stream"); + } + Ok(connections) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn parse(args: &[&str]) -> Result { + parse_args(args.iter().map(|a| a.to_string())) + } + + #[test] + fn no_options_connect_nothing() { + assert_eq!(parse(&[]).expect("no options failed"), StartupConnections::default()); + } + + #[test] + fn tcp_alone_is_the_no_imotions_case() { + assert_eq!( + parse(&["--tcp", "127.0.0.1:5001"]).expect("parse failed"), + StartupConnections { + tcp: Some("127.0.0.1:5001".into()), + mirror: None, + } + ); + } + + #[test] + fn tcp_and_mirror_together_is_the_imotions_case() { + assert_eq!( + parse(&["--tcp", "192.168.1.20:8089", "--mirror", "127.0.0.1:5001"]) + .expect("parse failed"), + StartupConnections { + tcp: Some("192.168.1.20:8089".into()), + mirror: Some("127.0.0.1:5001".into()), + } + ); + } + + #[test] + fn the_order_of_the_options_does_not_matter() { + assert_eq!( + parse(&["--mirror", "127.0.0.1:5001", "--tcp", "192.168.1.20:8089"]) + .expect("parse failed"), + StartupConnections { + tcp: Some("192.168.1.20:8089".into()), + mirror: Some("127.0.0.1:5001".into()), + } + ); + } + + #[test] + fn stray_whitespace_around_an_address_is_trimmed() { + // A .cmd file that builds the command line from a prompt can leave it. + assert_eq!( + parse(&["--tcp", " 127.0.0.1:5001 "]).expect("parse failed").tcp, + Some("127.0.0.1:5001".into()) + ); + } + + #[test] + fn a_mirror_without_tcp_is_refused() { + // The mirror copies the TCP stream, so it has nothing to copy alone. + let error = parse(&["--mirror", "127.0.0.1:5001"]).expect_err("accepted a lone mirror"); + assert!(error.to_string().contains("--tcp"), "unhelpful error: {error}"); + } + + #[test] + fn an_unknown_option_is_refused_and_named() { + let error = parse(&["--port", "5001"]).expect_err("accepted --port"); + assert!(error.to_string().contains("--port"), "unhelpful error: {error}"); + } + + #[test] + fn an_option_without_its_address_is_refused() { + let error = parse(&["--tcp"]).expect_err("accepted --tcp with no address"); + assert!(error.to_string().contains("--tcp"), "unhelpful error: {error}"); + } + + #[test] + fn an_option_followed_by_another_option_is_missing_its_address() { + let error = + parse(&["--tcp", "--mirror", "127.0.0.1:5001"]).expect_err("took --mirror as an address"); + assert!(error.to_string().contains("--tcp"), "unhelpful error: {error}"); + } + + #[test] + fn a_hostname_is_refused_like_in_the_address_box() { + // The address boxes take a literal IP and port; the options match them. + assert!(parse(&["--tcp", "localhost:5001"]).is_err()); + assert!(parse(&["--tcp", "127.0.0.1"]).is_err(), "accepted an address with no port"); + } + + #[test] + fn giving_the_same_option_twice_is_refused() { + let error = parse(&["--tcp", "127.0.0.1:5001", "--tcp", "127.0.0.1:5002"]) + .expect_err("accepted two --tcp"); + assert!(error.to_string().contains("--tcp"), "unhelpful error: {error}"); + } +} diff --git a/relay/src/main.rs b/relay/src/main.rs index 26a6a8b..d072763 100644 --- a/relay/src/main.rs +++ b/relay/src/main.rs @@ -1,4 +1,5 @@ mod bichannel; +mod cli; mod message; mod state; mod update; @@ -17,6 +18,20 @@ use iced::{ }; fn main() -> iced::Result { + // Before iced starts, so a mistake is reported in the console rather than + // after a window has opened. + let outputs = match cli::parse_args(std::env::args().skip(1)) { + Ok(outputs) => outputs, + Err(e) if e.downcast_ref::().is_some() => { + println!("{}", cli::USAGE); + std::process::exit(0); + } + Err(e) => { + eprintln!("relay: {e}\n\n{}", cli::USAGE); + std::process::exit(2); + } + }; + iced::application("RELAY", update, view) .window_size((450.0, 300.0)) .exit_on_close_request(false) @@ -36,6 +51,9 @@ fn main() -> iced::Result { "Error connecting to IPC during GUI initialization: {e:?}" )); }; + + // Nothing happens here when no options were given. + state.keep_connected(outputs); (state, Task::none()) }) } diff --git a/relay/src/message.rs b/relay/src/message.rs index 372f17e..2c39b65 100644 --- a/relay/src/message.rs +++ b/relay/src/message.rs @@ -67,8 +67,8 @@ pub(crate) enum ToTcpThreadMessage { } pub(crate) enum FromTcpThreadMessage { - /// TCP thread successfully connected to remote - Connected, + /// TCP thread successfully connected to remote (its address) + Connected(String), /// TCP thread disconnected (may include reason) Disconnected(String), /// A packet was sent to remote (In bytes) diff --git a/relay/src/state.rs b/relay/src/state.rs index ba8e533..c759c1a 100644 --- a/relay/src/state.rs +++ b/relay/src/state.rs @@ -53,6 +53,10 @@ pub(crate) const MIRROR_ADDR_HINT: &str = "127.0.0.1:5001"; /// cost the iMotions stream anything. They exist so a Python process that is /// gone or wedged is given up on rather than held open forever. const MIRROR_CONNECT_TIMEOUT: StdDuration = StdDuration::from_millis(500); + +/// How long an output given on the command line waits between attempts to +/// connect, at start-up and after a drop (#276). +pub(crate) const RECONNECT_INTERVAL: StdDuration = StdDuration::from_millis(500); const MIRROR_WRITE_TIMEOUT: StdDuration = StdDuration::from_millis(100); /// How many packets may wait for the mirror's writer thread. @@ -81,6 +85,60 @@ const MIRROR_QUEUE_CAPACITY: usize = 256; const MIRROR_SEND_BUFFER: usize = 32 * 1024; // --- State definition ------------------------------------------------------- +/// What the relay keeps trying to connect for one output given on the command +/// line (#276, docs/adr/0005). +#[derive(Debug, Default)] +pub(crate) struct Retry { + /// The address given on the command line. `None` once it was never given, + /// Disconnect was pressed, or the Instructor connected by hand instead. + pub target: Option, + /// Not before this instant. `None` means an attempt may start now: a + /// connection that was up has just been lost, or none has been tried yet. + next_attempt: Option, + /// Whether the event log already has the line for the current run of + /// failed attempts, so a retry loop adds one line, not one per attempt. + noted: bool, +} + +impl Retry { + fn with_target(target: Option) -> Retry { + Retry { + target, + ..Retry::default() + } + } + + fn is_due(&self, now: Instant) -> bool { + self.target.is_some() && self.next_attempt.map_or(true, |at| now >= at) + } + + /// An attempt failed, or the output is still not up: wait before the next. + fn retry_later(&mut self) { + self.next_attempt = Some(Instant::now() + RECONNECT_INTERVAL); + } + + /// A connection that was up has been lost, or an attempt has just started: + /// the next attempt may start at once. + fn retry_now(&mut self) { + self.next_attempt = None; + } + + /// True the first time it is called in a run of failures, so the caller + /// logs once. + fn note_once(&mut self) -> bool { + !std::mem::replace(&mut self.noted, true) + } + + fn restart(&mut self) { + self.noted = false; + self.next_attempt = None; + } + + fn clear(&mut self) { + *self = Retry::default(); + } +} + #[allow(unused)] pub(crate) struct State { pub elapsed_time: Duration, @@ -127,6 +185,13 @@ pub(crate) struct State { // Mirror destination: the Python side reads the same packets iMotions gets pub mirror_addr_field: String, pub mirror_connected: bool, + + // Outputs given on the command line, which the relay keeps connected + // (#276). A `Retry` with no target is only ever connected by clicking. + pub tcp_retry: Retry, + pub mirror_retry: Retry, + // A SetMirror is out and its outcome has not come back yet. + pub mirror_attempt_in_flight: bool, } impl Default for State { @@ -170,6 +235,10 @@ impl Default for State { mirror_addr_field: String::new(), mirror_connected: false, + + tcp_retry: Retry::default(), + mirror_retry: Retry::default(), + mirror_attempt_in_flight: false, } } } @@ -486,6 +555,10 @@ fn mirror_send( return; }; match active.packets.try_send(packet.to_string()) { + // The writer thread has gone: the mirror dropped or never connected, + // and its Disconnected event is on the way. That is not lag, so say + // nothing, and the event clears the mirror. + Err(crossbeam::channel::TrySendError::Disconnected(_)) => {} Ok(()) => { if active.lagging { let note = format!( @@ -498,7 +571,7 @@ fn mirror_send( active.dropped = 0; } } - Err(_) => { + Err(crossbeam::channel::TrySendError::Full(_)) => { active.dropped += 1; // One line per episode. At 160 packets a second, reporting each // drop would push everything else out of the capped event log. @@ -556,6 +629,19 @@ fn send_packet_and_debug(stream: &mut TcpStream, packet: &str) -> Result<()> { Ok(()) } +/// Report that a write to the TCP peer failed, so the GUI shows the drop and +/// can reap this thread. Returns the error for the thread to exit with. +fn tcp_link_lost( + child_bichannel: &mut ChildBiChannel, + error: anyhow::Error, +) -> Result<()> { + let reason = format!("TCP connection lost: {}", error); + human_log("TCP", &reason); + let _ = child_bichannel.set_is_conn_to_endpoint(false); + let _ = child_bichannel.send_to_parent(FromTcpThreadMessage::Disconnected(reason.clone())); + bail!(reason) +} + // --- State impl ------------------------------------------------------------- impl State { pub fn log_event(&mut self, event: String) { @@ -858,10 +944,15 @@ impl State { self.tcp_bichannel = Some(tcp_bichannel); let tcp_thread_handle = spawn(move || { - let mut stream = match TcpStream::connect(address) { + let mut stream = match TcpStream::connect(&address) { + // Disconnect TCP was pressed while the connect was under way: + // the connection is not wanted, so drop it and go quietly. + Ok(_) if child_bichannel.is_killswitch_engaged() => return Ok(()), Ok(stream) => { - human_log("TCP", "Successfully connected to iMotions server."); + human_log("TCP", &format!("Successfully connected to {}.", address)); let _ = child_bichannel.set_is_conn_to_endpoint(true); + let _ = child_bichannel + .send_to_parent(FromTcpThreadMessage::Connected(address.clone())); stream } Err(e) => { @@ -897,16 +988,18 @@ impl State { } for packet in packets_for_fields(&fields) { - let res = send_packet_and_debug(&mut stream, &packet); - let _ = child_bichannel.set_is_conn_to_endpoint(res.is_ok()); + // The first write after the peer closes can + // still succeed; the ones after it fail, and + // that is how a drop is noticed. + if let Err(e) = send_packet_and_debug(&mut stream, &packet) { + return tcp_link_lost(&mut child_bichannel, e); + } mirror_send(&mut mirror, &packet, &mut child_bichannel); } } ToTcpThreadMessage::SendRaw(packet) => { - let res = send_packet_and_debug(&mut stream, &packet); - let _ = child_bichannel.set_is_conn_to_endpoint(res.is_ok()); - if let Err(e) = res { - human_log("TCP", &format!("Test packet failed: {}", e)); + if let Err(e) = send_packet_and_debug(&mut stream, &packet) { + return tcp_link_lost(&mut child_bichannel, e); } // The mirror gets the test packet too, so the button // checks both links at once. @@ -944,6 +1037,193 @@ impl State { Ok(()) } + /// Connect the outputs given on the command line, and keep them connected: + /// each one is retried until it connects, and again after it drops, driven + /// by `Message::Update` (#276, docs/adr/0005). Pressing Disconnect TCP or + /// Disconnect mirror, or connecting that output by hand, stops the retrying + /// for it. With no outputs given this does nothing. + pub fn keep_connected(&mut self, outputs: crate::cli::StartupConnections) { + self.tcp_retry = Retry::with_target(outputs.tcp); + self.mirror_retry = Retry::with_target(outputs.mirror); + self.mirror_attempt_in_flight = false; + + // Show the addresses in the boxes, so Connect TCP and Connect mirror + // work on them if retrying is stopped. + if let Some(address) = self.tcp_retry.target.clone() { + self.tcp_addr_field = address.clone(); + if let Err(e) = self.tcp_connect(address) { + self.log_event(format!("TCP connect failed: {}", e)); + } + } + if let Some(address) = self.mirror_retry.target.clone() { + self.mirror_addr_field = address; + } + } + + /// The command-line TCP address, while it is not connected and so is being + /// waited for. + pub fn tcp_waiting_for(&self) -> Option<&str> { + self.tcp_retry + .target + .as_deref() + .filter(|_| !self.is_tcp_connected()) + } + + /// The command-line mirror address, while it is not connected and so is + /// being waited for. + pub fn mirror_waiting_for(&self) -> Option<&str> { + self.mirror_retry + .target + .as_deref() + .filter(|_| !self.mirror_connected) + } + + /// The Instructor pressed Connect TCP: they have taken the output over by + /// hand, so a later drop is not retried to the command-line address. + pub fn take_over_tcp(&mut self) { + self.tcp_retry.clear(); + } + + /// The Instructor pressed Connect mirror. See `take_over_tcp`. + pub fn take_over_mirror(&mut self) { + self.mirror_retry.clear(); + self.mirror_attempt_in_flight = false; + } + + /// Disconnect TCP: stop retrying, the mirror's too since it rides on the + /// TCP connection, and drop the connection if there is one. Works between + /// attempts, when there is no thread to disconnect. + pub fn press_disconnect_tcp(&mut self) { + let was_retrying = self.tcp_retry.target.is_some(); + self.tcp_retry.clear(); + self.mirror_retry.clear(); + self.mirror_attempt_in_flight = false; + if was_retrying { + self.log_event("No longer retrying TCP or the mirror".into()); + } + if self.tcp_thread_handle.is_some() { + if let Err(e) = self.tcp_disconnect() { + self.log_event(format!("TCP disconnect failed: {}", e)); + } + } else if !was_retrying { + self.log_event("TCP disconnect failed: TCP thread does not exist.".into()); + } + } + + /// Disconnect mirror: stop retrying it and close it. TCP is untouched. + pub fn press_disconnect_mirror(&mut self) { + if self.mirror_retry.target.is_some() { + self.log_event("No longer retrying the Python mirror".into()); + } + self.mirror_retry.clear(); + self.mirror_attempt_in_flight = false; + if let Err(e) = self.mirror_disconnect() { + self.log_event(format!("Python mirror disconnect failed: {}", e)); + } + } + + /// The TCP thread connected. + pub fn on_tcp_connected(&mut self, address: String) { + self.log_event(format!("TCP connected to {}", address)); + self.tcp_connected = true; + self.tcp_retry.restart(); + } + + /// The TCP thread reported that it failed to connect or lost its + /// connection, and has exited. + /// + /// With a command-line target the run of failures gets one log line and no + /// error banner, since the status line already says the relay is waiting. + /// A connection that was up is retried at once, so iMotions loses as little + /// as possible; failed attempts wait `RECONNECT_INTERVAL` between them. + pub fn on_tcp_lost(&mut self, reason: String) { + if self.tcp_retry.target.is_some() { + if self.tcp_connected { + self.tcp_retry.retry_now(); + } else { + self.tcp_retry.retry_later(); + } + if self.tcp_retry.note_once() { + self.log_event(format!( + "TCP disconnected: {}. Retrying until it connects", + reason + )); + } + } else { + self.log_event(format!("TCP disconnected: {}", reason)); + self.error_message = Some(reason); + } + // The thread has exited. Clear it so Connect TCP works again without + // pressing Disconnect TCP first. + self.reap_tcp_thread(); + } + + /// The mirror's writer thread connected. + pub fn on_mirror_connected(&mut self, address: String) { + self.log_event(format!("Python mirror connected to {}", address)); + self.mirror_connected = true; + self.mirror_attempt_in_flight = false; + self.mirror_retry.restart(); + } + + /// The mirror failed to connect, dropped, or was closed. Retried like the + /// TCP output, without the error banner, which the mirror never had. + pub fn on_mirror_lost(&mut self, reason: String) { + if self.mirror_retry.target.is_some() { + if self.mirror_connected { + self.mirror_retry.retry_now(); + } else { + self.mirror_retry.retry_later(); + } + if self.mirror_retry.note_once() { + self.log_event(format!( + "Python mirror stopped: {}. Retrying until it connects", + reason + )); + } + } else { + self.log_event(format!("Python mirror stopped: {}", reason)); + } + self.mirror_connected = false; + self.mirror_attempt_in_flight = false; + } + + /// Start whichever retries are due. Called on every `Message::Update`. + /// + /// The TCP retry is a new `tcp_connect`, started when no TCP thread exists. + /// The mirror retry is a new `SetMirror` to the live TCP thread. Between + /// attempts the TCP thread holds no mirror at all, so nothing is queued for + /// a mirror that is down and nothing reports it as lagging, and each + /// attempt connects on its own writer thread, so the iMotions stream is + /// never held up (docs/adr/0003). + pub fn retry_outputs(&mut self) { + let now = Instant::now(); + + if self.tcp_thread_handle.is_none() && self.tcp_retry.is_due(now) { + if let Some(address) = self.tcp_retry.target.clone() { + self.tcp_retry.retry_now(); + if let Err(e) = self.tcp_connect(address) { + self.log_event(format!("TCP connect failed: {}", e)); + } + } + } + + if self.mirror_retry.is_due(now) + && !self.mirror_connected + && !self.mirror_attempt_in_flight + && self.is_tcp_connected() + { + if let (Some(address), Some(bichannel)) = + (self.mirror_retry.target.clone(), self.tcp_bichannel.as_mut()) + { + match bichannel.send_to_child(ToTcpThreadMessage::SetMirror(Some(address))) { + Ok(()) => self.mirror_attempt_in_flight = true, + Err(_) => self.mirror_retry.retry_later(), + } + } + } + } + /// Point the mirror at `address`, so the Python side receives a copy of /// every packet (#245). /// @@ -984,6 +1264,8 @@ impl State { // The mirror lives in that thread, so it goes down with it. Without // this the GUI keeps claiming the mirror is connected. self.mirror_connected = false; + self.tcp_connected = false; + self.reset_mirror_attempts(); if !was_connected { spawn(move || { @@ -1007,8 +1289,16 @@ impl State { } self.tcp_bichannel = None; self.tcp_connected = false; - // The mirror lived in that thread, so it is gone with it. + // The mirror lived in that thread, so it is gone with it. A new TCP + // thread gets a fresh mirror attempt. self.mirror_connected = false; + self.reset_mirror_attempts(); + } + + /// A new TCP thread gets a fresh mirror attempt, at once. + fn reset_mirror_attempts(&mut self) { + self.mirror_attempt_in_flight = false; + self.mirror_retry.restart(); } pub fn is_tcp_connected(&self) -> bool { diff --git a/relay/src/update.rs b/relay/src/update.rs index 02fe298..e4176b8 100644 --- a/relay/src/update.rs +++ b/relay/src/update.rs @@ -11,6 +11,8 @@ use crate::{ }; fn connect_tcp_with_validation_and_save(state: &mut State, address: String) { + // Connecting by hand takes the output over from the command line. + state.take_over_tcp(); let trimmed = address.trim().to_string(); match State::validate_tcp_addr(&trimmed, IMOTIONS_ADDR_HINT) { Ok(()) => { @@ -74,17 +76,11 @@ pub(crate) fn update(state: &mut State, message: Message) -> Task { .unwrap_or_default(); for msg in tcp_messages { match msg { - FromTcpThreadMessage::Connected => { - state.log_event("TCP connected to iMotions".into()); - state.tcp_connected = true; + FromTcpThreadMessage::Connected(address) => { + state.on_tcp_connected(address); } FromTcpThreadMessage::Disconnected(reason) => { - state.log_event(format!("TCP disconnected: {}", reason)); - state.error_message = Some(reason); - state.tcp_connected = false; - // The thread has exited. Clear it so Connect TCP works - // again without pressing Disconnect TCP first. - state.reap_tcp_thread(); + state.on_tcp_lost(reason); } FromTcpThreadMessage::Sent(bytes) => { state.on_tcp_packet_sent(bytes); @@ -93,12 +89,10 @@ pub(crate) fn update(state: &mut State, message: Message) -> Task { state.log_event(format!("TCP send error: {}", err)); } FromTcpThreadMessage::MirrorConnected(address) => { - state.log_event(format!("Python mirror connected to {}", address)); - state.mirror_connected = true; + state.on_mirror_connected(address); } FromTcpThreadMessage::MirrorDisconnected(reason) => { - state.log_event(format!("Python mirror stopped: {}", reason)); - state.mirror_connected = false; + state.on_mirror_lost(reason); } FromTcpThreadMessage::MirrorLagging(reason) => { state.log_event(reason); @@ -106,6 +100,8 @@ pub(crate) fn update(state: &mut State, message: Message) -> Task { } } + state.retry_outputs(); + Task::none() } M::WindowCloseRequest(id) => { @@ -149,9 +145,7 @@ pub(crate) fn update(state: &mut State, message: Message) -> Task { Task::none() } M::DisconnectTcp => { - if let Err(e) = state.tcp_disconnect() { - state.log_event(format!("TCP disconnect failed: {}", e)); - } + state.press_disconnect_tcp(); Task::none() } M::AltitudeToggle(value) => { @@ -205,6 +199,7 @@ pub(crate) fn update(state: &mut State, message: Message) -> Task { Task::none() } M::ConnectMirror => { + state.take_over_mirror(); let address = state.mirror_addr_field.clone(); if let Err(e) = state.mirror_connect(address) { let msg = format!("Python mirror: {}", e); @@ -214,9 +209,7 @@ pub(crate) fn update(state: &mut State, message: Message) -> Task { Task::none() } M::DisconnectMirror => { - if let Err(e) = state.mirror_disconnect() { - state.log_event(format!("Python mirror disconnect failed: {}", e)); - } + state.press_disconnect_mirror(); Task::none() } M::SendPacket => { @@ -1095,4 +1088,345 @@ mod tests { ); assert!(state.last_send_timestamp.is_none()); } + + // --- #276: outputs given on the command line stay connected ------------- + + use crate::cli::StartupConnections; + use crate::state::RECONNECT_INTERVAL; + use std::io::ErrorKind; + + fn tcp_only(addr: &str) -> StartupConnections { + StartupConnections { + tcp: Some(addr.to_string()), + mirror: None, + } + } + + fn tcp_and_mirror(tcp: &str, mirror: &str) -> StartupConnections { + StartupConnections { + tcp: Some(tcp.to_string()), + mirror: Some(mirror.to_string()), + } + } + + // Sends one sample if a TCP thread exists to take it, as baton data would + // be forwarded, and does nothing while the relay is between connections. + fn offer_sample(state: &mut State, altitude: &str) { + if let Some(bichannel) = state.tcp_bichannel.as_mut() { + let _ = bichannel.send_to_child(ToTcpThreadMessage::Send(format!( + "{altitude};120;90;500;2;3;91;1.0" + ))); + } + } + + // Pumps updates, with samples flowing, until `listener` accepts a + // connection from it. Telemetry keeps arriving during a real flight, and a + // dropped peer is only noticed when a write to it fails. + fn pump_until_accepted(state: &mut State, listener: &TcpListener) -> Option { + listener.set_nonblocking(true).expect("set_nonblocking failed"); + for i in 0..1000 { + offer_sample(state, &format!("{}", 100 + i)); + let _ = update(state, Message::Update); + match listener.accept() { + Ok((stream, _)) => { + stream.set_nonblocking(false).expect("set_nonblocking failed"); + return Some(stream); + } + Err(e) if e.kind() == ErrorKind::WouldBlock => sleep(StdDuration::from_millis(10)), + Err(e) => panic!("accept failed: {e}"), + } + } + None + } + + // Pumps updates with samples flowing until `condition` holds. + fn pump_with_samples_until(state: &mut State, mut condition: impl FnMut(&State) -> bool) -> bool { + for i in 0..1000 { + offer_sample(state, &format!("{}", 500 + i)); + let _ = update(state, Message::Update); + if condition(state) { + return true; + } + sleep(StdDuration::from_millis(10)); + } + false + } + + // Pumps updates for several retry intervals and reports whether anything connected + // to `listener` in that time. + fn anything_connects_within_a_few_retries(state: &mut State, listener: &TcpListener) -> bool { + listener.set_nonblocking(true).expect("set_nonblocking failed"); + let deadline = Instant::now() + RECONNECT_INTERVAL * 4; + while Instant::now() < deadline { + offer_sample(state, "1"); + let _ = update(state, Message::Update); + match listener.accept() { + Ok(_) => return true, + Err(e) if e.kind() == ErrorKind::WouldBlock => sleep(StdDuration::from_millis(10)), + Err(e) => panic!("accept failed: {e}"), + } + } + false + } + + // Pumps updates long enough for a connect attempt already under way to + // finish. On Windows a refused connect to a closed port is only reported + // after about two seconds, during which a listener that appears can still + // accept it. + fn let_attempts_in_flight_finish(state: &mut State) { + let deadline = Instant::now() + StdDuration::from_secs(3); + while Instant::now() < deadline { + let _ = update(state, Message::Update); + sleep(StdDuration::from_millis(10)); + } + } + + // Reads lines until one contains `marker`, so packets sent before it (or + // lost while reconnecting) do not matter. + fn read_until_contains(stream: &TcpStream, marker: &str) -> Option { + stream + .set_read_timeout(Some(StdDuration::from_secs(5))) + .expect("set_read_timeout failed"); + let mut reader = BufReader::new(stream); + for _ in 0..2000 { + let mut line = String::new(); + match reader.read_line(&mut line) { + Ok(0) | Err(_) => return None, + Ok(_) if line.contains(marker) => return Some(line), + Ok(_) => {} + } + } + None + } + + #[test] + fn no_startup_outputs_leaves_the_relay_waiting_for_clicks() { + let mut state = State::default(); + state.keep_connected(StartupConnections::default()); + pump_update(&mut state, 5); + assert!(state.tcp_thread_handle.is_none(), "connected with no --tcp"); + assert!(state.tcp_bichannel.is_none()); + assert!(!state.mirror_connected); + } + + // The launcher starts the relay before the listener may be ready. + #[test] + fn a_tcp_output_that_starts_later_is_connected_without_clicks() { + let addr = closed_port_addr(); + let mut state = State::default(); + state.keep_connected(tcp_only(&addr)); + + // At least one failed attempt first. The window says it is waiting, so + // the error banner is not flashed on for each failed attempt. + assert!( + pump_until(&mut state, |s| s.event_log.iter().any(|e| e.contains("Retrying"))), + "no line in the event log says the relay is retrying" + ); + assert!(!state.is_tcp_connected()); + let deadline = Instant::now() + RECONNECT_INTERVAL * 8; + while Instant::now() < deadline { + let _ = update(&mut state, Message::Update); + assert_eq!( + state.error_message, None, + "the error banner shows each failed attempt while retrying" + ); + sleep(StdDuration::from_millis(10)); + } + + let listener = TcpListener::bind(&addr).expect("bind failed"); + let stream = pump_until_accepted(&mut state, &listener).expect("relay never connected"); + assert!( + pump_until(&mut state, |s| s.is_tcp_connected()), + "connection not shown as connected" + ); + + offer_sample(&mut state, "4321"); + assert!( + read_until_contains(&stream, "AltitudeSync;4321;4321").is_some(), + "the listener never received the sample" + ); + + let _ = state.tcp_disconnect(); + } + + // The main case for the launchers: the listener on --tcp is restarted + // mid-flight, and the relay must pick it up without touching X-Plane. + #[test] + fn a_tcp_output_reconnects_after_the_listener_restarts() { + let (listener, addr) = listener_on_free_port(); + let mut state = State::default(); + state.keep_connected(tcp_only(&addr)); + + let first = pump_until_accepted(&mut state, &listener).expect("relay never connected"); + assert!(pump_until(&mut state, |s| s.is_tcp_connected())); + + // The listener goes away mid-flight. + drop(first); + drop(listener); + assert!( + pump_with_samples_until(&mut state, |s| !s.is_tcp_connected()), + "relay never noticed the listener had gone" + ); + + // The listener comes back on the same address. + let listener = TcpListener::bind(&addr).expect("rebind failed"); + let second = pump_until_accepted(&mut state, &listener).expect("relay never reconnected"); + assert!(pump_until(&mut state, |s| s.is_tcp_connected())); + + offer_sample(&mut state, "8765"); + assert!( + read_until_contains(&second, "AltitudeSync;8765;8765").is_some(), + "the restarted listener never received the sample" + ); + + let _ = state.tcp_disconnect(); + } + + // With iMotions on --tcp and the listener on --mirror: the listener can + // start late, and iMotions is never held up while the mirror waits + // (docs/adr/0003). + #[test] + fn a_mirror_output_that_starts_later_gets_copies_and_imotions_never_waits() { + let (imotions, imotions_addr) = listener_on_free_port(); + let python_addr = closed_port_addr(); + let mut state = State::default(); + state.keep_connected(tcp_and_mirror(&imotions_addr, &python_addr)); + + let imotions_stream = + pump_until_accepted(&mut state, &imotions).expect("iMotions never connected"); + assert!(pump_until(&mut state, |s| s.is_tcp_connected())); + + // The mirror is still retrying; iMotions gets every sample regardless. + pump_update(&mut state, 30); + assert!(!state.mirror_connected); + offer_sample(&mut state, "1111"); + assert!( + read_until_contains(&imotions_stream, "AltitudeSync;1111;1111").is_some(), + "iMotions was held up by a mirror that is not there" + ); + + let python = TcpListener::bind(&python_addr).expect("bind failed"); + let python_stream = + pump_until_accepted(&mut state, &python).expect("the mirror never connected"); + assert!( + pump_until(&mut state, |s| s.mirror_connected), + "mirror never shown as connected" + ); + + offer_sample(&mut state, "2222"); + assert!(read_until_contains(&python_stream, "AltitudeSync;2222;2222").is_some()); + assert!(read_until_contains(&imotions_stream, "AltitudeSync;2222;2222").is_some()); + + let _ = state.tcp_disconnect(); + } + + #[test] + fn a_mirror_output_reconnects_after_python_restarts_mid_flight() { + let (imotions, imotions_addr) = listener_on_free_port(); + let (python, python_addr) = listener_on_free_port(); + let mut state = State::default(); + state.keep_connected(tcp_and_mirror(&imotions_addr, &python_addr)); + + let imotions_stream = + pump_until_accepted(&mut state, &imotions).expect("iMotions never connected"); + let first = pump_until_accepted(&mut state, &python).expect("the mirror never connected"); + assert!(pump_until(&mut state, |s| s.mirror_connected)); + + // Python goes away mid-flight: closed console window, or a crash. + drop(first); + drop(python); + assert!( + pump_with_samples_until(&mut state, |s| !s.mirror_connected), + "relay never noticed Python had gone" + ); + assert!(state.is_tcp_connected(), "iMotions went down with the mirror"); + + // Python is started again on the same address. No clicks. + let python = TcpListener::bind(&python_addr).expect("rebind failed"); + let second = pump_until_accepted(&mut state, &python).expect("the mirror never came back"); + assert!(pump_until(&mut state, |s| s.mirror_connected)); + + offer_sample(&mut state, "3333"); + assert!(read_until_contains(&second, "AltitudeSync;3333;3333").is_some()); + assert!( + read_until_contains(&imotions_stream, "AltitudeSync;3333;3333").is_some(), + "iMotions stopped receiving while the mirror reconnected" + ); + + let _ = state.tcp_disconnect(); + } + + // Retrying must not fight the Instructor: Disconnect means stop. + #[test] + fn pressing_disconnect_tcp_stops_the_retrying() { + let addr = closed_port_addr(); + let mut state = State::default(); + state.keep_connected(tcp_only(&addr)); + pump_update(&mut state, 30); + + let _ = update(&mut state, Message::DisconnectTcp); + + // An attempt already under way when Disconnect was pressed may still + // complete; what must stop is starting new ones. + let_attempts_in_flight_finish(&mut state); + let listener = TcpListener::bind(&addr).expect("bind failed"); + assert!( + !anything_connects_within_a_few_retries(&mut state, &listener), + "the relay kept retrying after Disconnect TCP" + ); + } + + #[test] + fn pressing_disconnect_mirror_stops_the_mirror_retrying_but_keeps_tcp() { + let (imotions, imotions_addr) = listener_on_free_port(); + let python_addr = closed_port_addr(); + let mut state = State::default(); + state.keep_connected(tcp_and_mirror(&imotions_addr, &python_addr)); + let _imotions_stream = + pump_until_accepted(&mut state, &imotions).expect("iMotions never connected"); + pump_update(&mut state, 30); + + let _ = update(&mut state, Message::DisconnectMirror); + + let_attempts_in_flight_finish(&mut state); + let python = TcpListener::bind(&python_addr).expect("bind failed"); + assert!( + !anything_connects_within_a_few_retries(&mut state, &python), + "the mirror kept retrying after Disconnect mirror" + ); + assert!(state.is_tcp_connected(), "Disconnect mirror dropped iMotions"); + + let _ = state.tcp_disconnect(); + } + + // A connection made by clicking now shows the drop too, so Connect TCP + // works again, but it is not retried: only command-line outputs are. + #[test] + fn a_clicked_tcp_connection_shows_the_drop_and_is_not_retried() { + let (listener, addr) = listener_on_free_port(); + let mut state = State::default(); + // What Connect TCP does, minus saving the address to the user's list. + state.tcp_connect(addr.clone()).expect("tcp_connect failed"); + let first = pump_until_accepted(&mut state, &listener).expect("never connected"); + assert!(pump_until(&mut state, |s| s.is_tcp_connected())); + + drop(first); + drop(listener); + assert!( + pump_with_samples_until(&mut state, |s| s.tcp_thread_handle.is_none()), + "the dead connection was kept" + ); + assert!(!state.is_tcp_connected()); + assert!( + state.event_log.iter().any(|e| e.contains("TCP disconnected")), + "the drop is missing from the event log: {:?}", + state.event_log + ); + + let listener = TcpListener::bind(&addr).expect("rebind failed"); + assert!( + !anything_connects_within_a_few_retries(&mut state, &listener), + "a clicked connection was retried" + ); + } } diff --git a/relay/src/view.rs b/relay/src/view.rs index 2b00c33..a96bf0b 100644 --- a/relay/src/view.rs +++ b/relay/src/view.rs @@ -152,8 +152,20 @@ fn human_bps(bps: f64) -> String { format!("{:.2} Gbps", gbps) } +/// How an output given on the command line is described while it is down. +fn waiting_text(address: &str) -> String { + format!("waiting for {address}, retrying") +} + fn tcp_connect_status_element(state: &State) -> UIElement { - text(format!("TCP Connection Status: {}", state.tcp_connected)).into() + // An address given on the command line is kept connected, so its status is + // shown live instead of waiting for Check TCP Connection Status. + let status = match (state.tcp_waiting_for(), state.tcp_retry.target.as_deref()) { + (Some(address), _) => waiting_text(address), + (None, Some(address)) => format!("connected to {address}"), + (None, None) => state.tcp_connected.to_string(), + }; + text(format!("TCP Connection Status: {status}")).into() } /// Simple helper: a button which triggers the app to verify the TCP connection state. @@ -228,9 +240,11 @@ fn mirror_addr_input(state: &State) -> UIElement { fn mirror_status_element(state: &State) -> UIElement { let status = if state.mirror_connected { - "Python mirror: connected" + "Python mirror: connected".to_string() + } else if let Some(address) = state.mirror_waiting_for() { + format!("Python mirror: {}", waiting_text(address)) } else { - "Python mirror: not connected" + "Python mirror: not connected".to_string() }; text(status).into() }