From d875f6868b69864ffea6936e89b6f7f5510efd99 Mon Sep 17 00:00:00 2001 From: Chadwick Boulay Date: Wed, 14 Oct 2026 16:25:41 -0400 Subject: [PATCH] Link liblsl-specific content to the liblsl docs instead of repeating it - Time Synchronization keeps the practical guidance (synchronize at import with LabRecorder and the XDF importers, online options, non-LSL clocks, header parameters, validation). The measurement theory moved to liblsl's time synchronization page (sccn/liblsl#336). Fixes a link to an old commit of inlet.h. - Getting LSL: the liblsl section is a short summary with the lag caveat, linking to Getting liblsl instead of repeating its list. - User's Guide: a short overview of discovery, data, clock offsets and recovery, instead of an outdated copy of liblsl's internals. - Examples: link to liblsl's examples folder and README for C/C++. - FAQs: high data rates mention the synchronous outlet mode (1.18) instead of a 2022 pull request; current release file names; the two Binaries warnings were comments (missing ::) and are now shown. - Documentation Syntax becomes Contributing to the documentation: where each part of the docs lives, how to build both, the :liblsl: role. --- docs/dev/doc_syntax.rst | 34 ++++++--- docs/dev/examples.rst | 61 +++------------- docs/info/faqs.rst | 22 +++--- docs/info/getting_lsl.rst | 29 ++------ docs/info/time_synchronization.rst | 113 +++++++++++++---------------- docs/info/user_guide.rst | 79 +++++--------------- 6 files changed, 121 insertions(+), 217 deletions(-) diff --git a/docs/dev/doc_syntax.rst b/docs/dev/doc_syntax.rst index c9337f8..8b8c609 100644 --- a/docs/dev/doc_syntax.rst +++ b/docs/dev/doc_syntax.rst @@ -1,6 +1,19 @@ -==================== -Documentation Syntax -==================== +================================= +Contributing to the documentation +================================= + +LSL's documentation lives in two places: + +* **This site** (``docs/`` in the `labstreaminglayer repository `__): + what LSL is, getting started, devices, tools, language bindings, troubleshooting. Written in + reStructuredText, described below. +* **The liblsl documentation** (``docs/`` in the `liblsl repository `__): + getting, building and linking liblsl, its configuration, how it works on the network, and the + C/C++ API reference. Written in Markdown (MyST) with Mermaid diagrams; build it with + ``pip install -r docs/requirements.txt`` and ``sphinx-build docs docs/_build/html`` (Doxygen + must be installed for the API reference). + +Both are published on Read the Docs and rebuilt on every push. Restructured Text (RST) ======================= @@ -9,10 +22,7 @@ Restructured Text (`Quickstart `_, `another Quickstart `_) is a text based markup language like markdown. -In combination with `Sphinx `_) it is used to build this documentation, -most notable for the -`official Labstreaminglayer documentation `_ -hosted on ReadTheDocs and (some time in the future) a standalone PDF doc. +In combination with `Sphinx `_ it is used to build this documentation. You can view the source code for `this page `_ @@ -26,10 +36,10 @@ The is built on every commit to the :repo:`Labstreaminglayer repository `. In order to avoid several commits until you get the formatting right, install -the Python ``sphinx`` package (either with pip or conda) and build the -documentation with +the requirements (``pip install -r docs/requirements.txt``) and build the +documentation in the ``docs`` folder with :samp:`make html`, or (if you don't have :command:`make` installed) -:samp:`sphinx-build . _build`. +:samp:`sphinx-build . _build/html`. @@ -81,6 +91,10 @@ Some pages are linked to quite often, so there are some shortcuts configured in :samp:`:lslrelease:`{LabRecorder}`` to an App's release page, i.e. :samp:`https://github.com/labstreaminglayer/App-{LabRecorder}/releases`. +:samp:`:liblsl:`{link text} <{guide/linking.html}>`` links to a page of the liblsl +documentation, e.g. :liblsl:`Using liblsl in your project `. Prefer this +for liblsl's guide pages (written in Markdown); for the API reference use the intersphinx links below. + `Intersphinx`_ ~~~~~~~~~~~~~~ diff --git a/docs/dev/examples.rst b/docs/dev/examples.rst index bdb41d9..244d4d5 100644 --- a/docs/dev/examples.rst +++ b/docs/dev/examples.rst @@ -17,9 +17,9 @@ API Documentation ***************** It is recommended that you clone the repository to get the respective code. The documentation is at the following locations: - * C: `C header file `__ - * C++: `C++ header file `__, - :doc:`in progress API documentation ` + * C and C++: the :doc:`liblsl API reference ` (from the + `C `__ and + `C++ `__ headers) * Python: `pylsl module `__ * Java: `JavaDocs `__ * C#: `LSL module `__ @@ -31,54 +31,15 @@ hopefully leave no questions unanswered. Note that a typical application will only need a small subset of the API (as used in the example programs). -C Example Programs: Basic to Advanced -************************************* +C and C++ Example Programs +************************** -These two example programs illustrate the bread-and-butter use of LSL as it is executing in almost any device module that comes with the distribution: - * `Sending a multi-channel time series into LSL. `__ - * `Receiving a multi-channel time series from LSL. `__ - -These two example programs illustrate a more special-purpose use case, namely sending arbitrary string-formatted data at irregular sampling rate. Such streams are used by programs that produce event markers, for example: - * `Sending a stream of strings with irregular timing. `__ - * `Receiving a stream of strings with irregular timing. `__ - -The last example shows how to attach properly formatted meta-data to a stream, and how to read it out again at the receiving end. While meta-data is strictly optional, it is very useful to make streams self-describing. LSL has adopted the convention to name meta-data fields according to the XDF file format specification whenever the content type matches (for example EEG, Gaze, MoCap, VideoRaw, etc); the spec is `here `__. Note that some older example programs (SendData/ReceiveData) predate this convention and name the channels inconsistently. - * `Handling stream meta-data. `__ - -C++ Example Programs: Basic to Advanced -*************************************** - -These two example programs illustrate the shortest amount of code that is necessary to get a C++ program linked to LSL: - * `Minimal data sending example. `__ - * `Minimal data receiving example. `__ - -These two example programs demonstrate how to write more complete LSL clients in C++ (they are 1:1 equivalents of the corresponding C programs): - * `Sending a multi-channel time series into LSL. `__ - * `Receiving a multi-channel time series from LSL. `__ - -These two programs transmit their data at the granularity of chunks instead of samples. This is mostly a convenience matter, since inlets and outlets can be configured to automatically batch samples into chunks for transmission. Note that for LSL the data is always a linear sequence of samples and data that is pushed as samples can be pulled out as chunks or vice versa. They also show how structs can be used to represent the sample data, instead of numeric arrays (which is mostly a syntactic difference): - * `Sending a multi-channel time series at chunk granularity. `__ - * `Receiving a multi-channel time series at chunk granularity. `__ - -These two example programs illustrate a more special-purpose use case, namely sending arbitrary string-formatted data at irregular sampling rate. Such streams are used by programs that produce event markers, for example. These are 1:1 equivalents of the corresponding C programs: - * `Sending a stream of strings with irregular timing. `__ - * `Receiving a stream of strings with irregular timing. `__ - -The last example shows how to attach properly formatted meta-data to a stream, and how to read it out again at the receiving end. While meta-data is strictly optional, it is very useful to make streams self-describing. LSL has adopted the convention to name meta-data fields according to the XDF file format specification whenever the content type matches (for example EEG, Gaze, MoCap, VideoRaw, etc); the spec is `here `__. Note that some older example programs (SendData/ReceiveData) predate this convention and name the channels inconsistently. - * `Handling stream meta-data. `__ - -C/C++ Special-Purpose Example Programs -************************************** -These programs illustrate some special use cases of LSL that are also relevant for C programmers. See the lsl\_c.h header for the corresponding C APIs (they are very similar to the C++ code shown here). - -This example illustrates in more detail how streams can be resolved on the network: - * `Resolving all streams on the lab network, one-shot and continuous. `__ - -This example shows how to query the full XML meta-data of a stream (which may be several megabytes large): - * `Retrieving the XML meta-data of a stream. `__ - -This example shows how to obtain time-correction values for a given stream. These time-correction values are offsets (in seconds) that are used to remap any stream's timestamps into the own local clock domain (just by adding the offset to the timestamp): - * `Querying the time-correction information for a stream. `__ +liblsl's `examples folder `__ has C and C++ +programs from basic to advanced: sending and receiving time series (sample by sample and in +chunks), string event markers, stream metadata, resolving all streams on the network, reading a +stream's full metadata, and querying time correction. Its +`README `__ walks through them, and +:liblsl:`Using liblsl in your project ` shows how to build them. Python Example Programs: Basic to Advanced ****************************************** diff --git a/docs/info/faqs.rst b/docs/info/faqs.rst index a2c7a25..7e976c3 100644 --- a/docs/info/faqs.rst +++ b/docs/info/faqs.rst @@ -169,9 +169,10 @@ pre-allocated buffer. Make sure that you use a recent version of liblsl (1.10 or later offers a faster network protocol) at both the sender and the receiver. -If you are writing an application that needs to push a lot of data, please show -your interest by commenting on the long-lingering `pull request to speed up pushes -`_ by reducing the number of data copies. +For outlets that push a lot of data to few inlets, liblsl 1.18 adds a synchronous (zero-copy) +mode: create the outlet with the ``transp_sync_blocking`` transport option, and each push writes +directly to the connected inlets instead of copying through a buffer. See +:liblsl:`data transport ` for its trade-offs. Chunk sizes ----------- @@ -256,20 +257,19 @@ how to name the file, i.e. :file:`lsl.dll` for Windows, :file:`liblsl.dylib` for MacOS / OS X. The `liblsl release page `_ -has multiple packages, generally called -:file:`liblsl-{version}-{system}.{extension}`, e.g. -:file:`liblsl-1.13.1-Linux64-bionic.deb` with the 64 bit Ubuntu Linux 18.04 -(Codename Bionic Beaver) package or -:file:`liblsl-1.13.1-Win32.zip` with the 32 bit Windows DLLs. +has packages named for the system and processor, e.g. +:file:`liblsl-{version}-noble_{amd64}.deb` for 64-bit Ubuntu 24.04 (Noble) or +:file:`liblsl-{version}-Win_{i386}.zip` for 32-bit Windows; +:liblsl:`Getting liblsl ` lists them all. -.. warning +.. warning:: Android also has ``.so`` shared objects, but has a different toolchain and architecture so the binaries (even if they're also named - :file:`liblsl.so`) are **not** interchangeable with ``.so`` files for regular + :file:`liblsl.so`) are **not** interchangeable with ``.so`` files for regular Linuxes. -.. warning +.. warning:: Embedded Linux devices typically have an ARM processor instead of an x86 / x64 processor so the default linux binaries won't work (resulting in an diff --git a/docs/info/getting_lsl.rst b/docs/info/getting_lsl.rst index 55bd628..f2b06bb 100644 --- a/docs/info/getting_lsl.rst +++ b/docs/info/getting_lsl.rst @@ -14,30 +14,13 @@ What you need depends on what you want to do: liblsl ====== -liblsl is the C/C++ library that every LSL application and binding uses. Its own documentation -covers in detail how to :liblsl:`get it `, -:liblsl:`use it in your project ` and -:liblsl:`build it `. In short: +liblsl, the C/C++ library under every LSL application and binding, comes as release downloads for +Windows, Linux, macOS and iOS, from package managers (conda-forge, vcpkg, Conan, Homebrew), or +built as part of your own CMake project. :liblsl:`Getting liblsl ` lists them +all; :liblsl:`Using liblsl in your project ` shows how to link it. -* **Build it with your project.** C/C++ projects using CMake can add liblsl with - ``FetchContent`` and link the ``LSL::lsl`` target. This always gives you exactly the version - you ask for, built with your compiler. -* **Release downloads.** Every `liblsl release `__ - has builds for Windows (x64, x86, ARM64, and MinGW), Ubuntu (``.deb``, x64/ARM), macOS (an - installer for ``lsl.framework``) and an ``xcframework`` for macOS and iOS. These are built and - tested by the liblsl repository's own CI. -* **Package managers.** liblsl is also available from: - - * `conda-forge `__: ``conda install -c conda-forge liblsl`` - * `vcpkg `__: ``vcpkg install liblsl`` - * `Conan `__ - * `Homebrew `__: ``brew install labstreaminglayer/tap/lsl`` - - .. warning:: - These packages are maintained separately from liblsl and are updated by hand after each - release, so they are often one or more releases behind (check the version they offer). - The best-supported routes are the release downloads and building liblsl with your project - through its CMake integration, which liblsl's CI tests on every change. +The package-manager versions are maintained separately and are often a release or more behind; +the release downloads and liblsl's own CMake integration are the best-supported routes. Apps and the full source tree ============================= diff --git a/docs/info/time_synchronization.rst b/docs/info/time_synchronization.rst index 604f9ad..455a470 100644 --- a/docs/info/time_synchronization.rst +++ b/docs/info/time_synchronization.rst @@ -3,65 +3,56 @@ Time Synchronization ******************** -General Information -################### - -Introduction -==================================== -LSL does not perform synchronization by default, but it does provide all of the information required for synchronization. Synchronization information includes timestamps for each sample (or chunk of samples) and clock offset measurements between the stream origin (outlet) and the stream consumer (inlet). Detailed descriptions of the timestamps and the clock offsets can be found below. - -File-recording applications should store all synchronization information to disk. File importers should then use the synchronization information from the entire span of the recording to perform synchronization. `LabRecorder `_ handles storage of the synchronization information in the XDF file for you. The official XDF importers (pyxdf or xdf-Matlab), whether used directly or integrated into a tool like MNE or EEGLAB, use the synchronization information in the XDF file and perform stream synchronization upon file import. Please use these tools for your data storage and import if possible. - -Online applications often do not require synchronization. Typically, what's actually needed is low latency; the receiving application should use small chunk sizes and process data for feedback as soon as possible. -However, there are use-cases requiring online synchronization such as event-triggered processing like P300- and SSVEP-based BCIs, because the time of the event is used to segment the neural data. When online synchronization is required, please set the `inlet's postprocessing flags appropriately `_. - -Synchronization Information -=========================== -The built-in time synchronization in LSL relies on two pieces of data that are being collected alongside with the actual sample data: - -1. A timestamp for each sample that is read from a local monotonic clock of the computer. The method for obtaining the timestamp value can be found `here `_. As of this writing, the clock is `std::chrono::steady_clock `_. From `this S/O answer `_, "`steady_clock` is like a stopwatch. It is great for timing how long something takes. But it can't tell you the time of day. It has no relationship whatsoever to a human calendar on any platform. On macOS `steady_clock` it is a count of nanoseconds since the computer booted." -2. Out-out-of-band clock synchronization information that is transmitted along with each data stream to the receiving computer and collected by those applications that are interested in synchronization features (a recording program, such as LabRecorder, for instance). This clock synchronization information consists of measurements of the momentary offset between the two involved clocks that are made at periodic intervals (every few seconds). - -Detailed Information -#################### - -Clock Offset Measurement -======================== -The clock offsets are measured using a protocol that is similar to the Network Time Protocol. Each offset measurement involves a brief sequence of n UDP packet exchanges (n=8 by default) between the two involved computers. Each packet exchange yields a) an estimate of the round-trip-time (RTT) between the two computers, and b) an estimate of the clock offset (OFS) at the time of the measurement with the round-trip-time factored out. The final offset estimate for the given measurement is the one offset value out of the n measured ones for which the associated RTT value was minimal (to attain a high chance of using only estimates that were taken under nominal packet transport conditions and with minimal effect of transmission delays). The process of repeated measurement and selection is called the “Clock Filter” algorithm in NTP. - -Network Packet Exchange -======================= -The packet exchange for obtaining a single raw OFS and RTT measurement works as follows. Suppose Alice initiates a query of Bob’s clock offset relative to her clock. Alice sends a packet P1 containing the timestamp t0 of when the packet was submitted to the network stack. Bob, upon receiving the packet, appends the time stamp t1 of when the packet was received at his end of the network. Bob then returns a packet P2 that contains t0, t1, and a timestamp t2 of when the return packet was submitted. The transmission is completed with Alice receiving P2 and taking the timestamp t3 of the time of receipt. The time duration of Bob’s processing is given by t2 - t1 and the overall time of the process is t3 - t0. The RTT of the exchange is (t3-t0) - (t2-t1). The clock offset OFS is ((t1-t0) + (t2 - t3)) / 2. - -Resulting Estimation Bias -========================= -The synchronization is correct if both the incoming and outgoing routes of the server have symmetrical nominal delay. If this delay differs, the synchronization has a systematic bias of half the difference between the forward and backward travel times. On a symmetric local area network, the travel time differences are a sum of random delays due to multi-threading, buffering or interruptions on one hand, and the systematic difference in the latency of the two involved network stacks. Since the clock filtering algorithm reduces the probability of the random effects on the estimates, the residual inaccuracy is dominated by the systematic difference between the latency of both network stacks (e.g. Linux vs. Windows), which can be expected to be reasonably well below a millisecond on modern hardware and operating systems (see also section Validation). - -Synchronization Process (Offset Correction) -=========================================== -Given a history of clock offset measurements between two computers, the timestamp of a remotely collected sample can be remapped into the local time domain to be comparable with all other available samples. In the simplest case this can be accomplished by adding the most recent clock offset value to each remotely collected timestamp. A more powerful approach would be to calculate a linear fit through the (perhaps noisy) clock offset measurements and use the resulting linear mapping to remap the timestamps, while an even more sophisticated approach might be to perform an outlier-resilient (robust) linear fit through the clock offsets to get a better linear mapping. The library itself does not perform any particular such mapping but instead leaves it to the user (i.e., the recipient of the data) to combine the history of offsets with the sample timestamps if desired. - -Synchronization for Online Processing -------------------------------------- -Most online processing applications are only ever receiving a single stream, or do not require fine-grained time alignment so that no online synchronization is necessary at all. For those applications that do need synchronized time-stamps, it usually suffices to add the last clock offset to the time stamps for each inbound stream. - -It is possible to average out the error in successive offset measurements to reduce it further, but it should be noted that this synchronization-related jitter is already small, and often the jitter in the raw time-stamp values even before offset correction is already at least 10x larger. These two sources of jitter can be cleanly addressed in a single post-processing step of the synchronized time stamps (see sections Time-Stamp Post-Processing (Dejitter) and Online Smoothing). - -Synchronization in Multi-Stream Recordings ------------------------------------------- -When data is recorded to disk, the reference recording program included with LSL (LabRecorder) will just collect all the available information, including time stamps and clock offsets for every remote stream and store it unmodified -- thus leaving it to the importer function to handle the time correction calculations post-hoc. This is to allow the user to employ their algorithm of choice even long after the data have been collected. The reference importer (load\_xdf) performs a linear fit through the clock offsets for each remote stream (thereby assuming a linear clock drift) and uses this linear mapping to remap given remotely collected timestamps. - -Time-Stamp Post-Processing (Dejitter) -------------------------------------- -After the time series have been synchronized (i.e., the clock offset has been corrected), it is important to note that there will almost always be a jitter (e.g., Gaussian noise) in the time stamps, which is not due to the synchronization but because the time-stamping itself usually does not happen exactly in regular intervals but on a somewhat random schedule (dictated by the perils of the hardware, the driver, and the operating system). For milisecond-accurate multi-stream alignment, it is therefore necessary to remove this random jitter as well, which can be done by applying a trend-adjusted smoothing algorithm to it. Smoothing can be done either online at the time of data collection, or offline at data import time. If the data are analyzed post-hoc rather than online, it is always better to perform it at import time, since one can do a better job given all the available data than doing it incrementally as the data is being collected. - -Smoothing At Import Time ------------------------- -Besides correcting for the clock domain, the XDF importer will, by default, as the last step do a final re-calculation of the timestamps of all streams that have regular sampling rate. This is desirable since the actual time stamps of samples or of blocks of samples are typically subject to a small jitter. The calculation handles each stream independently and first checks if the time series contains any interruptions (i.e., data loss) using a generous threshold on the observed sample-to-sample intervals. For each uninterrupted segment it then calculates a linear fit between the index of each respective sample and its timestamp (thus assuming a constant but arbitrary effective sampling rate) and then re-calculates from it the time stamps of all samples based on their respective index. - -Online Smoothing ----------------- -To smooth the time stamps online, multiple algorithms can be used. The simplest one is double exponential smoothing, which is relatively easy to implement in an online data processing system, although it should be noted that during the first few minutes of operation (up to 5) the resulting time stamps will still have a considerable error (above 1 ms). The forgetting factors of this algorithm need to be set depending on the sampling rate of the data and the amplitude of the jitter in the raw stamps, and can require some tuning until the desired accuracy is reached. A better alternative is Recursive Least Squares (RLS), which has essentially optimal convergence behavior, although it will still take a minute or two of warmup until the jitter after smoothing reaches an acceptable level (<1ms) in realistic settings. To achieve this precision, the forget factor should be set such that a sample that is 30 seconds old will have an effective weighting of 1/2 (this depends on the sampling rate); one can also use 60 or as much as 120 seconds to further increase the precision -- however, too large values can fail to track sufficiently fast non-linear clock drift due to room or computer temperature changes (empirically, a half time of x seconds will be able to track clock rate fluctuations that change on the order of 10*x seconds or more slowly). liblsl has this built in: enable it on an inlet with the ``proc_dejitter`` post-processing flag. Its half-time is the ``SmoothingHalftime`` setting (default 90 seconds) in the :doc:`configuration file `; see :liblsl:`time synchronization ` for how liblsl measures offsets and post-processes time stamps. Other algorithms, such as QR-RLS and the Kalman filter can also be used and will perform similarly to RLS (note: some formulations of RLS have numerical difficulties, depending on the order of operations, and should not be used as they can 'blow up' after a few minutes of use). +LSL does not change your data's timestamps by default, but it records everything needed to +synchronize streams from different devices and computers: + +1. **A timestamp for each sample** (or chunk), taken from the sending computer's local clock + (``lsl_local_clock()``). This is a monotonic clock that counts seconds from an arbitrary + starting point (often the time the computer booted); it has no relation to the time of day. +2. **Clock offsets** between the sending and the receiving computer, measured every few seconds + by exchanging short UDP packets, similar to the Network Time Protocol (NTP). Applications that + want them, such as a recorder, call ``time_correction()`` on their inlet. + +How liblsl measures the offsets, how accurate they are, the ways to turn them into synchronized +timestamps and how its built-in smoothing works are described in liblsl's +:liblsl:`time synchronization ` page. This page is about getting +synchronized data in practice. + +Recording: synchronize at import +================================ + +For data that is analyzed afterwards, record everything and synchronize when loading the file. +`LabRecorder `__ stores the raw +timestamps and all clock offset measurements of every stream in the XDF file. The XDF importers +(`pyxdf `__, +`xdf-Matlab `__, and the tools built on them such as +MNE-Python and EEGLAB) then: + +* fit a straight line through each stream's clock offsets (assuming a constant clock drift) and + use it to map the stream's timestamps onto the recording computer's clock, and +* for streams with a regular sampling rate, remove the jitter in the timestamps: they split the + stream at interruptions (data loss) and, for each uninterrupted segment, fit a line between + sample index and timestamp and recompute every timestamp from it. + +Because the raw data is kept, a different mapping can be chosen later. Please use these tools to +record and import data where possible. + +Online processing +================= + +Many online applications don't need synchronization at all: what they need is low latency, so +they should use small chunks and process data as soon as it arrives. Some do need synchronized +timestamps, for example event-related processing (P300 or SSVEP brain-computer interfaces), +where the time of an event is used to cut out the neural data. Then either: + +* set the inlet's post-processing flags (``proc_clocksync`` to add the latest clock offset, + ``proc_dejitter`` to smooth the timestamps of regularly sampled streams, or ``proc_ALL``), see + :cpp:enum:`lsl_processing_options_t`; or +* call ``time_correction()`` yourself and add its result to the timestamps of that inlet. + +Adding the latest offset is usually enough; the remaining error is small compared to the jitter +of the raw timestamps, which ``proc_dejitter`` removes. Its smoothing settles within a minute or +two (see :liblsl:`dejitter ` for how to tune it). Manual Synchronization With Non-LSL Clocks ========================================== @@ -72,7 +63,7 @@ The clock offset UDP stream described above used by LSL to synchronize streams f The most straightforward solution, which seems to be the most common, is to use hardware. For example, a common signal such as a digital pulse can be fed to an LSL-integrated device and to the non-LSL system, then an offline analysis of the two files can align the data sources via cross-correlation. However, a hardware solution is not always possible, especially in mobile experiments. -The next most straightforward solution is to create a map between LSL time and the _other_ clock, and this is probably most easily done in software by creating a simple LSL integration for that other clock. In this application, the _other_ clock and the LSL clock are read at approximately the same time (i.e., one line of code after the other) and the time of the other clock is sent as the data payload and the coincident LSL clock is sent as the time stamp in the call to `push_sample`. Subsequent offline analysis can load these clock-LSL pairs and create a mapping between the two. The most common mapping would be via a linear regression and that is probably sufficient in most cases (i.e., where the clock drift is consistent). Piecewise or moving-window mappings will be more accurate but also slower to calculate. +The next most straightforward solution is to create a map between LSL time and the *other* clock, and this is probably most easily done in software by creating a simple LSL integration for that other clock. In this application, the *other* clock and the LSL clock are read at approximately the same time (i.e., one line of code after the other) and the time of the other clock is sent as the data payload and the coincident LSL clock is sent as the time stamp in the call to ``push_sample``. Subsequent offline analysis can load these clock-LSL pairs and create a mapping between the two. The most common mapping would be via a linear regression and that is probably sufficient in most cases (i.e., where the clock drift is consistent). Piecewise or moving-window mappings will be more accurate but also slower to calculate. Note that the manual synchronization solutions described above only describe how to save data that will allow for offline synchronization during offline data analysis. This should satisfy most use cases. Online synchronization between the LSL clock and a non-LSL clock is still possible using the above software solution with a moving-window mapping (or exponential decay mapping) and setting the :cpp:enumerator:`proc_clocksync` or :cpp:enumerator:`proc_ALL` postprocessing flags at inlet creation time. Otherwise, you have to add the time offset returned by :cpp:func:`lsl::stream_inlet::time_correction()` to the timestamps to have them in your local domain. @@ -97,4 +88,4 @@ It is recommended that all LSL stream generators attach the following block to t Validation ========== -To see the synchronization capabilities of LSL in action, see https://sccn.ucsd.edu/download/mgrivich/LSL_Validation.html and especially https://sccn.ucsd.edu/download/mgrivich/LSL_Validation.html. \ No newline at end of file +To see the synchronization capabilities of LSL in action, see https://sccn.ucsd.edu/download/mgrivich/LSL_Validation.html. diff --git a/docs/info/user_guide.rst b/docs/info/user_guide.rst index 61400f6..00850f2 100644 --- a/docs/info/user_guide.rst +++ b/docs/info/user_guide.rst @@ -6,65 +6,20 @@ Basic Operation An effective use of LSL requires there to be at least one application streaming data and at least one application receiving data. The stream source, also called an "Outlet", advertises itself on the network. The receiver resolves the advertised stream and creates a connection to it, also called an "Inlet". For each Inlet, the Outlet buffers data and then the next N samples are transmitted from that buffer to the inlet application when the inlet requests it. -Stream advertisement and data transmission all happen using networking protocols. The networking in LSL works as follows. The only active participants in LSL networking are the outlet objects, the inlet objects, as well as calls to a resolve function (these are all implemented in the liblsl library). Most details of the communication are configurable, so in the following only the default behavior is explained for conciseness. - -Outlets -======= -An :term:`Stream Outlet` may use either IPv4, or IPv6, or both IP stacks in parallel. -In the latter case some of the sockets and threads are duplicated. - -The main outlet class is :cpp:class:`lsl::stream_outlet`. - -UDP Service Discovery ---------------------- -An outlet creates a group of UDP sockets on which it listens for incoming content-based queries. Such queries are used to find a stream of interest that has the desired meta-data (e.g., a particular content type or number of channels). An outlet matches each inbound query to its stream's meta-data and responds only in case of a match. Repeated queries are served by a query cache for efficiency. - -In particular, outlets listen via UDP on the multicast port (16571) for broadcast and multicast queries. Multiple sockets are created for this purpose, each of which listens on a different multicast address; these include both IPv4 and IPv6 addresses of the desired scope (for robustness to network conditions). The sockets are served from a single thread using asynchronous I/O for efficiency. While this still sounds like a lot of overhead, note that this thread is in waiting state for most of the time. - -In addition, outlets create a single regular (unicast) UDP socket that listens for content queries on a free port in the port range 16572-16603. Resolvers query it directly for machine-local streams and for the computers listed in ``KnownPeers``. This socket not only handles content queries but also responds to time estimation packets (used by inlets to estimate clock synchronization when a transmission is ongoing). - - -TCP Data Transmission ---------------------- -In addition each outlet creates a TCP server socket on a free port in the port range 16572-16603 to accept actual data transmission requests. There are two types of transmission requests: a request for streaming data (such a connection would be established when an inlet is used to pull samples from a particular provider) and a request for meta-data (such a connection would be established when an inlet is used to retrieve the full meta-data of a stream). TCP servers can also answer discovery queries; this is only used when the optional ``ResolveOverTCP`` setting is turned on, because it is slow. - -An outlet can handle an arbitrary number of parallel connections to serve any number of listeners. - -Resolve Operations -================== -When a client creates a Stream :term:`Resolver` to resolve a stream with desired properties on the -lab network, it issues a resolve operation. -A resolve operation temporarily opens a UDP socket per IP version through which queries are sent to the broadcast, multicast or unicast addresses that are configured as active; replies come back to the same socket. The resolve operation emits a packet across each channel in a brief burst and waits for responses, then sends the next burst. This is repeated until the required conditions are fulfilled (e.g., service was found, timeout, etc.). - -Inlets -====== -An :term:`Stream Inlet`, when created, initially does not create any sockets. -Sockets are created as needed for the following purposes: - -* When a transmission of streaming data is started by calling - :cpp:func:`lsl::stream_inlet::pull_sample()` or - :cpp:func:`lsl::stream_inlet::open_stream()`, a TCP connection is established to the outlet in - question and if successful, data is received over this connection until it breaks off or is - explicitly terminated. -* If the XML meta-data of the inlet is requested via :cpp:func:`lsl::stream_inlet::info`, - another TCP connection is established which persists until the meta-data has been transferred - from the outlet. -* When the inlet is used to estimate time-correction values via - :cpp:func:`lsl::stream_inlet::time_correction()` (as the recorder program does), a UDP socket is - opened which periodically (every few seconds) exchanges a brief series of packets with the - outlet's unicast UDP socket. -* If the connection to the outlet breaks off or is otherwise unsuccessful (for example due to a computer crash at the other end), and if the outlet was "recoverable", a resolve operation will be periodically scheduled every few seconds until the outlet in question is found again or the inlet is destroyed, or the active transmission is stopped. - -Clock synchronization ---------------------- - -Here we talk about important functions like the local clock -(:cpp:func:`lsl_local_clock`), a function to query a time offset -(:cpp:func:`lsl_time_correction_ex`) and a function to get the library version used -by the loaded library (:cpp:func:`lsl_library_version`). - -Further reading -=============== - -liblsl's documentation describes the mechanisms in detail, with the exact messages, ports and -timings: :liblsl:`How liblsl works `. +All of this is implemented in liblsl, the library that every LSL application and binding uses: + +* **Discovery.** An outlet answers queries for its stream over UDP (multicast, broadcast and + unicast). A resolve operation sends such queries, e.g. "all streams with type EEG", and collects + the descriptions of the matching streams. +* **Data.** An inlet connects to the outlet over TCP, receives the stream's samples into its + buffer, and fetches the full metadata on request. +* **Clock offsets.** When asked for time correction (as a recorder does), the inlet measures the + offset between the two computers' clocks every few seconds with short UDP exchanges; see + :doc:`time_synchronization`. +* **Recovery.** If the connection breaks off, for example because the sending program or + computer restarted, the inlet finds the stream again and reconnects, as long as the stream's + ``source_id`` identifies it uniquely. + +The network ports and settings involved are in :doc:`network-connectivity` and +:doc:`lslapicfg`. liblsl's documentation describes each mechanism in detail, with the exact +messages, ports and timings: :liblsl:`How liblsl works `.