Fast4DReg is a Fiji macro for drift correction for 2D and 3D videos and is able to correct drift in all x-, y- and/or z-directions. Fast4DReg creates intensity projections along both axes and estimates their drift using cross-correlation based drift correction, and then translates the video frame by frame (Figure 1). Additionally, Fast4DReg can be used for aligning multi-channel 2D or 3D images, which is particularly useful for instruments that suffer from channel misalignment.
Check-out a Napari plugin created by Marcel Issler based on Fast4DReg here
Fast4DReg consists of four scripts, and two sub-menus. The Legacy 2D menu stores estimate and apply functions used in the script. These function are macro callable. NanoJTable IO menu stores functions for opening and saving the NanoJ tables, also macro callable.
-
time_estimate+apply: This script estimates drift in a single-channel XYT or XYZT time series and applies the correction to the same dataset. Correction can be performed for multiple time series in batch. The script also outputs a settings.csv file with relevant script parameters and file paths. This file can be used to correct a similar time series, for example a separate channel from the same acquisition.
-
time_apply: This script applies a drift correction from settings.csv to another single-channel XYT or XYZT time series with the same number of frames.
-
channel_estimate+apply: This script estimates channel misalignment in a single XYC or XYZC hyperstack and applies the correction to the same dataset. Correction can be performed for multiple hyperstacks in batch. The script also outputs a settings.csv file with relevant script parameters and file paths. This file can be used to correct a similar multi-channel image.
-
channel_apply: This script applies a channel correction from settings.csv to another XYC or XYZC hyperstack.
xy-correction (2D and 3D)
- First Fast4DReg creates intensity lateral projections (average or maximum) at each time point to create 2D videos. This part is skipped if using 2D videos.
- Second, Fast4DReg uses a drift correction algorithm to estimate the linear x-y drift between two images by calculating their cross-correlation matrix (CCM). The location of the peak intensity in the CCM determines the linear shift between the two images. Depending on the data, either the first frame or the previous frame of the raw data can be set as the reference frame.
- Once the drift is estimated, the dataset can be directly corrected frame by frame according to the amount of estimated drift.
z-correction (3D only)
- Fast4DReg creates lateral intensity projections (average or maximum) at each time point to create 2D videos along the y- or z-axis.
- Fast4DReg uses a drift correction algorithm to estimate the linear z-drift between two images by calculating their cross-correlation matrix (CCM). The location of the peak intensity in the CCM determines the linear shift between the two images. Depending on the data, either the first frame or the previous frame of the raw data can be set as the reference frame.
- Once the z-drift is estimated, the dataset can be directly corrected frame by frame according to the amount of estimated drift.
For time registration of a multichannel time series, split the channels first. Estimate drift from the channel with the most stable structures (for example, endothelium instead of migrating cancer cells), then use time_apply to apply the correction to each separate channel. This channel-splitting workflow applies only to time registration.
Figure 1: Fast4DReg pipeline. Fast4DReg corrects for drift in x-y direction by first creating intensity projections to create 2D videos. Then it estimates the linear x-y drift between two images by calculating their cross-correlation matrix and applying the correction to the stack. To correct the drift in the z-direction Fast4DReg creates frontal intensity and corrects the drift as described above. Lateral and axial drift corrections can also be used independently. Fast4DReg outputs a folder containing the corrected images, drift plots, a drift table and a settings file that can be applied to correct another image with the same settings.
Fast4DReg is easy to install by enabling the Fast4DReg update site:
- Open ImageJ
- Navigate to Help -> Update -> Manage update sites
- Select Fast4DReg
- When selected select Close and Apply changes.
- Restart Fiji.
Fast4DReg is dependent on Bio-Formats, which can be installed through the Fiji update site:
- Open ImageJ
- Navigate to Help -> Update -> Manage update sites
- Select Bio-Formats
- When selected select Close and Apply changes.
- Restart Fiji.
Before starting
Time registration requires a single-channel XYT or XYZT time series. Separate XYZ volumes are not a time series. For a multichannel time series, split the channels before running this workflow; estimate drift from one channel, then use time_apply for each separate channel. Each input to time_apply must also be a single-channel XYT or XYZT time series with the same number of frames.
For channel registration, channels must be combined into one XYC or XYZC hyperstack. Separate XYZ channel files must be combined before running channel_estimate+apply or channel_apply. Do not split channels for channel registration.
Running the script
- Open the "time_estimate+apply" from the Fiji Plugins menu: Plugins -> Fast4DReg -> time_estimate+apply. The user interface opens.
Figure 2: Estimate and apply user interface
- In the user interface
- Experiment number: Will be used for the output folder identifier.
- Select the path to the file to be corrected: Navigate to your image by using Add files.. or drag and drop into the white box. Files can be 2D and 3D videos or a mixture of those.
- xy-drift correction: if you want to correct for xy-drift, tick the xy-drift correction box.
- Projection type: Select the projection type used for xy-drift estimation (maximum or average intensity). Average projection usually works better for very noisy images.
- Time averaging: This sets the number of frames to average together to make coarser time points on which the cross-correlation analysis will be run to calculate drift. Setting this value to 1 will calculate straight frame-to-frame cross-correlations and while this should capture drift very accurately, it will also be very susceptible to noise. Conversely, setting this value high will average out noise but will also give a lower sample of the drift (which is then interpolated).
- Maximum expected drift: This refers to the maximum expected drift between the first frame of the dataset and the last frame of the dataset in units of pixels. Setting this to 0 will allow the algorithm to automatically determine the drift without any limitations. It is only really worth changing this value from 0 if running the algorithm gives incorrect results with too large jumps in estimated drift.
- Reference frame: If this is set to ‘first frame (default, better for fixed)’ then every averaged group of frames will be compared to the first average group of frames to calculate drift. If this is set to ‘previous frame (better for live)’ then every averaged group of frames will be compared to the previous averaged group of frames. For static samples, it is best to compare to the first frame, and for live samples where there may be slow scale drift overlaying the faster scale sample motion, it is better to compare to the previous frame.
- Crop output: Crop output will crop out the black frame created by the image moving.
- z-drift correction: If you want to correct for z-drift, tick the z-drift correction box.
- Reslice mode: Reslice mode lets you decide if you want to create the projection along the x-axis (top) or y-axis (left).
- Projection type: Select the projection type used for z-drift estimation (maximum or average intensity). Average projection usually works better for very noisy images.
- Extend stack to fit: Extend stack to fit will create extra slices to the stack to ensure that the whole stack is saved while moving up and/or down.
- Save RAM: If ticked the image will be converted to 32-bit, but the original bit-depth is kept. This saves RAM and speeds up the process.
-
Click ok. The script will run.
-
When the script has completed the process, you will have the following files in a new folder: - corrected images - drift plots - drift tables - a settings file, that you can use to run the script on another image with identical parameters (e.g. other channel).
The folder will have an unique identifier: fileName + date + experiment number. If you plan to apply the correction to another channel, make sure not to move these files to another folder.
- Open the "time_apply" from the Fiji Plugins menu: Plugins -> Fast4DReg -> time_apply. The user interface opens.
Figure 3: Apply user interface
-
Select the path to the file to be corrected: Navigate to your image by using Add files.. or drag and drop into the white box. The files can be located in different folders and must have the same number of frames as the time series used to create the settings file.
-
Select the settings file (csv.): Navigate to your settings file (called settings.csv).
-
Select where to save the corrected images: All corrected images will be saved to this folder.
- Click ok.
Done!
Importance of file locations
Make sure not to move the drift table from the results folder as the path to the drift table is hardcoded to the settings.csv file.
Result images are black
Try disabling time averaging (1).
Operating system specificity
If you create a settings file with the estimate+apply script and use it to correct another image with the apply script, make sure you use the same operating system (e.g., PC, Mac, Linux).
- Joanna Pylvänäinen
- Romain F. Laine
- Bruno Manuel Santos Saraiva
- Ricardo Henriques
- Guillaume Jacquemet
Laine, R. F., Tosheva, K. L., Gustafsson, N., Gray, R., Almada, P., Albrecht, D., Risa, G. T., Hurtig, F., Lindås, A. C., Baum, B., Mercer, J., Leterrier, C., Pereira, P. M., Culley, S., & Henriques, R. (2019). NanoJ: a high-performance open-source super-resolution microscopy toolbox. Journal of physics D: Applied physics, 52(16), 163001. https://doi.org/10.1088/1361-6463/ab0261
Joanna W. Pylvänäinen, Romain F. Laine, Bruno M. S. Saraiva, Sujan Ghimire, Gautier Follain, Ricardo Henriques, Guillaume Jacquemet; Fast4DReg – fast registration of 4D microscopy datasets. J Cell Sci 15 February 2023; 136 (4): jcs260728. doi: https://doi.org/10.1242/jcs.260728
230923 Version 2.1 (the clean 'n fast dimensionalist)
Made compatible with headless execution.
230112 Version 2.1 (the clean 'n fast dimensionalist)
Fast4DReg code has been cleaned and the RAM saving mode improved. The RAM saving mode now runs significantly faster than before.
221014 Version 2.0 (the dimensionalist)
Fast4DReg runs now independent of NanoJ and is able to correct drift in both, 2D and 3D images.
220222 Version 1.0 (pre-print)
All four scripts have been tested and work well. Ready for pre-print.