Smart Align is a software tool for the correction of drift and distortions from image-series data. The image data can be from cameras and telescopes as well as parallel or serial (scanned) microscopes and can accept data from movie frames, time-series, focal-series, or camera-length series.
The Smart Align software includes special options for both parallel recorded data, such as cameras and transmission electron microscope (TEM) images, or for serial scanned images such as scanning transmission electron microscope (STEM) or scanning tunnelling microscope (STM) data.
The Smart Align software can be used in various modes, but the most common is for the distortion correction of atomic resolution STEM data. For this both rigid and non-rigid registration options are included.
A typical processing work-flow would be done in three stages:
- simple image translation, such as from stage/sample movement,
- affine correction (linear distortions: scale, shear) to correct stage-drift effects,
- non-linear distortion correction to correct for scanning-distortions.
In the special case of physical scanned probe microscopy (SPM), where tip-height may also be corrupted, an additional optional mode is available for STM/AFM tip-height correction.
The movies below show an example from a high-angle annular dark-field, HAADF STEM, time-series data-set from a crystal of strontium titanate in a  orientation. In this example, the STEM fast-scan direction is horizontal in every frame. An equivalent demonstration using telescope camera footage can be seen here.
The raw data:
After image-translation (rigid-alignment):
In this movie, the image-translation from stage drift has been corrected. As the sample was drifting towards the right in the raw data, after cropping the width of the image is reduced. The result shows the remaining non-linear scanning distortions as ‘wobbling’ horizontal bands through the image.
After non-rigid alignment (scan-distortion corrected):
In this final movie, the data have further been corrected for non-linear scanning distortions. This means that either the image frames can be averaged to improve the signal-noise ratio without any loss of resolution, or dynamic effects can be studied more easily.
Smart Align is available is available as a plug-in for Digital Micrograph but also can be trialled in MatLab.
[table id=2 /]
Download Smart Align
The MatLab™ version of Smart Align is free to use for academic and non-commercial use (with citation) and can be downloaded below.
MatLab Version User Manual
Smart Align is configured and run from the ‘wrapper’ file. Here you configure all the options and execute the code. The options in the wrapper are divided into:
- Data pre-processing,
- Rigid registration options,
- Non-rigid registration options,
- Display options,
- Developer / debug options (unsupported).
Data Pre-processing Options
trim_frames – this allows users to reduce the number of frames in the analysis, select 1 for yes, 0 for no. If trimming frames the user must also specify the number of the first_frame and last_frame to include. This trimming is useful for optimising the other registration settings on a few frames before running a very long series.
reverse_order – this allows the user to flip the order of the frames and takes effect after any previous frame trimming, select 1 for yes, 0 for no.
Rigid Registration Options
Non-rigid Registration Options
use_non_rigid_align – choose whether to use the non-rigid module of the software, select 1 for yes, 0 for no. If not used all further options in this section need not be specified.
is_time_series – select 1 for yes, 0 for no; choose only one of this and the following option. In this mode the whole data-set is averaged for the reference frame.
is_focal_series – select 1 for yes, 0 for no; choose only one of this and the previous option. In this mode only the average of the neighbouring frames is used as the reference frame.
non_rigid_cycles – enter the number of outer-most iteration cycles. This is the number of times the reference data will be updated and all offsets recalculated (must be an integer).
non_rigid_target – enter the target for the incremental offset value at which the inner-most iteration is advanced (units of pixels). A smaller number will enhance convergence but will increase calculation time.
lock_row_together – option to measure the offset of fast-scan lines as a whole. This option is customised for scanned microscope data and improves the robustness of measurements with noisy data, select 1 for yes, 0 for no.
scan_line_time – used to calculate the frequency axis labels on the frequency diagnosis panel. Should be entered in units of seconds per scan-line. This should be the time including any fly-back; that is, the time between lines, not the time of a line.
show_progress – this option enables visual feedback during the calculation and shows graphical outputs. This gives useful feedback when setting up a new calculation but can be disabled to improve speed once other settings have been optimised.
Supported File Types
Smart Align is optimised to read in data from a *.mat file. When loading the image-series it should be stored as a single variable named “input_file” with dimensions ‘height x width x n’. Data can also be read in as *.txt having been saved from Digital Micrograph. To do this the images should be a in stack form in DM, then saved as a single *.txt file. If necessary DM stack images can be built from individual DM images using the free HREM IPU tool. Importing stacks from DM via *.txt supports square images only.
After using Smart Align, the registered data can be written back out for import into Digital Micrograph. The variable name to write is ‘aligned_stack’, if a second signal was also used this is ‘second_aligned_stack’.
Highlighted uses of Smart Align
- “Atomic scale dynamics of a solid state chemical reaction directly determined by annular dark-field electron microscopy”, Timothy J. Pennycook, Lewys Jones, Henrik Pettersson, João Coelho, Megan Canavan, Beatriz Mendoza-Sanchez, Valeria Nicolosi & Peter D. Nellist.Scientific Reports 4 7555 (2014). DOI:10.1038/srep07555.