DigitalSurf format (SUR & PRO)

.sur and .pro is a format developed by digitalsurf to import/export data in the MountainsMap scientific analysis software. Target datasets are originally (micro)-topography maps and profile from imaging instruments: SEM, AFM, profilometery etc. RGB(A) images, multilayer surfaces and profiles are also supported. Even though it is essentially a surfaces format, 1D signals are supported for spectra and spectral maps. Specifically, this is the format used by Attolight for saving SEM-cathodoluminescence (SEM-CL) hyperspectral maps. This plugin was developed based on the MountainsMap software documentation.

Support for loading .sur and .pro files is complete, including parsing of custom metadata, and opening of files containing multiple objects. Some rare, deprecated object types (e.g. force curves) are not supported, due to no example data being available. Those can be added upon request to the module, if provided with example data and a explanations. Unlike hyperspy.signal, .sur and .pro objects can be used to represent heterogeneous data. For instance, float (topography) and int (rgb data) data can coexist along the same navigation dimension. Those are casted to a homogeneous floating-point representation upon loading.

Support for data saving is partial, as .sur and .pro do not support all features of hyperspy signals. Up to 3d data arrays with either 1d (series of images) or 2d (spectral maps) navigation space can be saved. .sur and .pro also do not support non-uniform axes and fitted models. Finally, MountainsMap maps intensities along an axis with constant spacing between numbers by enforcing an integer-representation of the data with scaling and offset. This means that export from float data is inherently lossy.

Within these limitations, all features from .sur and .pro fileformats are supported. Data compression and custom metadata allows a good interoperability of hyperspy and Mountainsmap. The file writer splits a signal into the suitable digitalsurf dataobject. Primarily by inspecting its dimension and datatype. If ambiguity remains, it inspects the names of signal axes and metadata.Signal.quantity. The criteria are listed here below:

Nav. dimension	Sig dimension	Extension and MountainsMap subclass
0	1	.pro: Spectrum (based on axes name), Profile (default)
0	2	.sur: BinaryImage (based on dtype), RGBImage (based on dtype), Surface (default)
1	0	.pro: same as (0,1)
1	1	.pro: Spectrum Serie (based on axes name), Profile Serie (default)
1	2	.sur: RGBImage Serie (based on dtype), Surface Series (default)
2	0	.sur: same as (0,2)
2	1	.sur: hyperspectralMap (default)

Axes named one of Wavelength, Energy, Energy Loss or E are considered spectral. A quantity named one of Height, Altitude, Elevation, Depth or Z is considered a surface. The difference between Surface and IntensitySurface stems from the AFM / profilometry origin of MountainsMap. “Surface” has its proper meaning of being a 2d-subset of 3d space, whereas “IntensitySurface” is a mere 2D mapping of an arbitrary quantity.

API functions

rsciio.digitalsurf.file_reader(filename, lazy=False)

Read a mountainsmap .sur or .pro file.

Parameters:

filenamestr, pathlib.Path

Filename of the file to read or corresponding pathlib.Path.

lazybool, default=False

Lazy loading is not supported.

Returns:

list of dict

List of dictionaries containing the following fields:

• ‘data’ – multidimensional numpy.ndarray or dask.array.Array

• ‘axes’ – list of dictionaries describing the axes containing the fields ‘name’, ‘units’, ‘index_in_array’, and either ‘size’, ‘offset’, and ‘scale’ or a numpy array ‘axis’ containing the full axes vector

• ‘metadata’ – dictionary containing the parsed metadata

• ‘original_metadata’ – dictionary containing the full metadata tree from the input file

When the file contains several datasets, each dataset will be loaded as separate dictionary.

rsciio.digitalsurf.file_writer(filename, signal: dict, set_comments: str = 'auto', is_special: bool = False, compressed: bool = True, comments: dict = {}, object_name: str = '', operator_name: str = '', absolute: int = 0, private_zone: bytes = b'', client_zone: bytes = b'')

Write a mountainsmap .sur or .pro file.

Parameters:

filenamestr, pathlib.Path

Filename of the file to read or corresponding pathlib.Path.

signaldict

Dictionary containing the signal object. Should contain the following fields:

• ‘data’ – multidimensional numpy array

• ‘axes’ – list of dictionaries describing the axes containing the fields ‘name’, ‘units’, ‘index_in_array’, and either ‘size’, ‘offset’, and ‘scale’ or a numpy array ‘axis’ containing the full axes vector

• ‘metadata’ – dictionary containing the metadata tree

set_commentsstr , default = ‘auto’

Whether comments should be a simplified version original_metadata (‘auto’), the raw original_metadata dictionary (‘raw’), skipped (‘off’), or supplied by the user as an additional kwarg (‘custom’).

is_specialbool , default = False

If True, NaN values in the dataset or integers reaching the boundary of the signed int-representation are flagged as non-measured or saturating, respectively. If False, those values are not flagged (converted to valid points).

compressedbool, default =True

If True, compress the data in the export file using zlib. Can help dramatically reduce the file size.

commentsdict, default = {}

Set a custom dictionnary in the comments field of the exported file. Ignored if set_comments is not set to ‘custom’.

object_namestr, default = ‘’

Set the object name field in the output file.

operator_namestr, default = ‘’

Set the operator name field in the exported file.

absoluteint, default = 0,

Unsigned int capable of flagging whether surface heights are relative (0) or absolute (1). Higher unsigned int values can be used to distinguish several data series sharing internal reference.

private_zonebytes, default = b’’,

Set arbitrary byte-content in the private_zone field of exported file metadata. Maximum size is 32.0 kB and content will be cropped if this size is exceeded.

client_zonebytes, default = b’’

Set arbitrary byte-content in the client_zone field of exported file metadata. Maximum size is 128 B and and content will be cropped if this size is exceeded.

rsciio.digitalsurf.parse_metadata(cmt: str, prefix: str = '$', delimiter: str = '=') → dict

Parse metadata from the comment field of a digitalsurf file, or any other str in similar formatting. Return it as a hyperspy-compatible nested dict.

Parameters:

cmtstr

Str containing contents of a digitalsurf file “comment” field.

prefixstr

Prefix character, must be present at the start of each line, otherwise the line is ignored. "$" for digitalsurf files, typically an empty string ("") when parsing from text files. Default is "$".

delimiterstr

Character that delimit key-value pairs in digitalsurf comment. Default is "=".

Returns:

dict

Nested dictionnary of the metadata.