hyperspy.external package

Submodules

hyperspy.external.progressbar module

hyperspy.external.progressbar.progressbar(*args, **kwargs)

Uses tqdm progressbar. This function exists for wrapping purposes only. Original docstring follows: —————————————-

Decorate an iterable object, returning an iterator which acts exactly like the original iterable, but prints a dynamically updating progressbar every time a value is requested.

iterable : iterable, optional
Iterable to decorate with a progressbar. Leave blank to manually manage the updates.
desc : str, optional
Prefix for the progressbar.
total : int, optional
The number of expected iterations. If unspecified, len(iterable) is used if possible. As a last resort, only basic progress statistics are displayed (no ETA, no progressbar). If gui is True and this parameter needs subsequent updating, specify an initial arbitrary large positive integer, e.g. int(9e9).
leave : bool, optional
If [default: True], keeps all traces of the progressbar upon termination of iteration.
file : io.TextIOWrapper or io.StringIO, optional
Specifies where to output the progress messages [default: sys.stderr]. Uses file.write(str) and file.flush() methods.
ncols : int, optional
The width of the entire output message. If specified, dynamically resizes the progressbar to stay within this bound. If unspecified, attempts to use environment width. The fallback is a meter width of 10 and no limit for the counter and statistics. If 0, will not print any meter (only stats).
mininterval : float, optional
Minimum progress update interval, in seconds [default: 0.1].
maxinterval : float, optional
Maximum progress update interval, in seconds [default: 10.0].
miniters : int, optional
Minimum progress update interval, in iterations. If specified, will set mininterval to 0.
ascii : bool, optional
If unspecified or False, use unicode (smooth blocks) to fill the meter. The fallback is to use ASCII characters 1-9 #.
disable : bool, optional
Whether to disable the entire progressbar wrapper [default: False].
unit : str, optional
String that will be used to define the unit of each iteration [default: it].
unit_scale : bool, optional
If set, the number of iterations will be reduced/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo, mega, etc.) [default: False].
dynamic_ncols : bool, optional
If set, constantly alters ncols to the environment (allowing for window resizes) [default: False].
smoothing : float, optional
Exponential moving average smoothing factor for speed estimates (ignored in GUI mode). Ranges from 0 (average speed) to 1 (current/instantaneous speed) [default: 0.3].
bar_format : str, optional
Specify a custom bar string formatting. May impact performance. If unspecified, will use ‘{l_bar}{bar}{r_bar}’, where l_bar is ‘{desc}{percentage:3.0f}%|’ and r_bar is ‘| {n_fmt}/{total_fmt} [{elapsed_str}<{remaining_str}, {rate_fmt}]’ Possible vars: bar, n, n_fmt, total, total_fmt, percentage, rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc.
initial : int, optional
The initial counter value. Useful when restarting a progress bar [default: 0].
position : int, optional
Specify the line offset to print this bar (starting from 0) Automatic if unspecified. Useful to manage multiple bars at once (eg, from threads).
gui : bool, optional
WARNING: internal parameter - do not use. Use tqdm_gui(...) instead. If set, will attempt to use matplotlib animations for a graphical output [default: False].

out : decorated iterator.

hyperspy.external.tifffile module

Read image and meta data from (bio)TIFF files. Save numpy arrays as TIFF.

Image and metadata can be read from TIFF, BigTIFF, OME-TIFF, STK, LSM, NIH, SGI, ImageJ, MicroManager, FluoView, SEQ and GEL files. Only a subset of the TIFF specification is supported, mainly uncompressed and losslessly compressed 2**(0 to 6) bit integer, 16, 32 and 64-bit float, grayscale and RGB(A) images, which are commonly used in bio-scientific imaging. Specifically, reading JPEG and CCITT compressed image data, chroma subsampling, or EXIF, IPTC, GPS, and XMP metadata is not implemented. Only primary info records are read for STK, FluoView, MicroManager, and NIH Image formats.

TIFF, the Tagged Image File Format aka Thousands of Incompatible File Formats, is under the control of Adobe Systems. BigTIFF allows for files greater than 4 GB. STK, LSM, FluoView, SGI, SEQ, GEL, and OME-TIFF, are custom extensions defined by Molecular Devices (Universal Imaging Corporation), Carl Zeiss MicroImaging, Olympus, Silicon Graphics International, Media Cybernetics, Molecular Dynamics, and the Open Microscopy Environment consortium respectively.

For command line usage run python tifffile.py –help

Author:Christoph Gohlke
Organization:Laboratory for Fluorescence Dynamics, University of California, Irvine
Version:2016.06.21

Requirements

Revisions

2016.06.21
Do not always memmap contiguous data in page series.
2016.05.13
Add option to specify resolution unit. Write grayscale images with extra samples when planarconfig is specified. Do not write RGB color images with 2 samples. Reorder TiffWriter.save keyword arguments (backwards incompatible).
2016.04.18
Pass 1932 tests. TiffWriter, imread, and imsave accept open binary file streams.
2016.04.13
Correctly handle reversed fill order in 2 and 4 bps images (bug fix). Implement reverse_bitorder in C.
2016.03.18
Fixed saving additional ImageJ metadata.
2016.02.22
Pass 1920 tests. Write 8 bytes double tag values using offset if necessary (bug fix). Add option to disable writing second image description tag. Detect tags with incorrect counts. Disable color mapping for LSM.
2015.11.13
Read LSM 6 mosaics. Add option to specify directory of memory-mapped files. Add command line options to specify vmin and vmax values for colormapping.
2015.10.06
New helper function to apply colormaps. Renamed is_palette attributes to is_indexed (backwards incompatible). Color-mapped samples are now contiguous (backwards incompatible). Do not color-map ImageJ hyperstacks (backwards incompatible). Towards supporting Leica SCN.
2015.09.25
Read images with reversed bit order (fill_order is lsb2msb).
2015.09.21
Read RGB OME-TIFF. Warn about malformed OME-XML.
2015.09.16
Detect some corrupted ImageJ metadata. Better axes labels for ‘shaped’ files. Do not create TiffTags for default values. Chroma subsampling is not supported. Memory-map data in TiffPageSeries if possible (optional).
2015.08.17
Pass 1906 tests. Write ImageJ hyperstacks (optional). Read and write LZMA compressed data. Specify datetime when saving (optional). Save tiled and color-mapped images (optional). Ignore void byte_counts and offsets if possible. Ignore bogus image_depth tag created by ISS Vista software. Decode floating point horizontal differencing (not tiled). Save image data contiguously if possible. Only read first IFD from ImageJ files if possible. Read ImageJ ‘raw’ format (files larger than 4 GB). TiffPageSeries class for pages with compatible shape and data type. Try to read incomplete tiles. Open file dialog if no filename is passed on command line. Ignore errors when decoding OME-XML. Rename decoder functions (backwards incompatible)
2014.08.24
TiffWriter class for incremental writing images. Simplified examples.
2014.08.19
Add memmap function to FileHandle. Add function to determine if image data in TiffPage is memory-mappable. Do not close files if multifile_close parameter is False.
2014.08.10
Pass 1730 tests. Return all extrasamples by default (backwards incompatible). Read data from series of pages into memory-mapped array (optional). Squeeze OME dimensions (backwards incompatible). Workaround missing EOI code in strips. Support image and tile depth tags (SGI extension). Better handling of STK/UIC tags (backwards incompatible). Disable color mapping for STK. Julian to datetime converter. TIFF ASCII type may be NULL separated. Unwrap strip offsets for LSM files greater than 4 GB. Correct strip byte counts in compressed LSM files. Skip missing files in OME series. Read embedded TIFF files.
2014.02.05
Save rational numbers as type 5 (bug fix).
2013.12.20
Keep other files in OME multi-file series closed. FileHandle class to abstract binary file handle. Disable color mapping for bad OME-TIFF produced by bio-formats. Read bad OME-XML produced by ImageJ when cropping.
2013.11.03
Allow zlib compress data in imsave function (optional). Memory-map contiguous image data (optional).
2013.10.28
Read MicroManager metadata and little endian ImageJ tag. Save extra tags in imsave function. Save tags in ascending order by code (bug fix).
2012.10.18
Accept file like objects (read from OIB files).
2012.08.21
Rename TIFFfile to TiffFile and TIFFpage to TiffPage. TiffSequence class for reading sequence of TIFF files. Read UltraQuant tags. Allow float numbers as resolution in imsave function.
2012.08.03
Read MD GEL tags and NIH Image header.
2012.07.25
Read ImageJ tags. ...

Notes

The API is not stable yet and might change between revisions.

Tested on little-endian platforms only.

Other Python packages and modules for reading bio-scientific TIFF files:

Acknowledgements

  • Egor Zindy, University of Manchester, for cz_lsm_scan_info specifics.
  • Wim Lewis for a bug fix and some read_cz_lsm functions.
  • Hadrien Mary for help on reading MicroManager files.
  • Christian Kliche for help writing tiled and color-mapped files.

References

  1. TIFF 6.0 Specification and Supplements. Adobe Systems Incorporated. http://partners.adobe.com/public/developer/tiff/
  2. TIFF File Format FAQ. http://www.awaresystems.be/imaging/tiff/faq.html
  3. MetaMorph Stack (STK) Image File Format. http://support.meta.moleculardevices.com/docs/t10243.pdf
  4. Image File Format Description LSM 5/7 Release 6.0 (ZEN 2010). Carl Zeiss MicroImaging GmbH. BioSciences. May 10, 2011
  5. File Format Description - LSM 5xx Release 2.0. http://ibb.gsf.de/homepage/karsten.rodenacker/IDL/Lsmfile.doc
  6. The OME-TIFF format. http://www.openmicroscopy.org/site/support/file-formats/ome-tiff
  7. UltraQuant(r) Version 6.0 for Windows Start-Up Guide. http://www.ultralum.com/images%20ultralum/pdf/UQStart%20Up%20Guide.pdf
  8. Micro-Manager File Formats. http://www.micro-manager.org/wiki/Micro-Manager_File_Formats
  9. Tags for TIFF and Related Specifications. Digital Preservation. http://www.digitalpreservation.gov/formats/content/tiff_tags.shtml

Examples

>>> data = numpy.random.rand(5, 301, 219)
>>> imsave('temp.tif', data)

>>> image = imread('temp.tif')
>>> numpy.testing.assert_array_equal(image, data)

>>> with TiffFile('temp.tif') as tif:
...     images = tif.asarray()
...     for page in tif:
...         for tag in page.tags.values():
...             t = tag.name, tag.value
...         image = page.asarray()
hyperspy.external.tifffile.imsave(file, data, **kwargs)

Write image data to TIFF file.

Refer to the TiffWriter class and member functions for documentation.

Parameters:
  • file (str or binary stream) – File name or writable binary stream, such as a open file or BytesIO.
  • data (array_like) – Input image. The last dimensions are assumed to be image depth, height, width, and samples.
  • kwargs (dict) – Parameters ‘byteorder’, ‘bigtiff’, ‘software’, and ‘imagej’, are passed to the TiffWriter class. Parameters ‘photometric’, ‘planarconfig’, ‘resolution’, ‘compress’, ‘colormap’, ‘tile’, ‘description’, ‘datetime’, ‘metadata’, ‘contiguous’ and ‘extratags’ are passed to the TiffWriter.save function.

Examples

>>> data = numpy.random.rand(2, 5, 3, 301, 219)
>>> imsave('temp.tif', data, compress=6, metadata={'axes': 'TZCYX'})
hyperspy.external.tifffile.imread(files, **kwargs)

Return image data from TIFF file(s) as numpy array.

The first image series is returned if no arguments are provided.

Parameters:
  • files (str, binary stream, or sequence) – File name, seekable binary stream, glob pattern, or sequence of file names.
  • key (int, slice, or sequence of page indices) – Defines which pages to return as array.
  • series (int) – Defines which series of pages in file to return as array.
  • multifile (bool) – If True (default), OME-TIFF data may include pages from multiple files.
  • pattern (str) – Regular expression pattern that matches axes names and indices in file names.
  • kwargs (dict) – Additional parameters passed to the TiffFile or TiffSequence asarray function.

Examples

>>> imsave('temp.tif', numpy.random.rand(3, 4, 301, 219))
>>> im = imread('temp.tif', key=0)
>>> im.shape
(4, 301, 219)
>>> ims = imread(['temp.tif', 'temp.tif'])
>>> ims.shape
(2, 3, 4, 301, 219)
hyperspy.external.tifffile.imshow(data, title=None, vmin=0, vmax=None, cmap=None, bitspersample=None, photometric='rgb', interpolation='nearest', dpi=96, figure=None, subplot=111, maxdim=8192, **kwargs)

Plot n-dimensional images using matplotlib.pyplot.

Return figure, subplot and plot axis. Requires pyplot already imported from matplotlib import pyplot.

Parameters:
  • bitspersample (int or None) – Number of bits per channel in integer RGB images.
  • photometric ({'miniswhite', 'minisblack', 'rgb', or 'palette'}) – The color space of the image data.
  • title (str) – Window and subplot title.
  • figure (matplotlib.figure.Figure (optional).) – Matplotlib to use for plotting.
  • subplot (int) – A matplotlib.pyplot.subplot axis.
  • maxdim (int) – maximum image width and length.
  • kwargs (optional) – Arguments for matplotlib.pyplot.imshow.
class hyperspy.external.tifffile.TiffFile(arg, name=None, offset=None, size=None, multifile=True, multifile_close=True, maxpages=None, fastij=True)

Bases: object

Read image and metadata from TIFF, STK, LSM, and FluoView files.

TiffFile instances must be closed using the ‘close’ method, which is automatically called when using the ‘with’ context manager.

pages

list of TiffPage – All TIFF pages in file.

series

list of TiffPageSeries – TIFF pages with compatible shapes and types.

micromanager_metadata

dict – Extra MicroManager non-TIFF metadata in the file, if exists.

All attributes are read-only.

Examples

>>> with TiffFile('temp.tif') as tif:
...     data = tif.asarray()
...     data.shape
(5, 301, 219)
asarray(key=None, series=None, memmap=False, tempdir=None)

Return image data from multiple TIFF pages as numpy array.

By default the first image series is returned.

Parameters:
  • key (int, slice, or sequence of page indices) – Defines which pages to return as array.
  • series (int or TiffPageSeries) – Defines which series of pages to return as array.
  • memmap (bool) – If True, return an read-only array stored in a binary file on disk if possible. The TIFF file is used if possible, else a temporary file is created.
  • tempdir (str) – The directory where the memory-mapped file will be created.
close()

Close open file handle(s).

filehandle

Return file handle.

filename

Return name of file handle.

fstat

Lazy object attribute whose value is computed on first access.

is_bigtiff

Lazy object attribute whose value is computed on first access.

is_fluoview

Lazy object attribute whose value is computed on first access.

is_imagej

Lazy object attribute whose value is computed on first access.

is_indexed

Lazy object attribute whose value is computed on first access.

is_lsm

Lazy object attribute whose value is computed on first access.

is_mdgel

Lazy object attribute whose value is computed on first access.

is_mediacy

Lazy object attribute whose value is computed on first access.

is_micromanager

Lazy object attribute whose value is computed on first access.

is_nih

Lazy object attribute whose value is computed on first access.

is_ome

Lazy object attribute whose value is computed on first access.

is_rgb

Lazy object attribute whose value is computed on first access.

is_scn

Lazy object attribute whose value is computed on first access.

is_stk

Lazy object attribute whose value is computed on first access.

is_vista

Lazy object attribute whose value is computed on first access.

series

Lazy object attribute whose value is computed on first access.

class hyperspy.external.tifffile.TiffWriter(file, bigtiff=False, byteorder=None, software='tifffile.py', imagej=False)

Bases: object

Write image data to TIFF file.

TiffWriter instances must be closed using the ‘close’ method, which is automatically called when using the ‘with’ context manager.

Examples

>>> data = numpy.random.rand(2, 5, 3, 301, 219)
>>> with TiffWriter('temp.tif', bigtiff=True) as tif:
...     for i in range(data.shape[0]):
...         tif.save(data[i], compress=6)
TAGS = {'smax_sample_value': 341, 'image_width': 256, 'photometric': 262, 'orientation': 274, 'rows_per_strip': 278, 'image_length': 257, 'tile_depth': 32998, 'bits_per_sample': 258, 'extra_samples': 338, 'tile_width': 322, 'planar_configuration': 284, 'image_depth': 32997, 'tile_offsets': 324, 'software': 305, 'predictor': 317, 'datetime': 306, 'image_description': 270, 'y_resolution': 283, 'smin_sample_value': 340, 'color_map': 320, 'resolution_unit': 296, 'page_name': 285, 'strip_byte_counts': 279, 'sample_format': 339, 'document_name': 269, 'samples_per_pixel': 277, 'tile_byte_counts': 325, 'subfile_type': 255, 'compression': 259, 'new_subfile_type': 254, 'tile_length': 323, 'x_resolution': 282, 'strip_offsets': 273}
TYPES = {'s': 2, 'B': 1, 'h': 8, 'I': 4, 'b': 6, '2I': 5, 'd': 12, 'H': 3, 'f': 11, 'Q': 16, 'i': 9, 'q': 17}
close(truncate=False)

Write remaining pages (if not truncate) and close file handle.

save(data, photometric=None, planarconfig=None, tile=None, contiguous=True, compress=0, colormap=None, description=None, datetime=None, resolution=None, metadata={}, extratags=())

Write image data and tags to TIFF file.

Image data are written in one stripe per plane by default. Dimensions larger than 2 to 4 (depending on photometric mode, planar configuration, and SGI mode) are flattened and saved as separate pages. The ‘sample_format’ and ‘bits_per_sample’ tags are derived from the data type.

Parameters:
  • data (numpy.ndarray) – Input image. The last dimensions are assumed to be image depth, height (length), width, and samples. If a colormap is provided, the dtype must be uint8 or uint16 and the data values are indices into the last dimension of the colormap.
  • photometric ({'minisblack', 'miniswhite', 'rgb', 'palette'}) – The color space of the image data. By default this setting is inferred from the data shape and the value of colormap.
  • planarconfig ({'contig', 'planar'}) – Specifies if samples are stored contiguous or in separate planes. By default this setting is inferred from the data shape. If this parameter is set, extra samples are used to store grayscale images. ‘contig’: last dimension contains samples. ‘planar’: third last dimension contains samples.
  • tile (tuple of int) – The shape (depth, length, width) of image tiles to write. If None (default), image data are written in one stripe per plane. The tile length and width must be a multiple of 16. If the tile depth is provided, the SGI image_depth and tile_depth tags are used to save volume data. Few software can read the SGI format, e.g. MeVisLab.
  • contiguous (bool) – If True (default) and the data and parameters are compatible with previous ones, if any, the data are stored contiguously after the previous one. Parameters ‘photometric’ and ‘planarconfig’ are ignored.
  • compress (int or 'lzma') – Values from 0 to 9 controlling the level of zlib compression. If 0, data are written uncompressed (default). Compression cannot be used to write contiguous files. If ‘lzma’, LZMA compression is used, which is not available on all platforms.
  • colormap (numpy.ndarray) – RGB color values for the corresponding data value. Must be of shape (3, 2**(data.itemsize*8)) and dtype uint16.
  • description (str) – The subject of the image. Saved with the first page only. Cannot be used with the ImageJ format.
  • datetime (datetime) – Date and time of image creation. Saved with the first page only. If None (default), the current date and time is used.
  • resolution ((float, float[, str]) or ((int, int), (int, int)[, str])) – X and Y resolutions in pixels per resolution unit as float or rational numbers. A third, optional parameter specifies the resolution unit, which must be None (default for ImageJ), ‘inch’ (default), or ‘cm’.
  • metadata (dict) – Additional meta data to be saved along with shape information in JSON or ImageJ formats in an image_description tag. If None, do not write a second image_description tag.
  • extratags (sequence of tuples) –

    Additional tags as [(code, dtype, count, value, writeonce)].

    code : int
    The TIFF tag Id.
    dtype : str
    Data type of items in ‘value’ in Python struct format. One of B, s, H, I, 2I, b, h, i, f, d, Q, or q.
    count : int
    Number of data values. Not used for string values.
    value : sequence
    ‘Count’ values compatible with ‘dtype’.
    writeonce : bool
    If True, the tag is written to the first page only.
class hyperspy.external.tifffile.TiffSequence(files, imread=<class 'hyperspy.external.tifffile.TiffFile'>, pattern='axes', *args, **kwargs)

Bases: object

Sequence of image files.

The data shape and dtype of all files must match.

files

list – List of file names.

shape

tuple – Shape of image sequence.

axes

str – Labels of axes in shape.

Examples

>>> tifs = TiffSequence("test.oif.files/*.tif")
>>> tifs.shape, tifs.axes
((2, 100), 'CT')
>>> data = tifs.asarray()
>>> data.shape
(2, 100, 256, 256)
exception ParseError

Bases: Exception

TiffSequence.asarray(memmap=False, tempdir=None, *args, **kwargs)

Read image data from all files and return as single numpy array.

If memmap is True, return an array stored in a binary file on disk. The args and kwargs parameters are passed to the imread function.

Raise IndexError or ValueError if image shapes do not match.

TiffSequence.close()
class hyperspy.external.tifffile.FileHandle(file, mode='rb', name=None, offset=None, size=None)

Bases: object

Binary file handle.

A limited, special purpose file handler that:

  • handles embedded files (for CZI within CZI files)
  • allows to re-open closed files (for multi file formats, such as OME-TIFF)
  • reads and writes numpy arrays and records from file like objects

Only ‘rb’ and ‘wb’ modes are supported. Concurrently reading and writing of the same stream is untested.

When initialized from another file handle, do not use it unless this FileHandle is closed.

name

str – Name of the file.

path

str – Absolute path to file.

size

int – Size of file in bytes.

is_file

bool – If True, file has a filno and can be memory-mapped.

All attributes are read-only.
close()

Close file.

closed
dirname
flush()

Flush write buffers if applicable.

is_file
memmap_array(dtype, shape, offset=0, mode='r', order='C')

Return numpy.memmap of data stored in file.

name
open()

Open or re-open file.

path
read(size=-1)

Read ‘size’ bytes from file, or until EOF is reached.

read_array(dtype, count=-1, sep='')

Return numpy array from file.

Work around numpy issue #2230, “numpy.fromfile does not accept StringIO object” https://github.com/numpy/numpy/issues/2230.

read_record(dtype, shape=1, byteorder=None)

Return numpy record from file.

seek(offset, whence=0)

Set file’s current position.

size
tell()

Return file’s current position.

write(bytestring)

Write bytestring to file.

write_array(data)

Write numpy array to binary file.

class hyperspy.external.tifffile.lazyattr(func)

Bases: object

Lazy object attribute whose value is computed on first access.

func
hyperspy.external.tifffile.natural_sorted(iterable)

Return human sorted list of strings.

E.g. for sorting file names.

>>> natural_sorted(['f1', 'f2', 'f10'])
['f1', 'f2', 'f10']
hyperspy.external.tifffile.decode_lzw(encoded)

Decompress LZW (Lempel-Ziv-Welch) encoded TIFF strip (byte string).

The strip must begin with a CLEAR code and end with an EOI code.

This is an implementation of the LZW decoding algorithm described in (1). It is not compatible with old style LZW compressed files like quad-lzw.tif.

hyperspy.external.tifffile.stripnull(string)

Return string truncated at first null character.

Clean NULL terminated C strings.

>>> stripnull(b'string\x00')
b'string'

Module contents