hyperspy.misc.utils module

class hyperspy.misc.utils.DictionaryTreeBrowser(dictionary=None, double_lines=False, lazy=True)

Bases: object

A class to comfortably browse a dictionary using a CLI.

In addition to accessing the values using dictionary syntax the class enables navigating a dictionary that constains nested dictionaries as attribures of nested classes. Also it is an iterator over the (key, value) items. The __repr__ method provides pretty tree printing. Private keys, i.e. keys that starts with an underscore, are not printed, counted when calling len nor iterated.

export : saves the dictionary in pretty tree printing format in a text: file.

keys : returns a list of non-private keys.

as_dictionary : returns a dictionary representation of the object.

set_item : easily set items, creating any necessary nodes on the way.

has_item: given a path, or part of a path, checks if the item exists.

get_item given a path, or part of a path, return the value of the item.

add_node : add all non existing nodes in a given path.

add_dictionary: add new items from dictionary.

Examples

>>> tree = DictionaryTreeBrowser()
>>> tree.set_item("Branch.Leaf1.color", "green")
>>> tree.set_item("Branch.Leaf2.color", "brown")
>>> tree.set_item("Branch.Leaf2.caterpillar", True)
>>> tree.set_item("Branch.Leaf1.caterpillar", False)
>>> tree
└── Branch
    ├── Leaf1
    │   ├── caterpillar = False
    │   └── color = green
    └── Leaf2
        ├── caterpillar = True
        └── color = brown
>>> tree.Branch
├── Leaf1
│   ├── caterpillar = False
│   └── color = green
└── Leaf2
    ├── caterpillar = True
    └── color = brown
>>> for label, leaf in tree.Branch:
...     print("%s is %s" % (label, leaf.color))
Leaf1 is green
Leaf2 is brown
>>> tree.Branch.Leaf2.caterpillar
True
>>> "Leaf1" in tree.Branch
True
>>> "Leaf3" in tree.Branch
False
>>>

When creating a DictionaryTreeBrowser lazily, the dictionary is added to the _lazy_attributes attribute. The first time a lazy attribute is called or the DictionaryTreeBrowser is printed, the DictionaryTreeBrowser processes the lazy attributes with the process_lazy_attributes method. DictionaryTreeBrowser is lazy by default, using non-lazy instances can be useful for debugging purposes.

_get_html_print_items(padding='', max_len=78, recursive_level=0): Recursive method that creates a html string for fancy display of metadata.

_get_print_items(padding='', max_len=78): Prints only the attributes that are not methods

_nested_get(item_path, wild=False, return_path=False): Search for an item key in a nested DictionaryTreeBrowser and yield a list of values. If wild is True, looks for any key that contains the string item (case insensitive). If part of a path is given, search for matching items and then make sure that the full path is contained.

_nested_get_iter(item, wild=False): Recursive function to search for an item key in a nested DictionaryTreeBrowser.

_process_dictionary(dictionary, double_lines): Process the provided dictionary to set the attributes

add_dictionary(dictionary, double_lines=False): Add new items from dictionary.

add_node(node_path)

Adds all the nodes in the given path if they don’t exist.

Parameters:: node_path (str) – The nodes must be separated by full stops (periods).

Examples

>>> dict_browser = DictionaryTreeBrowser({})
>>> dict_browser.add_node('First.Second')
>>> dict_browser.First.Second = 3
>>> dict_browser
└── First
    └── Second = 3

as_dictionary(): Returns its dictionary representation.

export(filename, encoding='utf8')

Export the dictionary to a text file

Parameters:

filename (str) – The name of the file without the extension that is txt by default
encoding (valid encoding str) –

get_item(item_path, default=None, full_path=True, wild=False, return_path=False)

Given a path, return it’s value if it exists, or default value if missing. May also perform a search whether an item key exists and then returns the value or a list of values for multiple occurences of the key – optionally returns the full path(s) in addition to its value(s).

The nodes of the path are separated using periods.

Parameters:

item_path (Str) – A string describing the path with each item separated by full stops (periods)
full_path (boolean) – If True (default), the full path to the item has to be given. If False, a search for the item key is performed (can include additional nodes preceding they key separated by full stops).
wild (boolean) – Only applies if full_path=False. If True, searches for any items where item matches a substring of the item key (case insensitive). Default is False.
return_path (boolean) – Only applies if full_path=False. Default False. If True, returns an additional list of paths to the item(s) that match key.
default – The value to return if the path or item does not exist. (default is None).

Examples

>>> dict = {'To' : {'be' : True}}
>>> dict_browser = DictionaryTreeBrowser(dict)
>>> dict_browser.get_item('To')
└── be = True
>>> dict_browser.get_item('To.be')
True
>>> dict_browser.get_item('To.be.or', 'default_value')
'default_value'
>>> dict_browser.get_nested_item('be')
True

has_item(item_path, default=None, full_path=True, wild=False, return_path=False)

Given a path, return True if it exists. May also perform a search whether an item exists and optionally returns the full path instead of boolean value.

The nodes of the path are separated using periods.

Parameters:

item_path (Str) – A string describing the path with each item separated by full stops (periods).
full_path (boolean) – If True (default), the full path to the item has to be given. If False, a search for the item key is performed (can include additional nodes preceding they key separated by full stops).
wild (boolean) – Only applies if full_path=False. If True, searches for any items where item matches a substring of the item key (case insensitive). Default is False.
return_path (boolean) – Only applies if full_path=False. If False (default), a boolean value is returned. If True, the full path to the item is returned, a list of paths for multiple matches, or default value if it does not exist.
default – The value to return for path if the item does not exist (default is None).

Examples

>>> dict = {'To' : {'be' : True}}
>>> dict_browser = DictionaryTreeBrowser(dict)
>>> dict_browser.has_item('To')
True
>>> dict_browser.has_item('To.be')
True
>>> dict_browser.has_item('To.be.or')
False
>>> dict_browser.has_item('be', full_path=False)
True
>>> dict_browser.has_item('be', full_path=False, return_path=True)
'To.be'

keys(): Returns a list of non-private keys.

process_lazy_attributes(): Run the DictionaryTreeBrowser machinery for the lazy attributes.

set_item(item_path, value)

Given the path and value, create the missing nodes in the path and assign to the last one the value

Parameters:: item_path (Str) – A string describing the path with each item separated by a full stops (periods)

Examples

>>> dict_browser = DictionaryTreeBrowser({})
>>> dict_browser.set_item('First.Second.Third', 3)
>>> dict_browser
└── First
   └── Second
        └── Third = 3

hyperspy.misc.utils._get_block_pattern(args, output_shape)

Returns the block pattern used by the blockwise function for a set of arguments give a resulting output_shape

Parameters:

args (list) – A list of all the arguments which are used for da.blockwise
output_shape (tuple) – The output shape for the function passed to da.blockwise given args

hyperspy.misc.utils.add_key_value(key, value): Returns the metadata value as a html string

hyperspy.misc.utils.attrsetter(target, attrs, value)

Sets attribute of the target to specified value, supports nested attributes. Only creates a new attribute if the object supports such behaviour (e.g. DictionaryTreeBrowser does)

Parameters:

target (object) –
attrs (string) – attributes, separated by periods (e.g. ‘metadata.Signal.Noise_parameters.variance’ )
value (object) –

Example

First create a signal and model pair:

>>> s = hs.signals.Signal1D(np.arange(10))
>>> m = s.create_model()
>>> m.signal.data
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

Now set the data of the model with attrsetter >>> attrsetter(m, ‘signal1D.data’, np.arange(10)+2) >>> self.signal.data array([2, 3, 4, 5, 6, 7, 8, 9, 10, 10])

The behaviour is identical to >>> self.signal.data = np.arange(10) + 2

hyperspy.misc.utils.check_long_string(value, max_len): Checks whether string is too long for printing in html metadata

hyperspy.misc.utils.closest_power_of_two(n): Returns next higher power of two, not the closest one as its name suggests.

hyperspy.misc.utils.find_subclasses(mod, cls)

Find all the subclasses in a module.

Parameters:

mod (module) –
cls (class) –

Return type:

dictonary in which key, item = subclass name, subclass

hyperspy.misc.utils.fsdict(nodes, value, dictionary)

Populates the dictionary ‘dic’ in a file system-like fashion creating a dictionary of dictionaries from the items present in the list ‘nodes’ and assigning the value ‘value’ to the innermost dictionary.

‘dic’ will be of the type: dic[‘node1’][‘node2’][‘node3’]…[‘nodeN’] = value where each node is like a directory that contains other directories (nodes) or files (values)

hyperspy.misc.utils.get_array_module(array)

Returns the array module for the given array

Parameters:: array (numpy or cupy array) – Array to determine whether numpy or cupy should be used
Returns:: module
Return type:: module

hyperspy.misc.utils.get_numpy_kwargs(array): Convenience funtion to return a dictionary containing the like keyword if numpy>=1.20.

Note

like keyword is an experimental feature introduced in numpy 1.20 and is pending on acceptance of NEP 35

hyperspy.misc.utils.get_object_package_info(obj)

Get info about object package

Returns:: dic – Dictionary containing package and package_version (if available)
Return type:: dict

hyperspy.misc.utils.guess_output_signal_size(test_data, function, ragged, **kwargs)

This function is for guessing the output signal shape and size. It will attempt to apply the function to some test data and then output the resulting signal shape and datatype.

Parameters:

test_data (NumPy array) – Data from a test signal for the function to be applied to. The data must be from a signal with 0 navigation dimensions.
function (function) – The function to be applied to the data
ragged (bool) – If the data is ragged then the output signal size is () and the data type is ‘object’
**kwargs (dict) – Any other keyword arguments passed to the function.

hyperspy.misc.utils.is_binned(signal, axis=-1)

Backwards compatibility check utility for is_binned attribute.

Can be removed in v2.0.

hyperspy.misc.utils.is_cupy_array(array)

Convenience function to determine if an array is a cupy array

Parameters:: array (array) – The array to determine whether it is a cupy array or not.
Returns:: True if it is cupy array, False otherwise.
Return type:: bool

hyperspy.misc.utils.is_hyperspy_signal(input_object)

Check if an object is a Hyperspy Signal

Parameters:: input_object (object) – Object to be tests
Returns:: If true the object is a subclass of hyperspy.signal.BaseSignal
Return type:: bool

hyperspy.misc.utils.multiply(iterable)

Return product of sequence of numbers.

Equivalent of functools.reduce(operator.mul, iterable, 1).

>>> product([2**8, 2**30])
274877906944
>>> product([])
1

hyperspy.misc.utils.nested_dictionary_merge(dict1, dict2): Merge dict2 into dict1 recursively

hyperspy.misc.utils.ordinal(value)

Converts zero or a postive integer (or their string representations) to an ordinal value.

>>> for i in range(1,13):
...     ordinal(i)
...
'1st'
'2nd'
'3rd'
'4th'
'5th'
'6th'
'7th'
'8th'
'9th'
'10th'
'11th'
'12th'

>>> for i in (100, '111', '112',1011):
...     ordinal(i)
...
'100th'
'111th'
'112th'
'1011th'

Notes

Author: Serdar Tumgoren https://code.activestate.com/recipes/576888-format-a-number-as-an-ordinal/ MIT license

hyperspy.misc.utils.parse_quantity(quantity, opening='(', closing=')')

Parse quantity of the signal outputting quantity and units separately. It looks for the last matching opening and closing separator.

Parameters:

quantity (string) –
opening (string) – Separator used to define the beginning of the units
closing (string) – Separator used to define the end of the units

Returns:

quantity_name (string)
quantity_units (string)

hyperspy.misc.utils.print_html(f_text, f_html): Print html version when in Jupyter Notebook

hyperspy.misc.utils.process_function_blockwise(data, *args, function, nav_indexes=None, output_signal_size=None, output_dtype=None, arg_keys=None, **kwargs)

Convenience function for processing a function blockwise. By design, its output is used as an argument of the dask map_blocks so that the function only gets applied to the signal axes.

Parameters:

data (np.ndarray) – The data for one chunk
*args (tuple) – Any signal the is iterated alongside the data in. In the form ((key1, value1), (key2, value2))
function (function) – The function to applied to the signal axis
nav_indexes (tuple) – The indexes of the navigation axes for the dataset.
output_signal_size (tuple) – The shape of the output signal. For a ragged signal, this is equal to 1
output_dtype (dtype) – The data type for the output.
arg_keys (tuple) – The list of keys for the passed arguments (args). Together this makes a set of key:value pairs to be passed to the function.
**kwargs (dict) – Any additional key value pairs to be used by the function (Note that these are the constants that are applied.)

hyperspy.misc.utils.replace_html_symbols(str_value): Escapes any &, < and > tags that would become invisible when printing html

hyperspy.misc.utils.rollelem(a, index, to_index=0)

Roll the specified axis backwards, until it lies in a given position.

Parameters:

a (list) – Input list.
index (int) – The index of the item to roll backwards. The positions of the items do not change relative to one another.
to_index (int, optional) – The item is rolled until it lies before this position. The default, 0, results in a “complete” roll.

Returns:

res – Output list.

Return type:

list

hyperspy.misc.utils.slugify(value, valid_variable_name=False)

Normalizes string, converts to lowercase, removes non-alpha characters, and converts spaces to hyphens.

Adapted from Django’s “django/template/defaultfilters.py”.

hyperspy.misc.utils.stack(signal_list, axis=None, new_axis_name='stack_element', lazy=None, stack_metadata=True, show_progressbar=None, **kwargs)

Concatenate the signals in the list over a given axis or a new axis.

The title is set to that of the first signal in the list.

Parameters:

signal_list (list of BaseSignal instances) – List of signals to stack.
axis ({None, int, str}) – If None, the signals are stacked over a new axis. The data must have the same dimensions. Otherwise the signals are stacked over the axis given by its integer index or its name. The data must have the same shape, except in the dimension corresponding to axis. If the stacking axis of the first signal is uniform, it is extended up to the new length; if it is non-uniform, the axes vectors of all signals are concatenated along this direction; if it is a FunctionalDataAxis, it is extended based on the expression of the first signal (and its sub axis x is handled as above depending on whether it is uniform or not).
new_axis_name (str) – The name of the new axis when axis is None. If an axis with this name already exists it automatically append ‘-i’, where i are integers, until it finds a name that is not yet in use.
lazy ({bool, None}) – Returns a LazySignal if True. If None, only returns lazy result if at least one is lazy.
stack_metadata ({bool, int}) – If integer, this value defines the index of the signal in the signal list, from which the metadata and original_metadata are taken. If True, the original_metadata and metadata of each signals are stacked and saved in original_metadata.stack_elements of the returned signal. In this case, the metadata are copied from the first signal in the list. If False, the metadata and original_metadata are not copied.
show_progressbar (None or bool) – If True, display a progress bar. If None, the default from the preferences settings is used.

Returns:

signal

Return type:

BaseSignal instance

Examples

>>> data = np.arange(20)
>>> s = hs.stack([hs.signals.Signal1D(data[:10]),
...               hs.signals.Signal1D(data[10:])])
>>> s
<Signal1D, title: Stack of , dimensions: (2, 10)>
>>> s.data
array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]])

hyperspy.misc.utils.str2num(string, **kargs)

Transform a a table in string form into a numpy array

Parameters:: string (string) –
Return type:: numpy array

hyperspy.misc.utils.swapelem(obj, i, j)

Swaps element having index i with element having index j in object obj IN PLACE.

Example

>>> L = ['a', 'b', 'c']
>>> swapelem(L, 1, 2)
>>> print(L)
['a', 'c', 'b']

hyperspy.misc.utils.to_numpy(array)

Returns the array as a numpy array. Raises an error when a dask array is provided.

Parameters:: array (numpy.ndarray or cupy.ndarray) – Array to convert to numpy array.
Returns:: array
Return type:: numpy.ndarray
Raises:: TypeError – When the input array type is not supported.

hyperspy.misc.utils.transpose(*args, signal_axes=None, navigation_axes=None, optimize=False)

Transposes all passed signals according to the specified options.

For parameters see BaseSignal.transpose.

Examples

>>> signal_iterable = [hs.signals.BaseSignal(np.random.random((2,)*(i+1)))
                       for i in range(3)]
>>> signal_iterable
[<BaseSignal, title: , dimensions: (|2)>,
 <BaseSignal, title: , dimensions: (|2, 2)>,
 <BaseSignal, title: , dimensions: (|2, 2, 2)>]
>>> hs.transpose(*signal_iterable, signal_axes=1)
[<BaseSignal, title: , dimensions: (|2)>,
 <BaseSignal, title: , dimensions: (2|2)>,
 <BaseSignal, title: , dimensions: (2, 2|2)>]
>>> hs.transpose(signal1, signal2, signal3, signal_axes=["Energy"])

hyperspy.misc.utils.underline(line, character='-'): Return the line underlined.