Writing packages that extend HyperSpy

New in version 1.5: External packages can extend HyperSpy by registering signals, components and widgets.

Warning

The mechanism to register extensions is in beta state. This means that it can change between minor and patch versions. Therefore, if you maintain a package that registers HyperSpy extensions, please verify that it works properly with any future HyperSpy release. We expect it to reach maturity with the release of HyperSpy 2.0.

External packages can extend HyperSpy by registering signals, components and widgets. Objects registered by external packages are “first-class citizens” i.e. they can be used, saved and loaded like any of those objects shipped with HyperSpy. Because of HyperSpy’s structure, we anticipate that most packages registering HyperSpy extensions will provide support for specific sorts of data.

Models can be provided by external packages too but don’t need to be registered. Instead, they are returned by the create_model method of the relevant hyperspy.signal.BaseSignal subclass, see for example, the hyperspy._signals.eds_tem.EDSTEM_mixin.create_model() of the EDSTEMSpectrum.

It is good practice to add all packages that extend HyperSpy to the list of known extensions regardless their maturity level. In this way we can avoid duplication of efforts and issues arising from naming conflicts.

At this point it is worth noting that HyperSpy’s main strength is its amazing community of users and developers. We trust that the developers of packages that extend HyperSpy will play by the same rules that have made the Python scientific ecosystem successful. In particular, avoiding duplication of efforts and being good community players by contributing code to the best matching project are essential for the sustainability of our software ecosystem.

Registering extensions

In order to register HyperSpy extensions you need to:

  1. Add the following line to your package’s setup.py:

    entry_points={'hyperspy.extensions': 'your_package_name =
              your_package_name'},

  2. Create a hyperspy_extension.yaml configuration file in your module’s root directory.

  3. Declare all new HyperSpy objects provided by your package in the hyperspy_extension.yaml file.

For a full example on how to create a package that extends HyperSpy see the HyperSpy Sample Extension package.

Creating new HyperSpy BaseSignal subclasses

When and where create a new BaseSignal subclass

HyperSpy provides most of its functionality through the different hyperspy.signal.BaseSignal subclasses. A HyperSpy “signal” is a class that contains data for analysis and functions to perform the analysis in the form of class methods. Functions that are useful for the analysis of most datasets are in the hyperspy.signal.BaseSignal class. All other functions are in specialized subclasses.

The flowchart below can help you decide where to add a new data analysis function. Notice that only if no suitable package exists for your function you should consider creating your own.

digraph G { A [label="New function needed"] B [label="Is it useful for data of any type and dimensions?",shape="diamond"] C [label="Contribute it to BaseSignal"] D [label="Does an SignalxD for the required dimension exist in HyperSpy",shape="diamond"] E [label="Contribute new SignalxD to HyperSpy"] F [label="Is the function useful only for some sort of data?",shape="diamond"] G [label="Contribute it to SignalxD"] H [label="Does an signal for that sort of data exists?",shape="diamond"] I [label="Contribute to package providing the relevant signal"] J [label="Create you own package and signal subclass to host the funtion"] A->B B->C [label="Yes"] B->D [label="No"] D->F [label="Yes"] D->E [label="No"] E->F F->H [label="Yes"] F->G [label="No"] H->I [label="Yes"] H->J [label="No"] }

Registering a new BaseSignal subclass

To register a new hyperspy.signal.BaseSignal subclass you must add it to the hyperspy_extension.yaml file as in the following example:

signals:
    MySignal:
        signal_type: "MySignal"
        signal_type_aliases:
        - MS
        - ThisIsMySignal
        # The dimension of the signal subspace. For example, 2 for images, 1 for
        # spectra. If the signal can take any signal dimension, set it to -1.
        signal_dimension: 1
        # The data type, "real" or "complex".
        dtype: real
        # True for LazySignal subclasses
        lazy: False
        # The module where the signal is located.
        module: my_package.signal

Note that HyperSpy uses signal_type to determine which class is the most appropriate to deal with a particular sort of data. Therefore, the signal type must be specific enough for HyperSpy to find a single signal subclass match for each sort of data.

Warning

HyperSpy assumes that only one signal subclass exists for a particular signal_type. It is up to external packages developers to avoid signal_type clashes, typically by collaborating in developing a single package per data type.

The optional signal_type_aliases are used to determine the most appropriate signal subclass when using hyperspy.signal.BaseSignal.set_signal_type(). For example, if the signal_type Electron Energy Loss Spectroscopy has an EELS alias, setting the signal type to EELS will correctly assign the signal subclass with Electron Energy Loss Spectroscopy signal type. It is good practice to choose a very explicit signal_type while leaving acronyms for signal_type_aliases.

Creating new HyperSpy model components

When and where create a new component

HyperSpy provides the hyperspy._components.expression.Expression component that enables easy creation of 1D and 2D components from mathematical expressions. Therefore, strictly speaking, we only need to create new components when they cannot be expressed as simple mathematical equations. However, HyperSpy is all about simplifying the interactive data processing workflow. Therefore, we consider that functions that are commonly used for model fitting, in general or specific domains, are worth adding to HyperSpy itself (if they are of common interest) or to specialized external packages extending HyperSpy.

The flowchart below can help you decide when and where to add a new hyperspy model hyperspy.component.Component. for your function you should consider creating your own.

digraph G { A [label="New component needed"] B [label="Can it be declared using Expression?",shape="diamond"] C [label="Can it be useful to other users?",shape="diamond"] D [label="Just use Expression"] E [label="Create new component using Expression"] F [label="Create new component from the scratch"] G [label="Is it useful for general users?",shape="diamond"] H [label="Contribute it to HyperSpy"] I [label="Does a suitable package for it exist?",shape="diamond"] J [label="Contribute it to the relevant package"] K [label="Create your own package to host it"] A->B B->C [label="Yes"] B->F [label="No"] C->E [label="Yes"] C->D [label="No"] E->G F->G G->H [label="Yes"] G->I [label="No"] I->J [label="Yes"] I->K [label="No"] }

Registering new components

All new components must be a subclass of hyperspy._components.expression.Expression. To register a new 1D component add it to the hyperspy_extension.yaml file as in the following example:

components1D:
  # _id_name of the component. It must be an UUID4. This can be generated
  # using ``uuid.uuid4()``. Also, many editors can automatically generate
  # UUIDs. The same UUID must be stored in the components ``_id_name`` attribute.
  fc731a2c-0a05-4acb-91df-d15743b531c3:
    # The module where the component class is located.
    module: my_package.components
    # The actual class of the component
    class: MyComponent1DClass

Equivalently, to add a new component 2D:

components2D:
  # _id_name of the component. It must be an UUID4. This can be generated
  # using ``uuid.uuid4()``. Also, many editors can automatically generate
  # UUIDs. The same UUID must be stored in the components ``_id_name`` attribute.
  2ffbe0b5-a991-4fc5-a089-d2818a80a7e0:
    # The module where the component is located.
    module: my_package.components
    class: MyComponent2DClass

Note

HyperSpy’s legacy components use their class name instead of an UUID as _id_name. This is for compatibility with old versions of the software. New components (including those provided through the extension mechanism) must use an UUID4 in order to i) avoid name clashes ii) make it easy to find the component online if e.g. the package is renamed or the component relocated.

Creating and registering new widgets and toolkeys

To generate GUIs of specific method and functions, HyperSpy use widgets and toolkeys:

  • widgets (typically ipywidgets or traitsui objects) generate GUIs,

  • toolkeys are functions to which it is possible to associate widgets to a signal method or to a module function.

An extension can declare new toolkeys and widgets. For example, the hyperspy-gui-traitsui and hyperspy-gui-ipywidgets provide widgets for toolkeys declared in HyperSpy.

Registering toolkeys

To register a new toolkey:

  1. declare a new toolkey, e. g. by adding the hyperspy.ui_registry.add_gui_method() decorator to the function you want to assign a widget to,

  2. register a new toolkey that you have declared in your package by adding it to the hyperspy_extension.yaml file as in the following example:

GUI:
  # In order to assign a widget to a function, that function must declare
  # a `toolkey`. The `toolkeys` list contains a list of all the toolkeys
  # provided by the extensions. In order to avoid name clashes, by convention
  # toolkeys must start by the name of the packages that provides them.
  toolkeys:
    - my_package.MyComponent

Registering widgets

In the example below we register a new ipywidget widget for the my_package.MyComponent toolkey of the previous example. The function simply returns the widget to display. The key module defines where the functions resides.

GUI:
  widgets:
    ipywidgets:
      # Each widget is declared using a dictionary with two keys, `module` and `function`.
      my_package.MyComponent:
        # The function that creates the widget
        function: get_mycomponent_widget
        # The module where the function resides.
        module: my_package.widgets