Machine learning

Introduction

HyperSpy provides easy access to several “machine learning” algorithms that can be useful when analysing multi-dimensional data. In particular, decomposition algorithms, such as principal component analysis (PCA), or blind source separation (BSS) algorithms, such as independent component analysis (ICA), are available through the methods described in this section.

The behaviour of some machine learning operations can be customised customised in the Machine Learning section Preferences.

Note

Currently the BSS algorithms operate on the result of a previous decomposition analysis. Therefore, it is necessary to perform a decomposition before attempting to perform a BSS.

Nomenclature

HyperSpy will decompose a dataset into two new datasets: one with the dimension of the signal space known as factors, and the other with the dimension of the navigation space known as loadings.

Decomposition

Decomposition techniques are most commonly applied as a means of noise reduction (or denoising) and dimensionality reduction.

Principal component analysis

One of the most popular decomposition methods is principal component analysis (PCA). To perform PCA on your dataset, run the decomposition() method:

>>> s.decomposition()

Note that the s variable must contain either a BaseSignal class or its subclasses, which will most likely have been loaded with the load() function, e.g. s = load('my_file.hdf5'). Also, the signal must be multi-dimensional, that is s.axes_manager.navigation_size must be greater than one.

Several algorithms exist for performing PCA, and the default algorithm in HyperSpy is SVD, which uses an approach called “singular value decomposition”. This method has many options, and for more information please read the method documentation.

Scree plots

PCA will sort the components in the dataset in order of decreasing variance. It is often useful to estimate the dimensionality of the data by plotting the explained variance against the component index. This plot is sometimes called a scree plot and it should drop quickly, eventually becoming a slowly descending line.

The point at which the scree plot becomes linear (often referred to as the elbow) is generally judged to be a good estimation of the dimensionality of the data (or equivalently, the number of components that should be retained - see below).

To obtain a scree plot for your dataset, run the plot_explained_variance_ratio() method:

>>> ax = s.plot_explained_variance_ratio()
../_images/screeplot.png

PCA scree plot

Note that in the figure, the first component has index 0. This is because Python uses zero based indexing i.e. the initial element of a sequence is found at index 0.

New in version 0.7.

Sometimes it can be useful to get the explained variance ratio as a spectrum, for example to plot several scree plots obtained using different data pre-treatmentd in the same figure using plot_spectra(). This can be achieved using get_explained_variance_ratio()

Denoising

One of the most popular uses of PCA is data denoising. This is achieved by using a limited set of components to make a model of the original, omitting the later components that ideally contain only noise. This is also known as dimensionality reduction.

To perform this operation with HyperSpy, run the get_decomposition_model() method, usually after estimating the dimension of your data using a scree plot. For example:

>>> sc = s.get_decomposition_model(components)

Note

The components argument can be one of several things (None, int, or list of ints):

  • if None, all the components are used to construct the model.
  • if int, only the given number of components (starting from index 0) are used to construct the model.
  • if list of ints, only the components in the given list are used to construct the model.

Sometimes, it is useful to examine the residuals between your original data and the decomposition model. You can easily calculate and display the residuals:

>>> (s - sc).plot()

Hint

Unlike most of the analysis functions, this function returns a new object, which in the example above we have called ‘sc’. You can perform operations on this new object later. It is a copy of the original s object, except that the data has been replaced by the model constructed using the chosen components.

Poissonian noise

Many decomposition methods such as PCA assume that the noise of the data follows a Gaussian distribution. In cases where your data is instead corrupted by Poisson noise, HyperSpy can “normalize” the data by performing a scaling operation, which can greatly enhance the result.

To perform Poissonian noise normalization:

More details about the scaling procedure can be found in [Keenan2004].

Robust principal component analysis

PCA is known to be very sensitive to the presence of outliers in data. These outliers can be the result of missing or dead pixels, X-ray spikes, or very low count data. If one assumes a dataset to consist of a low-rank component L corrupted by a sparse error component S, then Robust PCA (RPCA) can be used to recover the low-rank component for subsequent processing [Candes2011].

The default RPCA algorithm is GoDec [Zhou2011]. In HyperSpy it returns the factors and loadings of L, and can be accessed with the following code. You must set the “output_dimension” when using RPCA.

>>> s.decomposition(algorithm='RPCA_GoDec',
                    output_dimension=3)

HyperSpy also implements an online algorithm for RPCA developed by Feng et al. [Feng2013]. This minimizes memory usage, making it suitable for large datasets, and can often be faster than the default algorithm.

>>> s.decomposition(algorithm='ORPCA',
                    output_dimension=3)

The online RPCA implementation sets several default parameters that are usually suitable for most datasets. However, to improve the convergence you can “train” the algorithm with the first few samples of your dataset. For example, the following code will train ORPCA using the first 32 samples of the data.

>>> s.decomposition(algorithm='ORPCA',
                    output_dimension=3
                    training_samples=32)

Finally, online RPCA includes two alternative methods to the default solver, which can again improve the convergence and speed of the algorithm. The first is block-coordinate descent (BCD), and the second is based on stochastic gradient descent (SGD), which takes an additional parameter to set the learning rate.

>>> s.decomposition(algorithm='ORPCA',
                    output_dimension=3
                    method='BCD',
                    training_samples=32)

>>> s.decomposition(algorithm='ORPCA',
                    output_dimension=3
                    method='SGD',
                    learning_rate=1.1,
                    training_samples=32)

Non-negative matrix factorization

Another popular decomposition method is non-negative matrix factorization (NMF), which can be accessed in HyperSpy with:

>>> s.decomposition(algorithm='nmf')

Unlike PCA, NMF forces the components to be strictly non-negative, which can aid the physical interpretation of components for count data such as images, EELS or EDS.

Blind Source Separation

In some cases (it largely depends on the particular application) it is possible to obtain more physically interpretable set of components using a process called Blind Source Separation (BSS). For more information about blind source separation please see [Hyvarinen2000], and for an example application to EELS analysis, see [Pena2010].

To perform BSS on the result of a decomposition, run the blind_source_separation() method, e.g.:

s.blind_source_separation(number_of_components)

Note

Currently the BSS algorithms operate on the result of a previous decomposition analysis. Therefore, it is necessary to perform a Decomposition first.

Note

You must pass an integer number of components to ICA. The best way to estimate this number in the case of a PCA decomposition is by inspecting the Scree plots.

Visualizing results

HyperSpy includes a number of plotting methods for the results of decomposition and blind source separation. All the methods begin with “plot_”:

  1. plot_decomposition_results().
  2. plot_decomposition_factors().
  3. plot_decomposition_loadings().
  4. plot_bss_results().
  5. plot_bss_factors().
  6. plot_bss_loadings().

1 and 4 (new in version 0.7) provide a more compact way of displaying the results. All the other methods display each component in its own window. For 2 and 3 it is wise to provide the number of factors or loadings you wish to visualise, since the default is to plot all of them. For BSS, the default is the number you included when running the blind_source_separation() method.

Obtaining the results as BaseSignal instances

New in version 0.7.

The decomposition and BSS results are internally stored as numpy arrays in the BaseSignal class. Frequently it is useful to obtain the decomposition/BSS factors and loadings as HyperSpy signals, and HyperSpy provides the following methods for that purpose:

Saving and loading results

There are several methods for storing the result of a machine learning analysis.

Saving in the main file

If you save the dataset on which you’ve performed machine learning analysis in the HDF5 format (the default in HyperSpy) (see Saving data to files), the result of the analysis is also saved in the same file automatically, and it is loaded along with the rest of the data when you next open the file.

Note

This approach currently supports storing one decomposition and one BSS result, which may not be enough for your purposes.

Saving to an external file

Alternatively, you can save the results of the current machine learning analysis to a separate file with the save() method:

Save the result of the analysis
>>> s.learning_results.save('my_results')

Load back the results
>>> s.learning_results.load('my_results.npz')

Exporting in different formats

It is also possible to export the results of machine learning to any format supported by HyperSpy with:

These methods accept many arguments to customise the way in which the data is exported, so please consult the method documentation. The options include the choice of file format, the prefixes for loadings and factors, saving figures instead of data and more.

Note

Data exported in this way cannot be easily loaded into HyperSpy’s machine learning structure.