hyperspy.utils.parallel_pool module

class hyperspy.utils.parallel_pool.ParallelPool(num_workers=None, ipython_kwargs=None, ipyparallel=None)

Bases: object

Creates a ParallelPool by either looking for a ipyparallel client and then creating a load_balanced_view, or by creating a multiprocessing pool

setup()

sets up the requested pool

sleep()

sleeps for the requested (or timeout) time

has_pool

Boolean if the pool is available and active.

Type

Bool

pool

The pool object.

Type

{ipyparallel.load_balanced_view, multiprocessing.Pool}

ipython_kwargs

The dictionary with Ipyparallel connection arguments.

Type

dict

timeout

Timeout for either pool when waiting for results.

Type

float

num_workers

The number of workers actually created (may be less than requested, but can’t be more).

Type

int

timestep

Can be used as “ticks” to adjust CPU load when building upon this class.

Type

float

is_ipyparallel

If the pool is ipyparallel-based

Type

bool

is_multiprocessing

If the pool is multiprocessing-based

Type

bool

Creates the ParallelPool and sets it up.

Parameters
  • num_workers ({None, int}) – the (max) number of workers to create. If less are available, smaller number is actually created.

  • ipyparallel ({None, bool}) – which pool to set up. True - ipyparallel. False - multiprocessing. None - try ipyparallel, then multiprocessing if failed.

  • ipython_kwargs ({None, dict}) – arguments that will be passed to the ipyparallel.Client when creating. Not None implies ipyparallel=True.

property has_pool

Returns bool if the pool is ready and set-up

property is_ipyparallel

Returns bool if the pool is ipyparallel-based

property is_multiprocessing

Returns bool if the pool is multiprocessing-based

setup(ipyparallel=None)

Sets up the pool.

Parameters

ipyparallel ({None, bool}) – if True, only tries to set up the ipyparallel pool. If False - only the multiprocessing. If None, first tries ipyparallel, and it does not succeed, then multiprocessing.

sleep(howlong=None)

Sleeps for the required number of seconds.

Parameters

howlong ({None, float}) – How long the pool should sleep for in seconds. If None (default), sleeps for “timestep”

property timestep