hyperspy.samfire_utils package¶

Subpackages¶

Submodules¶

hyperspy.samfire_utils.fit_tests module¶

hyperspy.samfire_utils.global_strategies module¶

class hyperspy.samfire_utils.global_strategies.HistogramStrategy(bins='freedman')¶

Bases: hyperspy.samfire_utils.strategy.GlobalStrategy

hyperspy.samfire_utils.local_strategies module¶

class hyperspy.samfire_utils.local_strategies.ReducedChiSquaredStrategy¶

Bases: hyperspy.samfire_utils.strategy.LocalStrategy

Reduced chi-squared Local strategy of the SAMFire. Uses reduced chi-squared as the weight, and exponential decay as the decay function.

hyperspy.samfire_utils.local_strategies.exp_decay(distances)¶

Exponential decay function.

hyperspy.samfire_utils.samfire_kernel module¶

hyperspy.samfire_utils.samfire_kernel.multi_kernel(ind, m_dic, values, optional_components, _args, result_q, test_dict)¶

hyperspy.samfire_utils.samfire_kernel.single_kernel(model, ind, values, optional_components, _args, test)¶

hyperspy.samfire_utils.samfire_pool module¶

class hyperspy.samfire_utils.samfire_pool.SamfirePool(**kwargs)¶

Bases: hyperspy.utils.parallel_pool.ParallelPool

Creates and manages a pool of SAMFire workers. For based on ParallelPool - either creates processes using multiprocessing, or connects and sets up ipyparallel load_balanced_view.

Ipyparallel is managed directly, but multiprocessing pool is managed via three of Queues:

• Shared by all (master and workers) for distributing “load-balanced”

work. - Shared by all (master and workers) for sending results back to the master - Individual queues from master to each worker. For setting up and addressing individual workers in general. This one is checked with higher priority in workers.

prepare_workers()¶

given SAMFire object, populates the workers with the required information. In case of multiprocessing, starts worker listening to the queues.

update_optional_names()¶

updates the optional names in all workers

ping_workers()¶

pings all workers. Stores the one-way trip time and the process_id (pid) of each worker if available

add_jobs()¶

adds the requested number of jobs to the queue

parse()¶

parses the messages, returned from the workers.

colect_results()¶

collects all currently available results and parses them

run()¶

runs the full procedure until no more pixels are left to run in the SAMFire

stop()¶

stops the pool, (for ipyparallel) clears the memory

setup()¶

sets up the ipyparallel or multiprocessing pool (collects to the client or creates the pool)

sleep()¶

sleeps for the specified time, by default timestep

has_pool¶

Bool – Boolean if the pool is available and active

pool¶

{ipyparallel.load_balanced_view, multiprocessing.Pool} – The pool object

ipython_kwargs¶

dict – The dictionary with Ipyparallel connection arguments.

timeout¶

float – Timeout for either pool when waiting for results

num_workers¶

int – The number of workers actually created (may be less than requested, but can’t be more)

timestep¶

float – The timestep between “ticks” that the result queues are checked. Higher timestep means less frequent checking, which may reduce CPU load for difficult fits that take a long time to finish.

ping¶

dict – If recorded, stores one-way trip time of each worker

pid¶

dict – If available, stores the process-id of each worker

add_jobs(needed_number=None)

Adds jobs to the job queue that is consumed by the workers.

Parameters:	needed_number ({None, int}) – The number of jobs to add. If None (default), adds need_pixels

collect_results(timeout=None)¶

Collects and parses all results, currently not processed due to being in the queue.

Parameters:	timeout ({None, flaot}) – the time to wait when collecting results. If None, the default timeout is used

need_pixels¶

Returns the number of pixels that should be added to the processing queue. At most is equal to the number of workers.

parse(value)

Parses the value, returned from the workers.

Parameters:

value (tuple of the form (keyword, the_rest)) – keyword currently can be one of [‘pong’, ‘Error’, ‘result’]. For each of the keywords, “the_rest” is a tuple of different elements, but generally the first one is always the worker_id that the result came from. In particular: (‘pong’, (worker_id, pid, pong_time, optional_message_str)) (‘Error’, (worker_id, error_message_string)) (‘result’, (worker_id, pixel_index, result_dict, bool_if_result_converged))

ping_workers(timeout=None)

Pings the workers and records one-way trip time and (if available) pid of the worker.

Parameters:	timeout ({None, flaot}) – the time to wait when collecting results after sending out the ping. If None, the default timeout is used

prepare_workers(samfire)

Prepares the workers for work, in case of multiprocessing starts listening

Parameters:	samfire (samfire) – the SAMFire object that will be using the pool

run()

Runs the full process of adding jobs to the processing queue, listening to the results and updating SAMFire as needed. Stops when timed out or no pixels are left to run.

stop()

Stops the appropriate pool and (if ipyparallel) clears the memory and history.

update_optional_names()

Updates the optional names on the workers

hyperspy.samfire_utils.samfire_worker module¶

class hyperspy.samfire_utils.samfire_worker.Worker(identity, individual_queue=None, shared_queue=None, result_queue=None)¶

Bases: object

change_timestep(value)¶

compare_models()¶

create_model(signal_dict, model_letter)¶

fit(component_comb)¶

generate_component_combinations()¶

generate_values_iterator(turned_on_names)¶

listen()¶

parse(result)¶

ping(message=None)¶

reset()¶

run_pixel(ind, value_dict)¶

send_results(current=False)¶

set_optional_names(optional_names)¶

set_values(name_list, iterator)¶

setup_test(test_string)¶

sleep(howlong=None)¶

start_listening()¶

stop_listening()¶

hyperspy.samfire_utils.samfire_worker.create_worker(identity, individual_queue=None, shared_queue=None, result_queue=None)¶

hyperspy.samfire_utils.strategy module¶

class hyperspy.samfire_utils.strategy.GlobalStrategy(name)¶

Bases: hyperspy.samfire_utils.strategy.SamfireStrategy

A SAMFire strategy that operates in “parameter space” - i.e the pixel positions are not important, and only parameter value distributions are segmented to be used as starting point estimators.

clean()¶

Purges the currently saved values (not the database).

plot(fig=None)¶

Plots the current database of histograms

Parameters:	fig ({None, HistogramTilePlot}) – If given updates the plot.

refresh(overwrite, given_pixels=None)¶

Refreshes the database (i.e. constructs it again from scratch)

segmenter = None¶

values(ind=None)¶

Returns the saved most frequent values that should be used for prediction

class hyperspy.samfire_utils.strategy.LocalStrategy(name)¶

Bases: hyperspy.samfire_utils.strategy.SamfireStrategy

A SAMFire strategy that operates in “pixel space” - i.e calculates the starting point estimates based on the local averages of the pixels. Requires some weighting method (e.g. reduced chi-squared).

clean()¶

Purges the currently saved values.

plot(fig=None)¶

Plots the current marker in a flat image

Parameters:	fig ({Image, None}) – if an already plotted image, then updates. Otherwise creates a new one.
Returns:	fig – the resulting image. If passed again, will be updated (computationally cheaper operation).
Return type:	Image

radii¶

A tuple of >=0 floats that show the “radii of relevance”

refresh(overwrite, given_pixels=None)¶

Refreshes the marker - recalculates with the current values from scratch.

Parameters:

overwrite (Bool) – If True, all but the given_pixels will be recalculated. Used when part of already calculated results has to be refreshed. If False, only use pixels with marker == -scale (by default -1) to propagate to pixels with marker >= 0. This allows “ignoring” pixels with marker < -scale (e.g. -2). given_pixels (boolean numpy array) – Pixels with True value are assumed as correctly calculated.

samf¶

The SAMFire that owns this strategy.

values(ind)¶

Returns the current starting value estimates for the given pixel. Calculated as the weighted local average. Only returns components that are active, and parameters that are free.

Parameters:	ind (tuple) – the index of the pixel of interest.
Returns:	values – A dictionary of estimates, structured as {component_name: {parameter_name: value, ...}, ...} for active components and free parameters.
Return type:	dict

weight¶

A Weight object, able to assign significance weights to separate pixels or maps, given the model.

class hyperspy.samfire_utils.strategy.SamfireStrategy¶

Bases: object

A SAMFire strategy base class.

close_plot = None¶

name = ''¶

remove()¶

Removes this strategy from its SAMFire

samf = None¶

update(ind, isgood)¶

Updates the database and marker with the given pixel results

Parameters:	ind (tuple) – the index with new results isgood (bool) – if the fit was successful.

hyperspy.samfire_utils.strategy.make_sure_ind(inds, req_len=None)¶

Given an object, constructs a tuple of floats the required length. Either removes items that cannot be cast as floats, or adds the last valid item until the required length is reached.

Parameters:	inds (sequence) – the sequence to be constructed into tuple of floats req_len ({None, number}) – The required length of the output
Returns:	indices
Return type:	tuple of floats

hyperspy.samfire_utils.strategy.nearest_indices(shape, ind, radii)¶

Returns the slices to slice a given size array to get the required size rectangle around the given index. Deals nicely with boundaries.

Parameters:	shape (tuple) – the shape of the original (large) array ind (tuple) – the index of interest in the large array (centre) radii (tuple of floats) – the distances of interests in all dimensions around the centre index.
Returns:	slices (tuple of slices) – The slices to slice the large array to get the required region. center (tuple of ints) – The index of the original centre (ind) position in the new (sliced) array.

Module contents¶