Mixed measures processes - MetricsReloaded.processes.mixed_measures_processes

This module provides classes for performing the evaluation processes of

instance segmentation, multi label instance segmentation,

multilabel object detection and multi class classification.

Performing the process associated with instance segmentation

class MetricsReloaded.processes.mixed_measures_processes.MixedLocSegPairwiseMeasure(pred, ref, list_predimg, list_refimg, pred_prob, measures_overlap=[], measures_boundary=[], measures_mt=[], measures_pcc=[], measures_detseg=[], connectivity=1, pixdim=[], empty=False, dict_args={})

This class is for use in the context of instance segmentation when identification of location, classification of instances and contour segmentation are required This is initialised with the following parameters:

Parameters:

• pred – np array with prediction of class of individual elements

• ref – np array with reference class of individual elements

• list_predimg – list of the prediction images (one per element to consider in the comparison)

• list_refimg – list of reference images (one per element to consider as reference instance in the comparison)

• pred_prob – np array of predicted class probabilities

• measures_overlap – list of choices of measures of overlap

• measures_boundary – list of choices of measures assessing boundary agreement

• measures_mt – list of choices of measures using multiple probability thresholds

• measures_pcc – list of choices of measures of per class counting in terms of classification of instances

• measures_detseg – consideration (list) of metrics combining both segmentation and detection performance

• dict_args – dictionary with relevant arguments for the metrics

segmentation_quality()

Calculates the segmentation quality in an instance segmentation case with SQ defined as the average IoU over TP instances

detection_quality()

Calculates the detection quality for a case of instance segmentation defined as the F1 score

panoptic_quality()

Calculates the panopitic quality defined as the production between detection quality and segmentation quality

Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollár. 2019. Panoptic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9404–9413.

Returns:

PQ

to_dict_mt()

Transform to dictionary the result of the multi threshold metrics

to_dict_det()

Transform to dictionary the results of the detection metrics (includes also panoptic quality metric)

to_pd_seg()

Transforms the segmentation results to a pandas dataframe

Performing the process associated with multiple labels in instance segmentation

class MetricsReloaded.processes.mixed_measures_processes.MultiLabelLocSegPairwiseMeasure(pred_class, ref_class, pred_loc, ref_loc, pred_prob, list_values, names=[], measures_pcc=[], measures_overlap=[], measures_boundary=[], measures_detseg=[], measures_mt=[], per_case=True, flag_map=True, file=[], connectivity=1, pixdim=[], empty=False, assignment='Greedy_IoU', localization='iou', thresh=0.5, flag_fp_in=True, dict_args={})

This class represents the processing for instance segmentation on true positive Characterised by the predicted classes and associated reference classes

Parameters:

pred_class – list for each considered case of classes predicted

:param ref_class:list for each considered case of reference classes :param pred_loc: list for each considered case of the individual image considering an individual predicted element :param ref_loc: list for each considered case of the individual images considering individual reference elements (note that ref_loc and ref_class entities are matching) :param pred_prob: :param list_values: list of possible label values :param measures_pcc: list of per class counting measures to be derived during the process :param measures_overlap: list of overlap (segmentation) measures to be derived during the process :param measures_boundary: list of boundary measures to be derived during the comparison process :param measures_detseg: list of measures combining detection and segmentation to be derived during the comparison process :param measures_mt: list of multi-threshold measures to be derived during the comparison process :param per_case: flag indicating whether each case should be derived individually or if all cases should be assessed at once :param flag_map: flag indicating whether an image for either predicted TP, reference TP, FP or FN element should be generated :param file: list of files (reference or prediction) that led to each case :param assignment: type of chosen assignment choice among [] :param localization: type of localization strategy chosen for the assignment :param thresh: :param flag_fp_in: flag to consider the false positive elements in the assessment :param dict_args: dictionary for additional arguments related to the chosen metrics.

create_nifti_image(list_maps, file_ref, category)

Creates a nifti image of either the true positives, true negatives, false positives or false negatives :param list_maps: list of np.arrays containing an element to add to the final image :param file_ref: reference nifti file to use for saving the final image :param category: category description of the elements being saved (classically TP TN FP FN)

per_label_dict()

According to the specifications of metrics to be used and the type of assignment and localization, performs, per label value in list_values the processing per case (overall prediction associated to overall reference image). This is organised in multiple steps:

• identification for each predicted case of the items considered as of the class specified by label considered

• identification for each associated case, the items considered as of the class specified by the considered label

• creation of the associated list of individual images of elements selected beforehand both in the prediction images and the reference images (the images are listed in the same order with one element per image)

• assigment procedure based on the segmentation images

• derivation of metrics either on a case by case basis or grouping all cases together.

Performing the process associated with multiple labels in instance segmentation

class MetricsReloaded.processes.mixed_measures_processes.MultiLabelLocMeasures(pred_class, ref_class, pred_loc, ref_loc, pred_prob, list_values, names=[], measures_pcc=[], measures_mt=[], per_case=False, assignment='Greedy IoU', localization='iou', thresh=0.5, flag_fp_in=True, pixdim=[], dict_args={})

Class for the processing of multilabel object detection processes :param pred_class: list for each considered case of classes predicted :param ref_class:list for each considered case of reference classes :param pred_loc: list for each considered case of the individual image considering an individual predicted element :param ref_loc: list for each considered case of the individual images considering individual reference elements (note that ref_loc and ref_class entities are matching) :param pred_prob: :param list_values: list of possible label values :param measures_pcc: list of per class counting measures to be derived during the process :param measures_mt: list of multi-threshold measures to be derived during the comparison process :param per_case: flag indicating whether each case should be derived individually or if all cases should be assessed at once :param assignment: type of chosen assignment choice among [] :param localization: type of localization strategy chosen for the assignment :param thresh: :param flag_fp_in: flag to consider the false positive elements in the assessment :param dict_args: dictionary for additional arguments related to the chosen metrics.

Performing the process associated with multiple labels in classification (semantic segmentation or image level classification)

class MetricsReloaded.processes.mixed_measures_processes.MultiLabelPairwiseMeasures(pred, ref, pred_proba, list_values, names=[], measures_pcc=[], measures_mt=[], measures_mcc=[], measures_overlap=[], measures_boundary=[], measures_calibration=[], connectivity_type=1, per_case=False, pixdim=[], empty=False, dict_args={})

Semantic segmentation or Image wide classification with possibility of multiple labels :param pred: :param ref: :param pred_proba: :param list_values: :param names: :param measures_pcc: :param measures_mt: :param measures_mcc: :param measures_overlap: :param measures_boundary: :param measures_calibration: :param connectivity_type: :param per_case: :param pixdim: :param empty: :param dict_args: