pyEdgeEval.evaluators package

Submodules

pyEdgeEval.evaluators.base module

class pyEdgeEval.evaluators.base.BaseEvaluator

Bases: object

dataset_root = None

pred_root = None

split = None

property sample_names

abstract set_sample_names(**kwargs)

Placeholder for setting sample_names

abstract set_eval_params(**kwargs)

Placeholder for setting evaluation parameters

abstract eval_params()

Placeholder for getting evaluation parameters

abstract evaluate(**kwargs)

Placeholder for the main evaluation function

class pyEdgeEval.evaluators.base.BaseBinaryEvaluator

Bases: BaseEvaluator

class pyEdgeEval.evaluators.base.BaseMultilabelEvaluator

Bases: BaseEvaluator

CLASSES = None

abstract evaluate_category(**kwargs)

Placeholder for the category evaluation function

evaluate(categories, thresholds, nproc, save_dir, print_metrics=('ODS_threshold', 'ODS_recall', 'ODS_precision', 'ODS_f1', 'AUC'))

pyEdgeEval.evaluators.binary_evaluator module

Custom Evaluator for binary-label datasets

Make it easy to create evaluators by subclassing

pyEdgeEval.evaluators.bsds module

class pyEdgeEval.evaluators.bsds.BSDS500Evaluator(dataset_root: str, pred_root: str, split: str = 'test')

Bases: BaseBinaryEvaluator

GT_DIR = 'groundTruth'

GT_SUFFIX = '.mat'

PRED_SUFFIX = '.png'

set_sample_names(sample_names=None)

set_eval_params(scale: float = 1.0, apply_thinning: bool = True, apply_nms: bool = False, max_dist: float = 0.0075, **kwargs) → None

property eval_params

evaluate(thresholds, nproc, save_dir, no_split_dir=False)

pyEdgeEval.evaluators.cityscapes module

Cityscapes Evaluator

The original evaluator where GTs are downsampled (interpolated) from full scale.

class pyEdgeEval.evaluators.cityscapes.CityscapesEvaluator(dataset_root: str, pred_root: str, split: str = 'val', thin: bool = False, gt_dir=None, pred_suffix=None, **kwargs)

Bases: BaseMultilabelEvaluator

Cityscapes dataset evaluator

CLASSES = ('road', 'sidewalk', 'building', 'wall', 'fence', 'pole', 'traffic light', 'traffic sign', 'vegetation', 'terrain', 'sky', 'person', 'rider', 'car', 'truck', 'bus', 'train', 'motorcycle', 'bicycle')

ORIG_GT_DIR = 'gtFine'

RAW_EDGE_SUFFIX = '_gtProc_raw_edge.png'

THIN_EDGE_SUFFIX = '_gtProc_thin_edge.png'

RAW_ISEDGE_SUFFIX = '_gtProc_raw_isedge.png'

THIN_ISEDGE_SUFFIX = '_gtProc_thin_isedge.png'

SEG_SUFFIX = '_gtFine_labelTrainIds.png'

EDGE_SUFFIX = None

ISEDGE_SUFFIX = None

GT_DIR = 'gtEval'

PRED_SUFFIX = '_leftImg8bit.png'

set_sample_names(sample_names=None, split_file=None)

priortizes sample_names more than split_file

set_pred_suffix(suffix: str)

set_eval_params(eval_mode=None, scale: float = 0.5, apply_thinning: bool = False, apply_nms: bool = False, instance_sensitive: bool = True, max_dist: float = 0.0035, skip_if_nonexistent: bool = False, kill_internal: bool = False, **kwargs) → None

property eval_params

evaluate_category(category, thresholds, nproc, save_dir)

pyEdgeEval.evaluators.half_cityscapes module

Cityscapes Evaluator with forced half scale

Need to create half scale edge GTs with the prefix _half_edge.png

This is different from CASENet, SEAL, and DFF way of evaluating.

The evaluation outcomes are generally lower because we have lower recall.

class pyEdgeEval.evaluators.half_cityscapes.HalfCityscapesEvaluator(dataset_root: str, pred_root: str, split: str = 'val', thin: bool = False, gt_dir=None, pred_suffix=None, **kwargs)

Bases: CityscapesEvaluator

Half-scale Cityscapes dataset evaluator

• used GTs that are preprocessed to half scale

• half scale evaluations are common for this dataset to speed up the process

RAW_EDGE_SUFFIX = '_gtProc_half_raw_edge.png'

THIN_EDGE_SUFFIX = '_gtProc_half_thin_edge.png'

RAW_ISEDGE_SUFFIX = '_gtProc_half_raw_isedge.png'

THIN_ISEDGE_SUFFIX = '_gtProc_half_thin_isedge.png'

set_eval_params(eval_mode=None, apply_thinning: bool = False, apply_nms: bool = False, instance_sensitive: bool = True, max_dist: float = 0.0035, skip_if_nonexistent: bool = False, kill_internal: bool = False, **kwargs) → None

property eval_params

pyEdgeEval.evaluators.multilabel_evaluator module

Custom Evaluator for Multi-label datasets

Make it easy to create evaluators by subclassing

pyEdgeEval.evaluators.otf_cityscapes module

On-The-Fly Evaluator

• lazy generation of GTs

• if the scale is half (0.5), the output is generally the same as HalfCityscapesEvaluator

class pyEdgeEval.evaluators.otf_cityscapes.OTFCityscapesEvaluator(dataset_root: str, pred_root: str, split: str = 'val', thin: bool = False, gt_dir=None, pred_suffix=None, **kwargs)

Bases: CityscapesEvaluator

On-The-Fly cityscapes dataset evaluator

• On-The-Fly (OTF) creation of GT edges

  • needs GT segmentation and instance maps

• Scales the masks first before generating the edges

  • non-OTF could create edges that are too thin if scaled down

SEG_SUFFIX = '_gtFine_labelIds.png'

INST_SUFFIX = '_gtFine_instanceIds.png'

property eval_params

evaluate_category(category, thresholds, nproc, save_dir)

pyEdgeEval.evaluators.sbd module

class pyEdgeEval.evaluators.sbd.SBDEvaluator(dataset_root: str, pred_root: str, split: str = 'val', **kwargs)

Bases: BaseMultilabelEvaluator

SBD evaluator

CLASSES = ('aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor')

CLS_DIR = 'cls'

INST_DIR = 'inst'

GT_SUFFIX = '.mat'

PRED_SUFFIX = '.bmp'

set_sample_names(sample_names=None, split_file=None)

priortizes sample_names more than split_file

set_eval_params(eval_mode=None, scale: float = 1.0, apply_thinning: bool = False, apply_nms: bool = False, instance_sensitive: bool = True, max_dist: float = 0.02, skip_if_nonexistent: bool = False, kill_internal: bool = False, **kwargs) → None

property eval_params

evaluate_category(category, thresholds, nproc, save_dir)

Module contents

class pyEdgeEval.evaluators.BSDS500Evaluator(dataset_root: str, pred_root: str, split: str = 'test')

Bases: BaseBinaryEvaluator

GT_DIR = 'groundTruth'

GT_SUFFIX = '.mat'

PRED_SUFFIX = '.png'

set_sample_names(sample_names=None)

set_eval_params(scale: float = 1.0, apply_thinning: bool = True, apply_nms: bool = False, max_dist: float = 0.0075, **kwargs) → None

property eval_params

evaluate(thresholds, nproc, save_dir, no_split_dir=False)

class pyEdgeEval.evaluators.CityscapesEvaluator(dataset_root: str, pred_root: str, split: str = 'val', thin: bool = False, gt_dir=None, pred_suffix=None, **kwargs)

Bases: BaseMultilabelEvaluator

Cityscapes dataset evaluator

CLASSES = ('road', 'sidewalk', 'building', 'wall', 'fence', 'pole', 'traffic light', 'traffic sign', 'vegetation', 'terrain', 'sky', 'person', 'rider', 'car', 'truck', 'bus', 'train', 'motorcycle', 'bicycle')

ORIG_GT_DIR = 'gtFine'

RAW_EDGE_SUFFIX = '_gtProc_raw_edge.png'

THIN_EDGE_SUFFIX = '_gtProc_thin_edge.png'

RAW_ISEDGE_SUFFIX = '_gtProc_raw_isedge.png'

THIN_ISEDGE_SUFFIX = '_gtProc_thin_isedge.png'

SEG_SUFFIX = '_gtFine_labelTrainIds.png'

EDGE_SUFFIX = None

ISEDGE_SUFFIX = None

GT_DIR = 'gtEval'

PRED_SUFFIX = '_leftImg8bit.png'

set_sample_names(sample_names=None, split_file=None)

priortizes sample_names more than split_file

set_pred_suffix(suffix: str)

set_eval_params(eval_mode=None, scale: float = 0.5, apply_thinning: bool = False, apply_nms: bool = False, instance_sensitive: bool = True, max_dist: float = 0.0035, skip_if_nonexistent: bool = False, kill_internal: bool = False, **kwargs) → None

property eval_params

evaluate_category(category, thresholds, nproc, save_dir)

class pyEdgeEval.evaluators.SBDEvaluator(dataset_root: str, pred_root: str, split: str = 'val', **kwargs)

Bases: BaseMultilabelEvaluator

SBD evaluator

CLASSES = ('aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor')

CLS_DIR = 'cls'

INST_DIR = 'inst'

GT_SUFFIX = '.mat'

PRED_SUFFIX = '.bmp'

set_sample_names(sample_names=None, split_file=None)

priortizes sample_names more than split_file

set_eval_params(eval_mode=None, scale: float = 1.0, apply_thinning: bool = False, apply_nms: bool = False, instance_sensitive: bool = True, max_dist: float = 0.02, skip_if_nonexistent: bool = False, kill_internal: bool = False, **kwargs) → None

property eval_params

evaluate_category(category, thresholds, nproc, save_dir)

class pyEdgeEval.evaluators.OTFCityscapesEvaluator(dataset_root: str, pred_root: str, split: str = 'val', thin: bool = False, gt_dir=None, pred_suffix=None, **kwargs)

Bases: CityscapesEvaluator

On-The-Fly cityscapes dataset evaluator

• On-The-Fly (OTF) creation of GT edges

  • needs GT segmentation and instance maps

• Scales the masks first before generating the edges

  • non-OTF could create edges that are too thin if scaled down

SEG_SUFFIX = '_gtFine_labelIds.png'

INST_SUFFIX = '_gtFine_instanceIds.png'

property eval_params

evaluate_category(category, thresholds, nproc, save_dir)

class pyEdgeEval.evaluators.HalfCityscapesEvaluator(dataset_root: str, pred_root: str, split: str = 'val', thin: bool = False, gt_dir=None, pred_suffix=None, **kwargs)

Bases: CityscapesEvaluator

Half-scale Cityscapes dataset evaluator

• used GTs that are preprocessed to half scale

• half scale evaluations are common for this dataset to speed up the process

RAW_EDGE_SUFFIX = '_gtProc_half_raw_edge.png'

THIN_EDGE_SUFFIX = '_gtProc_half_thin_edge.png'

RAW_ISEDGE_SUFFIX = '_gtProc_half_raw_isedge.png'

THIN_ISEDGE_SUFFIX = '_gtProc_half_thin_isedge.png'

set_eval_params(eval_mode=None, apply_thinning: bool = False, apply_nms: bool = False, instance_sensitive: bool = True, max_dist: float = 0.0035, skip_if_nonexistent: bool = False, kill_internal: bool = False, **kwargs) → None

property eval_params