metrics

Compute the confusion matrix.

Parameters:

• preds (Tensor) –

  Predicted class indices, shape \((N,)\).

• target (Tensor) –

  Ground truth class indices, shape \((N,)\).

• num_classes (int) –

  Total number of classes.

• ignore_index (Optional[int], default: None ) –

  Class index to exclude from computation.

Returns:

• Tensor –

  Confusion matrix of shape \((\text{num\_classes}, \text{num\_classes})\) where

• Tensor –

  cm[i, j] is the number of points with true class i

• Tensor –

  predicted as class j.

Compute per-class intersection and union counts.

Parameters:

• preds (Tensor) –

  Predicted class indices, shape \((N,)\).

• target (Tensor) –

  Ground truth class indices, shape \((N,)\).

• num_classes (int) –

  Total number of classes.

• batch (Optional[Tensor], default: None ) –

  Optional per-point batch index for per-sample counts. One row is emitted per sample (even for samples whose points are all ignored, which count as zero).

• ignore_index (Optional[int], default: None ) –

  Class index to exclude. Points where target == ignore_index are dropped, and the returned intersection/union at this index are \(0\).

Returns:

• Tensor –

  Tuple \((\text{intersection}, \text{union})\), each of shape \((\text{num\_classes},)\)

• Tensor –

  or \((\text{batch\_size}, \text{num\_classes})\) if batch is provided.

Compute the Intersection over Union (IoU) for each class.

Parameters:

• preds (Tensor) –

  Predicted class indices, shape \((N,)\).

• target (Tensor) –

  Ground truth class indices, shape \((N,)\).

• num_classes (int) –

  Total number of classes.

• batch (Optional[Tensor], default: None ) –

  Optional per-point batch index for per-sample IoU.

• ignore_index (Optional[int], default: None ) –

  Class index to exclude from computation. The returned IoU at this index will be \(0\).

• default (float | Tensor, default: 0.0 ) –

  Value returned for classes with zero union (avoids division by zero).

Returns:

• Tensor –

  Per-class IoU tensor of shape \((\text{num\_classes},)\)

• Tensor –

  or \((\text{batch\_size}, \text{num\_classes})\) if batch is provided.

Compute the mean Intersection over Union (mIoU).

Averages IoU over all classes except ignore_index; a class absent from the whole input (zero union) counts as IoU \(0\), matching sklearn's jaccard_score(zero_division=0). Toolboxes that average only over present classes (a nanmean over nonzero unions) report a higher value on splits missing a class, so compare published numbers accordingly. With batch, each sample is averaged only over the classes present in it (nonzero union), so a perfect prediction scores \(1\) regardless of how many of the dataset's classes the sample contains.

Parameters:

• preds (Tensor) –

  Predicted class indices, shape \((N,)\).

• target (Tensor) –

  Ground truth class indices, shape \((N,)\).

• num_classes (int) –

  Total number of classes.

• batch (Optional[Tensor], default: None ) –

  Optional per-point batch index for per-sample mIoU, averaged over each sample's present classes. A sample whose points are all ignored scores \(0\).

• ignore_index (Optional[int], default: None ) –

  Class index to exclude from the mean.

Returns:

• Tensor –

  Scalar mIoU value or per-batch mIoU value if batch is provided.

Per-shape IoU averaged over the parts of the shape's category (the ShapeNetPart protocol).

Each shape is scored only over the part labels its category owns (e.g. ShapeNetPart's Airplane owns parts \([0, 1, 2, 3]\)); a part absent from both the prediction and the target counts as IoU \(1\).

Parameters:

• preds (Tensor) –

  Predicted part indices, shape \((N,)\).

• target (Tensor) –

  Ground truth part indices, shape \((N,)\).

• part_ids (Sequence[Sequence[int]]) –

  Part labels owned by each category, e.g. ShapeNetPart.seg_ids.values().

• category (Tensor) –

  Per-shape category index into part_ids, shape \((B,)\).

• batch (Tensor) –

  Per-point shape index, shape \((N,)\).

Returns:

• Tensor –

  Per-shape IoU tensor of shape \((B,)\).

ShapeNetPart instance and class mean IoU.

part_iou scores each shape over its category's parts; the instance mIoU averages these per-shape IoUs over all shapes, and the class mIoU averages them per category first, then over the categories present in category.

Parameters:

• preds (Tensor) –

  Predicted part indices, shape \((N,)\).

• target (Tensor) –

  Ground truth part indices, shape \((N,)\).

• part_ids (Sequence[Sequence[int]]) –

  Part labels owned by each category, e.g. ShapeNetPart.seg_ids.values().

• category (Tensor) –

  Per-shape category index into part_ids, shape \((B,)\).

• batch (Tensor) –

  Per-point shape index, shape \((N,)\).

Returns:

• Dict[str, float] –

  A dict {"ins_mIoU": ..., "cls_mIoU": ...}.

>>> part_ids = [[0, 1], [2, 3]]
>>> preds = torch.tensor([0, 1, 2, 2])
>>> target = torch.tensor([0, 1, 2, 3])
>>> category = torch.tensor([0, 1])
>>> batch = torch.tensor([0, 0, 1, 1])
>>> part_mean_iou(preds, target, part_ids, category, batch)
{'ins_mIoU': 0.625, 'cls_mIoU': 0.625}

Compute the overall prediction accuracy.

Parameters:

• preds (Tensor) –

  Predicted class indices, shape \((N,)\).

• target (Tensor) –

  Ground truth class indices, shape \((N,)\).

• ignore_index (Optional[int], default: None ) –

  Class index to exclude from computation.

Returns:

• float –

  Scalar accuracy value, or 0.0 when no points remain after ignore_index masking.

Compute the accuracy for each class.

Parameters:

• preds (Tensor) –

  Predicted class indices, shape \((N,)\).

• target (Tensor) –

  Ground truth class indices, shape \((N,)\).

• num_classes (int) –

  Total number of classes.

• ignore_index (Optional[int], default: None ) –

  Class index to exclude. The returned accuracy at this index will be 0.

• eps (float, default: 1e-10 ) –

  Small constant to avoid division by zero.

Returns:

• Tensor –

  Per-class accuracy tensor of shape \((\text{num\_classes},)\).

3D detection mean average precision over one or more IoU thresholds (same IoU for every class).

Dataset- and model-agnostic: predictions and targets are packed dicts of parameterized boxes (see box_corners) carrying a per-box scene index, so any detector emitting (boxes, scores, labels, batch) is scored the same way. mAP@t averages the per-class AP over the classes present in the targets. Targets may carry an ignore_mask (see Boxes3D); an ignore box's labels entry names the class it excuses, and unmatched predictions of that class overlapping it are not penalized. Predictions may carry an ignore_mask of their own; flagged predictions are excluded from scoring entirely (the KITTI min-height rule). Use average_precision3d for per-class IoU thresholds.

Parameters:

• preds (Sequence[Detection3D]) –

  Packed predictions (one decode output per batch), each {"boxes": (N, 7), "scores": (N,), "labels": (N,), "batch": (N,)}.

• targets (Sequence[Boxes3D]) –

  Packed ground truth aligned to preds batch-for-batch, each {"boxes", "labels", "batch"}.

• iou_thresholds (Sequence[float], default: (0.25, 0.5) ) –

  IoU thresholds at which mAP@t is reported.

• interpolation (Interpolation, default: 'all' ) –

  AP interpolation: "all" integrates the full precision-recall curve; "r11" / "r40" sample the KITTI 11- / 40-point recall grids.

Returns:

• Dict[str, float] –

  A dict {"mAP@0.25": ..., "mAP@0.5": ...} keyed by threshold.

Per-class 3D AP, each class scored at its own IoU threshold (e.g. KITTI Car@0.7, Ped/Cyc@0.5).

Like mean_average_precision3d but reports one AP per class at a class-specific IoU, the convention of the KITTI / nuScenes detection metrics. Targets may carry an ignore_mask (see Boxes3D): an ignore box's labels entry names the class it excuses (e.g. an ignored KITTI Van attributed to Car), and unmatched predictions of that class overlapping it are not counted as false positives. Predictions may carry an ignore_mask of their own; flagged predictions are excluded from scoring entirely (the KITTI min-height rule).

Parameters:

• preds (Sequence[Detection3D]) –

  Packed predictions aligned to targets batch-for-batch.

• targets (Sequence[Boxes3D]) –

  Packed ground truth, each {"boxes", "labels", "batch"} with an optional ignore_mask.

• iou_per_class (Mapping[int, float]) –

  IoU threshold per class index, e.g. {0: 0.7, 1: 0.5, 2: 0.5}.

• class_names (Optional[Sequence[str]], default: None ) –

  Optional names for the output keys (indexed by class); falls back to the index.

• interpolation (Interpolation, default: 'all' ) –

  AP interpolation: "all" integrates the full precision-recall curve; "r11" / "r40" sample the KITTI 11- / 40-point recall grids.

Returns:

• Dict[str, float] –

  A dict {"AP/<class>": ap, ..., "mAP": mean} (the mean is over iou_per_class).

Mask of boxes whose BEV center distance from the sensor origin is strictly below their class range.

Parameters:

• boxes (Tensor) –

  Boxes \((N, 7)\) or \((N, 9)\) of \((c_x, c_y, c_z, d_x, d_y, d_z, \theta[, v_x, v_y])\).

• labels (Tensor) –

  Per-box class index into ranges, shape \((N,)\).

• ranges (Sequence[float]) –

  Maximum BEV range per class index, in the coordinate unit.

Returns:

• Tensor –

  Boolean keep mask of shape \((N,)\).

>>> boxes = torch.tensor([[3.0, 4, 0, 4, 2, 1.5, 0], [0, 41, 0, 0.5, 0.5, 1, 0]])
>>> filter_boxes_by_range(boxes, torch.tensor([0, 1]), ranges=[50.0, 40.0])
tensor([ True, False])

The nuScenes detection metrics: per-class AP, mAP, the five TP errors and the NDS.

Follows the official protocol of the nuScenes benchmark (nuScenes: A Multimodal Dataset for Autonomous Driving). Predictions are matched per sample and class by BEV center distance: in descending score order each prediction greedily takes the closest still-unmatched ground-truth box strictly below the threshold. AP interpolates precision at 101 recall points \(0.00, 0.01, \ldots, 1.00\), drops recalls up to min_recall, subtracts min_precision, clamps at \(0\), averages and rescales by the remaining precision span; mAP averages over class_names and dist_thresholds. The TP errors ATE (BEV center distance), ASE ($1 - $ IoU of center- and yaw-aligned boxes), AOE (absolute yaw difference, modulo \(\pi\) for barrier), AVE (L2 xy-velocity difference) and AAE ($1 - $ attribute accuracy) average the cumulative-mean error curve of the tp_threshold matches from min_recall to the highest achieved recall; a class without matches scores the full error of \(1\). The officially excluded pairs (traffic_cone: AOE/AVE/AAE, barrier: AVE/AAE) are left out of the per-metric means, and \(\text{NDS} = (5 \cdot \text{mAP} + \sum_\text{tp} (1 - \min(1, \text{err}))) / 10\).

Boxes are filtered before scoring: each sample keeps its max_boxes_per_sample highest-scoring predictions, boxes farther from the sensor origin (BEV) than their class range are dropped on both sides, and ground-truth boxes with gt_num_points == 0 are removed. When velocity columns or attributes are absent (on either side), AVE / AAE fall back to the full penalty of \(1.0\) per class.

Parameters:

• pred_boxes (Tensor) –

  Predicted boxes \((M, 7)\) of \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\), or \((M, 9)\) with \((v_x, v_y)\) velocity columns appended.

• pred_scores (Tensor) –

  Per-box confidence, shape \((M,)\).

• pred_labels (Tensor) –

  Per-box class index into class_names, shape \((M,)\).

• pred_batch (Tensor) –

  Per-box sample index, shape \((M,)\).

• gt_boxes (Tensor) –

  Ground-truth boxes \((K, 7)\) or \((K, 9)\), like pred_boxes.

• gt_labels (Tensor) –

  Per-box class index into class_names, shape \((K,)\).

• gt_batch (Tensor) –

  Per-box sample index, shape \((K,)\).

• class_names (Sequence[str]) –

  Class name per label index; barrier and traffic_cone get their official special handling by name.

• gt_num_points (Optional[Tensor], default: None ) –

  Optional per-box point count, shape \((K,)\); boxes with exactly \(0\) points are removed (unknown counts of \(-1\) are kept).

• pred_attributes (Optional[Tensor], default: None ) –

  Optional per-box attribute id, shape \((M,)\). Without it AAE is \(1.0\).

• gt_attributes (Optional[Tensor], default: None ) –

  Optional per-box attribute id, shape \((K,)\); a negative id marks a box without an attribute, which is skipped in the AAE mean. Without it AAE is \(1.0\).

• class_ranges (Optional[Mapping[str, float]], default: None ) –

  Maximum BEV evaluation range per class name; defaults to the official ranges (50 m car/truck/bus/trailer/construction_vehicle, 40 m pedestrian/motorcycle/bicycle, 30 m traffic_cone/barrier). A name missing from the mapping is not range-filtered.

• dist_thresholds (Sequence[float], default: (0.5, 1.0, 2.0, 4.0) ) –

  Matching thresholds in meters the AP is averaged over.

• tp_threshold (float, default: 2.0 ) –

  Matching threshold in meters of the TP-error metrics.

• max_boxes_per_sample (int, default: 500 ) –

  Per-sample cap on scored predictions (highest scores kept).

• min_recall (float, default: 0.1 ) –

  Recall up to which the AP and TP-error curves are clipped.

• min_precision (float, default: 0.1 ) –

  Precision subtracted before the AP mean.

Returns:

• Dict[str, float] –

  A flat dict with AP/<class> (averaged over dist_thresholds), mAP, mATE, mASE, mAOE,

• Dict[str, float] –

  mAVE, mAAE and NDS.

>>> zero = torch.tensor([0])
>>> pred_boxes = torch.tensor([[0.25, 0.0, 0.0, 4.0, 2.0, 1.5, 0.0]])
>>> gt_boxes = torch.tensor([[0.0, 0.0, 0.0, 4.0, 2.0, 1.5, 0.0]])
>>> metrics = nuscenes_detection_metrics(
...     pred_boxes, torch.tensor([0.9]), zero, zero, gt_boxes, zero, zero, class_names=["car"]
... )
>>> f"{metrics['AP/car']:.2f} {metrics['mATE']:.2f} {metrics['NDS']:.3f}"
'1.00 0.25 0.775'

Derive per-box nuScenes attribute ids from predicted velocities (the standard speed heuristic).

A box moving faster than speed_threshold (BEV speed, m/s) gets its class's moving attribute, a slower box the parked / stopped / standing default; barrier and traffic_cone carry no attribute (id \(-1\)). The returned ids index the official 8-entry attribute table (attribute.json order), the id space of the pred_attributes / gt_attributes arguments of nuscenes_detection_metrics.

Parameters:

• labels (Tensor) –

  Per-box class index into class_names, shape \((M,)\) long.

• velocity (Tensor) –

  Per-box BEV velocity \((v_x, v_y)\), shape \((M, 2)\).

• class_names (Sequence[str]) –

  Class name per label index (the official 10 detection class names).

• speed_threshold (float, default: 1.0 ) –

  BEV speed in m/s above which a box counts as moving.

Returns:

• Tensor –

  Per-box attribute id, shape \((M,)\) long, \(-1\) for classes without attributes.

>>> labels = torch.tensor([0, 0, 1])
>>> velocity = torch.tensor([[3.0, 0.0], [0.5, 0.0], [2.0, 0.0]])
>>> nuscenes_velocity_attributes(labels, velocity, class_names=("car", "barrier")).tolist()
[0, 2, -1]

Reduce one scene's instance predictions to the compact match record scored by instance_average_precision.

Predicted masks are dense \((K, N)\) booleans: a few hundred masks over a \(\sim 50\text{k}\)-point scene is only tens of MB and intersections reduce to bincounts, while index lists would be ragged and no smaller. The returned record holds per-instance counts and pairwise intersections only, so nothing mask-sized outlives the call and a whole validation split can be accumulated scene by scene.

Ground truth instances are the unique gt_instance ids among points with a non-negative id and a valid semantic label; points whose gt_label equals ignore_index are void, and predictions overlapping them are excused accordingly during scoring. Each instance must carry a single semantic label. Intersections are recorded for same-class (prediction, instance) pairs only.

Parameters:

• pred_masks (Tensor) –

  Per-instance point masks, shape \((K, N)\) bool.

• pred_labels (Tensor) –

  Per-instance class indices, shape \((K,)\).

• pred_scores (Tensor) –

  Per-instance confidences, shape \((K,)\).

• gt_instance (Tensor) –

  Per-point ground-truth instance ids, shape \((N,)\); negative marks no instance.

• gt_label (Tensor) –

  Per-point semantic labels in the instance-class space, shape \((N,)\).

• ignore_index (int, default: -1 ) –

  Semantic label marking void points.

Returns:

• Dict[str, Tensor] –

  A dict of CPU tensors: pred_labels, pred_scores, pred_counts, pred_void (per prediction),

• Dict[str, Tensor] –

  gt_labels, gt_counts (per ground-truth instance, ordered by ascending id), and the nonzero

• Dict[str, Tensor] –

  same-class intersections as pair_pred, pair_gt, pair_inter.

>>> masks = torch.tensor([[True, True, True, True]])
>>> match = instance_matches(
...     masks, torch.tensor([0]), torch.tensor([0.9]), torch.tensor([0, 0, 1, -1]), torch.tensor([0, 0, 0, -1])
... )
>>> match["gt_counts"].tolist(), match["pair_inter"].tolist(), match["pred_void"].tolist()
([2, 1], [2, 1], [1])

Point-mask instance-segmentation AP over per-scene instance_matches records.

Follows the standard indoor instance-segmentation protocol (the ScanNet benchmark): per class and IoU threshold, ground-truth instances greedily consume overlapping predicted masks above the threshold, duplicates on a matched instance count as false positives with the lower score, and an unmatched prediction whose void / small-instance point fraction exceeds the threshold is excused. The AP integrates the score-swept precision-recall curve with centered recall steps. mAP averages per-class APs over the thresholds \(0.5, 0.55, \ldots, 0.9\); mAP@0.5 and mAP@0.25 report the single-threshold values. Classes without any ground-truth instance are excluded from the means and get no AP/<class> entry.

Parameters:

• matches (Sequence[Mapping[str, Tensor]]) –

  One instance_matches record per scene.

• num_classes (int) –

  Number of instance classes.

• class_names (Optional[Sequence[str]], default: None ) –

  Optional names for the AP/<class> keys; falls back to the class index.

• min_points (int, default: 100 ) –

  Minimum point count for a prediction or ground-truth instance to be scored; smaller ground-truth instances count as ignore regions.

Returns:

• Dict[str, float] –

  A dict {"AP/<class>": ap, ..., "mAP": ..., "mAP@0.5": ..., "mAP@0.25": ...} where each

• Dict[str, float] –

  AP/<class> is that class's AP averaged over the \(0.5{:}0.05{:}0.9\) thresholds.

>>> masks = torch.tensor([[True, True, True, False], [False, False, False, True]])
>>> match = instance_matches(
...     masks,
...     torch.tensor([0, 1]),
...     torch.tensor([0.9, 0.8]),
...     torch.tensor([0, 0, 0, 1]),
...     torch.tensor([0, 0, 0, 1]),
... )
>>> out = instance_average_precision([match], num_classes=2, min_points=1)
>>> out["mAP"], out["mAP@0.5"], out["mAP@0.25"]
(1.0, 1.0, 1.0)