detection

3D box detection metrics: per-batch box matches and their average precision.

Classes:

BoxMatches –

One batch of box predictions reduced against its ground truth (the output of box_matches).

Functions:

box_matches –

Match one batch of box predictions to its ground truth, the record scored by box_average_precision.
box_average_precision –

3D detection average precision (AP) of matched box predictions.

BoxMatches [source] ¶

Bases: TypedDict

One batch of box predictions reduced against its ground truth (the output of box_matches).

Holds what scoring needs and nothing box-sized: \(P\) predictions are left after the prediction ignore_mask, \(G\) ground-truth boxes after the target ignore_mask. A prediction with no same-class box (or ignore region) in its sample carries \(-1\) in the matching fields.

Attributes:

pred_scores (Tensor) –

Per-prediction confidence score, shape \((P,)\).
pred_labels (Tensor) –

Per-prediction class, shape \((P,)\).
pred_iou (Tensor) –

Largest IoU with a same-class ground-truth box of the same sample, shape \((P,)\).
pred_gt (Tensor) –

Row of that box in gt_labels, shape \((P,)\).
pred_ignore_iou (Tensor) –

Largest IoU with a same-class ignore region of the same sample, shape \((P,)\).
gt_labels (Tensor) –

Per-box class of the ground truth, shape \((G,)\).

box_matches [source] ¶

box_matches(
    preds: Detection3D, target: Boxes3D
) -> BoxMatches

Match one batch of box predictions to its ground truth, the record scored by box_average_precision.

Which ground-truth box a prediction overlaps most, and by how much, depends on neither the IoU threshold nor on what the other predictions matched. The oriented IoU is therefore paid here, once per batch and on the device of the inputs, and nothing box-sized outlives the call: a whole validation split accumulates as a few flat tensors per batch, so keep one record per batch in a list and score the list.

Target boxes flagged by ignore_mask (see Boxes3D) are ignore regions rather than ground truth; their labels entry names the class they excuse. Predictions flagged by their own ignore_mask are left out of the record, so they can neither match a box nor count as a false positive (the KITTI min-height rule).

Parameters:

preds (Detection3D) –

Packed predictions (one decode output), {"boxes", "scores", "labels", "batch"} with an optional ignore_mask.
target (Boxes3D) –

Packed ground truth aligned to preds, {"boxes", "labels", "batch"} with an optional ignore_mask.

Returns:

BoxMatches –

The BoxMatches record of the batch, as CPU tensors.

Shape

preds["boxes"]: \((P, 7)\)
target["boxes"]: \((G, 7)\)
output: six tensors of shape \((P',)\) or \((G',)\), the predictions and boxes left after the ignore masks

Example

>>> boxes = torch.tensor([[0.0, 0.0, 0.0, 2.0, 2.0, 2.0, 0.0], [9.0, 9.0, 9.0, 1.0, 1.0, 1.0, 0.0]])
>>> index = torch.zeros(2, dtype=torch.long)
>>> preds = {"boxes": boxes, "scores": torch.tensor([0.9, 0.4]), "labels": index, "batch": index}
>>> match = box_matches(preds, {"boxes": boxes[:1], "labels": index[:1], "batch": index[:1]})
>>> match["pred_iou"].tolist(), match["pred_gt"].tolist()
([1.0, 0.0], [0, 0])
>>> box_average_precision([match], iou_threshold=0.5)
1.0

box_average_precision [source] ¶

box_average_precision(
    matches: Sequence[BoxMatches],
    *,
    iou_threshold: Union[float, Mapping[int, float]] = ...,
    average: Literal["macro"] = ...,
    num_classes: Optional[int] = ...,
    class_names: Optional[Sequence[str]] = ...,
    interpolation: Interpolation = ...,
) -> float

box_average_precision(
    matches: Sequence[BoxMatches],
    *,
    iou_threshold: Union[float, Mapping[int, float]] = ...,
    average: Literal["none"],
    num_classes: Optional[int] = ...,
    class_names: None = ...,
    interpolation: Interpolation = ...,
) -> Tensor

box_average_precision(
    matches: Sequence[BoxMatches],
    *,
    iou_threshold: Union[float, Mapping[int, float]] = ...,
    average: Literal["none"],
    num_classes: Optional[int] = ...,
    class_names: Sequence[str],
    interpolation: Interpolation = ...,
) -> Dict[str, float]

box_average_precision(
    matches: Sequence[BoxMatches],
    *,
    iou_threshold: Union[float, Mapping[int, float]] = 0.5,
    average: Literal["macro", "none"] = "macro",
    num_classes: Optional[int] = None,
    class_names: Optional[Sequence[str]] = None,
    interpolation: Interpolation = "all",
) -> Union[float, Tensor, Dict[str, float]]

3D detection average precision (AP) of matched box predictions.

Dataset- and model-agnostic: box_matches reduces any detector's packed (boxes, scores, labels, batch) output to the same record, one per batch, and the records of a whole split are scored together. Within a class, predictions claim their best ground-truth box in descending score order: the first one above the IoU threshold is a true positive, the others are false positives, and the AP is the area under the resulting precision-recall curve.

With one iou_threshold for every class, the classes that have ground truth are scored. With one per class index, exactly those classes are scored, and one without ground truth scores \(0\) (its predictions are all false positives).

Parameters:

matches (Sequence[BoxMatches]) –

The box_matches record of every evaluated batch.
iou_threshold (Union[float, Mapping[int, float]], default: 0.5 ) –

IoU a match must exceed: one value for every class (e.g. 0.25), or one per class index (e.g. KITTI's {0: 0.7, 1: 0.5, 2: 0.5}).
average (Literal['macro', 'none'], default: 'macro' ) –

"macro" returns the mean AP (mAP) over the scored classes; "none" returns the per-class AP.
num_classes (Optional[int], default: None ) –

Number of classes, i.e. the length of the average="none" output; defaults to the number of class_names, else to the largest class index met plus one.
class_names (Optional[Sequence[str]], default: None ) –

Name of each class index; with average="none" the per-class AP comes back as a {name: ap} dict instead of a tensor.
interpolation (Interpolation, default: 'all' ) –

AP interpolation: "all" integrates the full precision-recall curve; "r11" / "r40" sample the KITTI 11- / 40-point recall grids.

Returns:

Union[float, Tensor, Dict[str, float]] –

The mAP as a float with average="macro" (\(0\) when no class is scored), or the per-class AP, shape
Union[float, Tensor, Dict[str, float]] –

\((C,)\) float64, with average="none" (a {name: ap} dict when class_names is given), holding NaN for
Union[float, Tensor, Dict[str, float]] –

the classes that are not scored.

Shape

output: scalar, or \((C,)\) with average="none"

Example

>>> boxes = torch.tensor([[0.0, 0.0, 0.0, 2.0, 2.0, 2.0, 0.0], [9.0, 9.0, 9.0, 1.0, 1.0, 1.0, 0.0]])
>>> labels, batch = torch.tensor([0, 1]), torch.tensor([0, 0])
>>> preds = {"boxes": boxes, "scores": torch.tensor([0.9, 0.4]), "labels": labels, "batch": batch}
>>> matches = [box_matches(preds, {"boxes": boxes[:1], "labels": labels[:1], "batch": batch[:1]})]
>>> box_average_precision(matches, iou_threshold=0.25)
1.0
>>> box_average_precision(matches, iou_threshold={0: 0.7, 1: 0.5}, average="none")
tensor([1., 0.], dtype=torch.float64)
>>> box_average_precision(matches, average="none", class_names=["Car", "Cyclist"])
{'Car': 1.0, 'Cyclist': nan}